CN113225396A - Hot spot data packet distribution method and device, electronic equipment and medium - Google Patents

Hot spot data packet distribution method and device, electronic equipment and medium Download PDF

Info

Publication number
CN113225396A
CN113225396A CN202110487441.9A CN202110487441A CN113225396A CN 113225396 A CN113225396 A CN 113225396A CN 202110487441 A CN202110487441 A CN 202110487441A CN 113225396 A CN113225396 A CN 113225396A
Authority
CN
China
Prior art keywords
data packet
inventory
hot spot
deduction
storage cluster
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110487441.9A
Other languages
Chinese (zh)
Other versions
CN113225396B (en
Inventor
陈元元
金伟潼
邹同海
杨阿琴
陈阳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Tencent Domain Computer Network Co Ltd
Original Assignee
Shenzhen Tencent Domain Computer Network Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Tencent Domain Computer Network Co Ltd filed Critical Shenzhen Tencent Domain Computer Network Co Ltd
Priority to CN202110487441.9A priority Critical patent/CN113225396B/en
Publication of CN113225396A publication Critical patent/CN113225396A/en
Application granted granted Critical
Publication of CN113225396B publication Critical patent/CN113225396B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
    • H04L67/5682—Policies or rules for updating, deleting or replacing the stored data

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the application discloses a hot spot data packet shunting method, a hot spot data packet shunting device, electronic equipment and a medium. The method comprises the following steps: determining hot spot data packets from the plurality of data packets, determining a target pulling quantity according to the consumption speed of the hot spot data packets, and storing the target pulling quantity inventory of the hot spot data packets pulled from the storage cluster into a cache region. By adopting the embodiment of the application, the load of the storage cluster is reduced, and the operation and maintenance cost is reduced. Optionally, the data packets related to the present application may be stored in a block chain, and then the inventory of the hot-spot data packets may be pulled from the block chain and stored in a cache region.

Description

Hot spot data packet distribution method and device, electronic equipment and medium
Technical Field
The present application relates to the field of computer technologies, and in particular, to a hot spot data packet offloading method and apparatus, an electronic device, and a medium.
Background
With the development of computer technology, it is often easy to face a high concurrency phenomenon, for example, when a large number of requests access a certain data packet, a hot data packet occurs, which further easily causes a high load of a storage cluster and a service crash. At present, in order to reduce the load of a storage cluster and prevent service collapse and the like when a hot spot data packet occurs, a hot spot data packet isolation mode is usually adopted to reduce the influence. However, the inventor finds in practice that reducing the influence of the hot spot data packets by isolating the hot spot data packets requires consuming more human resources and has higher operation and maintenance costs.
Disclosure of Invention
The embodiment of the application provides a hot spot data packet shunting method, a hot spot data packet shunting device, electronic equipment and a medium, which are beneficial to reducing the load of a storage cluster and reducing the operation and maintenance cost.
In one aspect, an embodiment of the present application provides a hot spot data packet offloading method, including:
determining a hot spot data packet from a plurality of data packets;
determining a target pulling quantity according to the consumption speed of the hotspot data packets, wherein the consumption speed is used for indicating the deduction speed of the corresponding inventory of each data packet;
and pulling the target pulling amount of the inventory of the hot spot data packets from the storage cluster, and storing the pulled inventory in a cache region.
On the other hand, an embodiment of the present application provides a hot spot data packet offloading device, where the device includes:
the hot spot data packet determining module is used for determining a hot spot data packet from a plurality of data packets;
the processing module is used for determining a target pulling quantity according to the consumption speed of the hotspot data packets, wherein the consumption speed is used for indicating the deduction speed of the corresponding inventory of each data packet;
the processing module is further configured to pull the inventory of the target pull quantity of the hotspot data packets from the storage cluster, and store the pulled inventory into a cache region.
In another aspect, an embodiment of the present application provides an electronic device, where the electronic device includes a processor and a storage device, where the processor is connected to the storage device, where the storage device is configured to store computer program instructions, and the processor is configured to perform the following steps:
determining a hot spot data packet from a plurality of data packets;
determining a target pulling quantity according to the consumption speed of the hotspot data packets, wherein the consumption speed is used for indicating the deduction speed of the corresponding inventory of each data packet;
and pulling the target pulling amount of the inventory of the hot spot data packets from the storage cluster, and storing the pulled inventory in a cache region.
In yet another aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer program instructions are stored, and when executed by a processor, the computer program instructions are configured to perform the following steps:
determining a hot spot data packet from a plurality of data packets;
determining a target pulling quantity according to the consumption speed of the hotspot data packets, wherein the consumption speed is used for indicating the deduction speed of the corresponding inventory of each data packet;
and pulling the target pulling amount of the inventory of the hot spot data packets from the storage cluster, and storing the pulled inventory in a cache region.
According to the embodiment of the application, the hot spot data packets can be determined from the plurality of data packets, the target pulling quantity is determined according to the consumption speed of the hot spot data packets, and the inventory of the target pulling quantity for pulling the hot spot data packets from the storage cluster is stored in the cache region, so that the hot spot data packets are shunted, the load of the storage cluster can be reduced, and the operation and maintenance cost is reduced.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a diagram illustrating an effect of access request times for an activity according to an embodiment of the present application;
fig. 2 is a schematic diagram of a hot spot data packet offloading method according to an embodiment of the present disclosure;
fig. 3 is a schematic flowchart of a hot spot data packet offloading method according to an embodiment of the present disclosure;
fig. 4 is a schematic diagram illustrating an effect of a data packet storage method according to an embodiment of the present application;
fig. 5 is a schematic diagram illustrating an effect of a data packet configuration interface according to an embodiment of the present application;
fig. 6 is a schematic flowchart of another hot spot data packet offloading method according to an embodiment of the present disclosure;
fig. 7 is a schematic application flow diagram of a hot spot data packet offloading method according to an embodiment of the present disclosure;
fig. 8 is a schematic application flow diagram of another hot spot data packet offloading method according to an embodiment of the present application;
FIG. 9 is a schematic illustration of an activity effect provided by an embodiment of the present application;
fig. 10 is a schematic application flow chart of another hot spot data packet offloading method according to an embodiment of the present application;
fig. 11 is a schematic structural diagram of a hot spot packet offloading device according to an embodiment of the present disclosure;
fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.
The technical scheme of the application can be applied to electronic equipment, the electronic equipment can be a server, for example, an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, and also can be a cloud server providing basic cloud computing services such as cloud service, a cloud database, cloud computing, cloud functions, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, big data and an artificial intelligence platform.
The embodiment of the application provides a hot spot data packet shunting scheme, which can determine a target pulling quantity according to characteristic information of a hot spot data packet, such as consumption speed, and store the inventory of the target pulling quantity of the hot spot data packet from a storage cluster into a cache region, so that when a data packet deduction instruction is received, the inventory of the hot spot data packet is deducted from the cache region, thereby realizing shunting of the hot spot data packet, and therefore, the load of the storage cluster can be reduced, and the operation and maintenance cost can be reduced.
A packet in the present application may refer to data stored in a key-value structure (key-value structure), and one key-value may be referred to as a packet. The data stored in the key-value structure generally has the characteristics of fast query speed, large data storage amount, high concurrency support and the like, the database stored in the key-value structure may be referred to as a key-value storage system, and the value corresponding to the key may be quickly queried through the key, such as Redis, Berkeley DB or MemcacheDB, and the like, without limitation herein. The data packet in the present application may also be data stored in a table structure, such as a hash table (hash table), and the data packet may also be data in other forms, which is not limited in the present application.
It is understood that the hotspot data packet may also be called a hot key, a hotspot packet or other names, and the application is not limited thereto.
The cache region in this application may refer to a cache region corresponding to the electronic device, such as a local storage region, where the cache region may be ehcache or hashmap, and the cache region may also be a shared memory (shared memory), which is not limited herein. The shared memory may be a large memory accessible by different Central Processing Units (CPUs) in a multiprocessor computer system.
The technical scheme of the application can also be applied to the field of block chains. The blockchain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism and an encryption algorithm. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product services layer, and an application services layer. For example, the storage cluster referred to in the application may be a blockchain, each node on the blockchain may be configured to store a plurality of data packets, and the electronic device may pull an inventory of hot-spot data packets from the blockchain to a cache area. For another example, the record of the inventory of the hotspot packet may also be recorded through the blockchain, that is, the time when the inventory of the hotspot packet is deducted, the amount of the deducted inventory, the user information for deducting the inventory of the hotspot packet, and the like are recorded through the blockchain, so as to query the consumption record of the hotspot packet.
In some scenarios, for example, in a large-scale activity, such as a second killing activity of a high-value commodity or a gift package, the gift package or the commodity a involved in the activity may correspond to the data package a, and if the number of access requests of the data package a is greater than a preset value of the number of access requests, the data package a may be determined as a hot data package, i.e., a hot key, generally, the access requests for the hot data package in the activity are concurrent and higher at an early stage of the activity, and the access amount in a middle and later stage of the activity gradually slows down. For example, referring to fig. 1, fig. 1 is a schematic diagram illustrating an effect of a number of access requests of an activity provided by an embodiment of the present application, where a position shown by 101 is a peak in the activity process, and it can be seen that the peak in the activity process is obvious, a large number of access requests are received in a short time, and a high load is easily caused on a storage cluster. Therefore, the embodiment of the application provides a scheme for pulling the inventory storage of the hot spot data packet from the storage cluster to the cache region, and the hot spot data packet can be cached in the local cache region to realize shunting. Therefore, when the hot spot data packet is deducted subsequently, the inventory of the hot spot data packet can be deducted from the cache region preferentially, so that the times of deducting the inventory of the hot spot data packet from the storage cluster can be reduced, the load of the storage cluster is reduced under the condition of not depending on caching or isolating the hot spot data packet, and the adverse effect of the hot spot data packet on the storage cluster can be reduced. Referring to fig. 2, fig. 2 is a schematic view of a hot spot data packet offloading method provided in an embodiment of the present application, in which an electronic device pulls an inventory of hot spot data packets from a storage cluster and stores the inventory in a cache region, and a data packet deduction instruction deducts the hot spot data packets from the cache region.
In a possible implementation manner, the hot spot data packet splitting scheme provided by the present application can be applied to a scene such as a game. In the process of operating the game, users are generally attracted by planning game activities, such as accepting gift bags from registered games or inviting unregistered users to accept gift bags, so as to attract new users, or obtaining gift bags by inviting users who are not online for a long time to log in the game again, so as to attract old users. When such activities are carried out, the gift bag for attracting the user can easily generate a large number of access requests, namely, the data package storing the gift bag can easily become a hot spot data package, and through the implementation of the scheme, the pressure of the hot spot data package on the storage cluster can be reduced, so that the optimization of the game platform is realized.
It is to be understood that the foregoing scenarios are only examples, and do not constitute a limitation on application scenarios of the technical solutions provided in the embodiments of the present application, and the technical solutions of the present application may also be applied to other scenarios. For example, as can be known by those skilled in the art, with the evolution of system architecture and the emergence of new service scenarios, the technical solution provided in the embodiments of the present application is also applicable to similar technical problems.
Based on the above description, an embodiment of the present application provides a hot spot data packet offloading method. Referring to fig. 3, the hot spot packet offloading method may include steps S301 to S303.
S301, determining a hotspot data packet from the plurality of data packets.
The plurality of data packets may be data packets stored in a storage cluster, where the storage cluster may be a conventional storage system, a distributed storage system, or a cloud storage system, and this is not limited herein. Optionally, the hotspot data packet may be a data packet whose storage cluster access speed meets a preset speed condition (e.g., exceeds a speed threshold), where the storage cluster access speed may be the number of access requests for the hotspot data packet received in unit time, and the unit time may be 1 second or 1 minute, which is not limited herein; the hot spot data packet may also be a data packet whose pressure value of the storage cluster satisfies a preset pressure condition (e.g., exceeds a pressure threshold), where the pressure value may be a value calculated according to the number of accesses in a unit time, and is not limited herein. That is to say, when the data packet meets the hot spot data packet determination condition, the data packet is determined as the hot spot data packet, and the data packet determination condition may be that the storage cluster access speed of the data packet meets a preset speed condition, or that the pressure value of the storage cluster meets a preset pressure condition, and the like, which is not limited herein.
For example, in one possible implementation, determining a hotspot packet from a plurality of packets may include the following steps: determining local access speeds of a plurality of data packets according to the access request information aiming at each data packet; acquiring weight information, and determining the access speed of the storage cluster corresponding to each data packet according to the local access speed and the weight information; and determining the data packet with the access speed of the storage cluster meeting the preset speed condition as a hot spot data packet.
The access request information may include information such as a purpose of the access request or a time of the access request, the access request may be a read request, a write request, a read-write request, or the like, and the time of the access request may be a time of the request received by the client or a time of the access request received by the electronic device, and is not limited herein. The local access speed may be used to indicate the number of access requests received by the electronic device per unit time, for example, the unit time is 1 second, and if the electronic device detects that the access request is received 5000 times within 1 second, the local access speed is 5000 times/second. The unit time may refer to an adjacent time period in which a time length before the local access speed of the plurality of packets is determined from the access request information for each packet is a unit time, for example, a time point in which the local access speed of the plurality of packets is determined from the access request information for each packet is t (0), the unit time may refer to t (-1) -t (0), the t (-1) is before t (0) and a time length between t (-1) and t (0) is a unit time, and the number of times of access requests received in the unit time is the local access speed; the unit time may also refer to a time period in which the number of access requests is the largest in a unit time within a time period of a preset time length before the local access speed of the plurality of packets is determined according to the access request information for each packet, for example, a time point at which the local access speed of the plurality of packets is determined according to the access request information for each packet is t (0), the time period of the preset time length is t (-10) -t (0), the t (-10) is before t (0) and the time length between t (-10) and t (0) is greater than or equal to the unit time, then it may be detected that t (-5) -t (-4) between t (-10) -t (0) is the time period in which the number of access requests is the largest, then t (-5) -t (-4) is the unit time, the number of access requests received in the unit time is the local access speed.
The weight information may be used to indicate a correspondence between a local access speed and a storage cluster access speed, the storage cluster access speed may be used to indicate the number of access requests received in a unit time by the storage cluster, and the correspondence between the local access speed and the storage cluster access speed may be a multiple relation or other mathematical relations, which is not limited herein. For example, in the hot spot packet offloading system shown in fig. 2, if a storage cluster and at least one electronic device, such as a server, exist, it may be obtained that a weight value of each server in the system is corresponding to W (the W corresponding to each electronic device may be the same or different), that is, a weight value of a remote storage is locally accessed, for example, the weight value may be obtained by accessing a locally scheduled Agent program through an Agent program (e.g., Proxy); it is also possible to obtain a composite weight value of the storage cluster as W _ a, for example, communicating with an Agent to obtain a composite of all weights stored remotely. The Agent program (Proxy) may be configured to receive an access request from a user and send the request to a server, and the Agent program may be a client program of a user interface. Then, it may be obtained that an occupation ratio (i.e., an amplification value of a remote storage node) corresponding to each electronic device is W _ a/W (assumed to be denoted as K, that is, K is W _ a/W), and if the relationship between the local access speed and the storage cluster access speed (i.e., the weight information) is that the storage cluster access speed is the local access speed, K, it is detected that the number of access requests received in the electronic device for the packet a in unit time is 500, that is, the local access speed for the packet a is 500 times/second, then the storage cluster access speed may be 500 times/second.
For example, optionally, the preset speed condition may be that the storage cluster access speed for the packet is greater than or equal to a speed threshold, that is, when the storage cluster access speed of the packet is greater than or equal to the speed threshold, the packet is determined as a hotspot packet. For example, a data packet a, a data packet B, and a data packet C exist in the storage cluster, the access speed of the storage cluster corresponding to each data packet is 5000 times/second, 6000 times/second, and 7000 times/second, respectively, and if the speed threshold is 5500 times/second, the data packet B and the data packet C may be determined as a hot spot data packet. Optionally, the preset speed condition may also be that a hotspot tag is detected to exist in the data packet, for example, if it is determined by experience that the data packet may become a hotspot data packet, the data packet is hotspot tagged, and when it is detected that the hotspot tag exists in the data packet, the data packet is determined as a hotspot data packet.
As another example, optionally, the data packets with the top N local access speeds ordered from high to low may be counted by selecting a Least Recently Used (LRU) method, where N is a positive integer. The LRU is a commonly used page replacement algorithm that selects the least recently used page to be evicted. For example, the method assigns an access field to each data packet, the access field is used for recording the time t (t >0) that the data packet has elapsed since the data packet was accessed last time, when a data packet needs to be eliminated, the data packet with the maximum t value in the existing data packets, namely the least recently used page, is selected for elimination, so as to obtain the first N data packets that are not eliminated, and the local access speeds of the first N data packets correspond to { S1, S2, S3 · · SN }, so that the local access speeds of the first N data packets only need to be counted, the calculation amount is reduced, and the efficiency is improved. And detecting the data packets of which the access speeds of the storage clusters corresponding to the first N data packets meet the preset speed condition, and determining the data packets meeting the preset speed condition as the hot spot data packets.
In a possible implementation, determining a hotspot packet from the plurality of packets may further include: determining local pressure values of a plurality of data packets according to the access request information aiming at each data packet; acquiring weight information, and determining a storage cluster pressure value corresponding to each data packet according to the local pressure value and the weight information; and determining the data packet with the storage cluster pressure value meeting the preset pressure condition as a hot spot data packet.
The local pressure value is used to indicate the pressure or influence of the data packet on the electronic device, and may be calculated according to the local access speed of each data packet, for example. Optionally, the local pressure value and the local access speed may be in a proportional relationship, and the larger the local access speed is, the larger the local pressure value is. Further optionally, the influence factor of each data packet may be counted by selecting an LRU method, where the influence factor may be used to determine a relationship between the number of access requests of each data packet and a pressure value of the corresponding data packet to the electronic device, in other words, the influence of the number of access requests of each data packet on the electronic device may be different, and then a local pressure value of the corresponding data packet may be calculated by the influence factor and the number of access requests, where the local pressure value is sorted from high to low into the first N data packets, where N is a positive integer, and the local pressure value of the first N data packets is corresponding to { T1, T2, T3 · · TN }, so that only the local pressure values of the first N data packets need to be counted, thereby reducing the calculation amount and improving the efficiency.
The weight information may be used to indicate a corresponding relationship between a pressure value and a storage cluster pressure value, and the storage cluster access pressure value may be used to indicate a pressure or an influence of a data packet on a storage cluster. Optionally, the corresponding relationship between the local pressure value and the storage cluster pressure value may be a multiple relationship, or may be other mathematical relationships, which is not limited herein. For example, in the hot-spot packet offloading system shown in fig. 2, if there is a storage cluster and at least one server, it may be obtained that a weight value of each server in the system is corresponding to W (the W corresponding to each electronic device may be the same or different), that is, a weight value of a remote storage is locally accessed, for example, the weight value may be obtained by accessing a locally scheduled Agent program through an Agent program (e.g., Proxy); it is also possible to obtain a composite weight value of the storage cluster as W _ a, for example, communicating with an Agent to obtain a composite of all weights stored remotely. The Agent program (Proxy) is used for receiving the access request of the user and sending the access request to the target position indicated by the access request, and the Agent program can be a client program of the user interface. Then, it may be obtained that the occupation ratio (i.e., the amplification value of the remote storage node) corresponding to each electronic device is W _ a/W (assumed to be K, that is, K is W _ a/W) according to the weight value and the comprehensive weight value, and if the correspondence relationship (that is, the weight information) between the local pressure value and the storage cluster pressure value is the storage cluster pressure value which is local pressure value K, and it is detected that the local pressure value for the packet a in the server is TN, then it may be obtained that the storage cluster pressure value is TN K.
For example, optionally, the preset pressure condition may be that the storage cluster pressure value for the packet is greater than or equal to a pressure threshold, and when the storage cluster pressure value of the packet is greater than or equal to the pressure threshold, the packet is determined as a hot spot packet. For example, if packet a, packet B, and packet C exist in the storage cluster, the storage cluster pressure value corresponding to each packet is X, Y and Z, the pressure thresholds P, X, Y, Z, and P are all greater than 0, and P is greater than X, Y and less than Z, then packet C may be determined as a hot spot packet. Optionally, the preset pressure condition may also be that a hotspot tag is detected to exist in the data packet, for example, if it is determined by the related personnel according to experience that the data packet may become a hotspot data packet, the data packet is hotspot tagged, and when it is detected that the hotspot tag exists in the data packet, the data packet is determined as a hotspot data packet.
In a possible implementation manner, when the storage cluster stores the plurality of data packets, the fragmentation processing may be performed on the storage cluster, so as to store the plurality of data packets into a plurality of nodes in the storage cluster, where the plurality of nodes may refer to a plurality of storage devices in the storage cluster, and then each node in the storage cluster may correspond to one fragment, where one or more data packets may be stored in the one fragment. Therefore, the load aiming at the storage cluster can be dispersed to a plurality of sub-slices, and the adverse effect caused by hot spot data packets is reduced. Optionally, after the fragmentation processing is performed on the multiple data packets, a corresponding relationship between each fragment and the data packet may be recorded by constructing a route, and when an access request is detected, a fragment position stored in the data packet to which the access request is directed is determined for the access request through the route. After the storage cluster pressure value is determined, the fragmentation pressure value of the data packet for each fragment can be determined through the information recorded by the route, so that the hot spot data packet is determined according to the fragmentation pressure value of the data packet. For example, a packet whose sliced pressure value satisfies a preset pressure condition may be determined as a hot packet.
Optionally, when the storage cluster stores the hotspot data packet, the hotspot data packet may be stored in a plurality of segments of the storage cluster, so as to reduce a pressure value of the hotspot data packet on the storage cluster, in other words, the inventory of the hotspot data packet is stored on the plurality of segments, and the plurality of segments share access requests of the hotspot data packet, thereby reducing adverse effects of the hotspot data packet on the storage cluster. Referring to fig. 4, fig. 4 is a schematic diagram illustrating an effect of a data packet storage method according to an embodiment of the present application, in which an inventory of a data packet may be distributed to multiple segments in a storage cluster.
In one possible implementation, when receiving an access request for a data packet, a load balancing component (e.g., CL5) is typically used to enable the access request for the data packet to be evenly distributed to each shard in the storage cluster or to each electronic device, so as to avoid that a shard or an electronic device is overloaded. Optionally, the corresponding relationship between the local access speed and the storage cluster access speed may be calculated through the weight information, so that the access requests for the hotspot data packets are distributed to the segments of the storage cluster or the electronic devices in a balanced manner through the service discovery component. The service discovery component can be used for executing the operation corresponding to the access request according to the relevant information of the access request.
In a possible implementation manner, the inventory amount and the related conditions corresponding to each data packet of the plurality of data packets can be configured through a data packet configuration interface, so that a user can deduct each data packet according to data in the data packet configuration interface. For example, referring to fig. 5, fig. 5 is a schematic diagram illustrating an effect of a data package configuration interface according to an embodiment of the present application, where a data package may be a gift package, and in the data package configuration interface, a product name, an item type, a single-time obtaining quantity, an item unit price, a probability, or gift package quantity information of the data package may be configured, which is not limited herein. Optionally, the data packet configuration interface may be applied to an e-commerce or game marketing campaign to configure a commodity or a gift bag appearing in the campaign, so as to implement a back flow or a refresh through the commodity or the gift bag. In some scenarios, when the configuration is performed through the data packet configuration interface, the stock quantity (i.e., the quantity of the gift package limit) and the single-time obtaining quantity (i.e., the quantity deducted by the data packet deduction instruction of one time during the activity of the user) of the data packet are generally required to be configured, generally, if the stock quantity of the configured data packet is small and is less than the demand, the unit price of the corresponding item is higher, such as a rare item or a rare item, and if the stock quantity of the configured data packet is larger and is greater than the demand, the unit price of the corresponding item is relatively low; in some scenarios, when configuring through the packet configuration interface, only the stock quantity of the packets is generally configured for a part of the packets for the pull back stream, and information such as the quantity obtained at a time is not limited.
S302, determining the target pulling quantity according to the consumption speed of the hot spot data packets.
The consumption rate is used to indicate the deduction rate of the corresponding stock of each data packet, in other words, the reduced stock quantity of the data packet in unit time. The target pull number is used to indicate an inventory number of data packets that the electronic device pulls from the storage cluster.
Optionally, the consumption speed may be calculated or predicted according to a packet deduction instruction for the hot spot packet, the consumption speed may also be calculated or predicted according to historical access request information of the hot spot packet, and the consumption speed may also be predicted according to experience of a relevant person, so as to set the consumption speed of the packet. Further alternatively, the consumption rate may be a deduction inventory amount in a certain unit time, or may be an average deduction inventory amount in a plurality of unit times, and the like, and the present application is not limited thereto. Therefore, the target pulling quantity can be determined according to the actual consumption condition, namely the target pulling quantity is pulled according to the requirement, the quantity stored in the cache region can be consumed in a short time, and the storage space of the cache region is prevented from being occupied by the hot spot data packet for a long time.
For example, in one possible implementation, the consumption rate may be predicted based on a packet subtraction instruction for the hotspot packet, the packet subtraction instruction indicating an inventory of subtracted packets. Specifically, if N times of data packet deduction instructions are received in unit time and the N times of data packet deduction instructions indicate that M stocks of the hotspot data packets are deducted, the consumption speed of the hotspot data packets is M/s. The unit time refers to an adjacent time period in which the time length before the consumption speed of the hot spot data packet is obtained is the unit time, for example, the time point of obtaining the consumption speed of the hot spot data packet is t (0), the unit time may refer to t (-1) -t (0), the t (-1) is before the t (0) and the time length between t (-1) and t (0) is the unit time, and the data packet deduction instruction received between t (-1) and t (0) indicates that the stock of the deducted hot spot data packet is the corresponding consumption speed; the unit time may also be a time period in which the packet subtraction instruction indicates the most amount of subtracted inventory in a unit time within a time period of a preset length before the consumption speed of the hot spot packet is obtained, for example, the time point of obtaining the consumption speed of the hot spot packet is t (0), the time period of the preset time length is t (-10) -t (0), the t (-10) is before the t (0), and the time length between the t (-10) and the t (0) is greater than or equal to the unit time, it may be detected that t (-5) -t (-4) between t (-10) -t (0) is the time period in which the packet subtraction instruction indicates the most amount of subtracted inventory, t (-5) -t (-4) is the unit time, and the packet subtraction instruction received at t (-5) -t (-4) indicates that the subtracted inventory of the hot spot packet is a pair The rate of consumption should be.
For another example, in one possible implementation, the consumption rate may also be predicted based on historical access request information for the hotspot packets. The historical access request information may be access request information in a time period of a preset time length before the consumption rate is obtained, for example, the number of access requests received within 10 seconds before the consumption rate is obtained, or the access rate of the storage cluster of the hotspot packet, which is not limited herein. For example, a mathematical relationship between the historical access request information and the consumption speed may be determined according to the historical access request information, so that after the historical access request information is obtained, the consumption speed of the hotspot data packet is determined according to the mathematical relationship. For example, the historical access request information indicates that the number of access requests received within 1 second before the consumption speed is obtained is Y, and if the mathematical relationship between the historical access request information and the consumption speed can be every H access requests, 1 inventory of hot spot data packets can be correspondingly deducted, so that the consumption speed of the hot spot data packet can be obtained as Y/H, where Y is a positive integer and H is a positive number. For another example, after the local access speed of the hotspot data packet is obtained, the storage cluster access speed of the hotspot data packet is calculated according to the weight information, where the storage cluster access speed is U times/second, and if the mathematical relationship between the historical access request information and the consumption speed may be that the storage cluster access speed is multiplied by P, where P is an integer, the consumption speed of the hotspot data packet may be obtained as U times/second.
As another example, in one possible implementation, the consumption rate may also be predicted empirically by the relevant person. For example, when a relevant person takes a large-scale activity, the data packet related to a certain commodity may be a hot spot data packet, and according to the past activity experience, M goods can be predicted to be sold in unit time when the commodity is sold, that is, the number of inventory deducted in unit time is M, and the consumption speed of the hot spot data packet is M/second.
In one possible embodiment, determining the target pull number according to the consumption rate of the hotspot packet may be determined by a mathematical relationship between the consumption rate and the target pull number. For example, the consumption rate of the hot spot packet is 5000 pieces/second, and if the mathematical relationship between the consumption rate and the target pull count may be the target pull count G, the target pull count is 5000 pieces G.
In a possible implementation manner, determining the target pull number according to the consumption speed of the hotspot packet may further include: and acquiring the consumption speed and the magic reading time of the hot spot data packet, and determining the target pulling quantity according to the consumption speed and the magic reading time. The magic reading time is used for indicating the time that the stock of each Data packet generates magic reading, and the magic reading refers to a phenomenon of generating magic reading (Phantom Data) when a certain transaction reads a record in a certain range and another transaction inserts a new record in the range, and the current transaction reads the record in the range again, for example, when the stock management end of the Data packet queries the stock records in a certain query range of the hot Data packet, a Data packet deduction instruction indicates to deduct the stock of 1 hot Data packet, that is, to insert a new stock record in the query range, and when the stock management end of the Data packet queries the stock records in the query range again, the inserted new stock record is displayed, that is, the magic reading is generated. The magic reading time can be set through a configuration file, and can also be calculated according to the time of generating the magic reading in the actual service process, which is not limited here. Determining the target pull number according to the consumption speed and the magic read time may be to obtain the target pull number through a mathematical relationship between the consumption speed, the magic read time, and the target pull number. For example, when the consumption rate is 5000 pieces/second and the magic reading time is 1s, the target pulling number is 5000 pieces of consumption rate and the magic reading time according to the mathematical relationship.
S303, the target pulling amount of the inventory of the hot-point data packets is pulled from the storage cluster, and the pulled inventory is stored in the cache region.
When the target pull quantity inventory of the hot spot data packets is pulled from the storage cluster, the inventory of the hot spot data packets in the storage cluster is correspondingly reduced, when the pulled target pull quantity inventory is stored in the cache region, the hot spot data packets can be stored in the cache region, and the inventory of the hot spot data packets is determined as the inventory corresponding to the target pull quantity, namely, the reduced inventory of the hot spot data packets in the storage cluster is increased in the cache region. For example, if the inventory number corresponding to the hotspot data packet a in the storage cluster is 5000, and the electronic device has 200 (i.e., the target pull quantity) inventories in the storage cluster, the inventory number of the hotspot data packet a in the storage cluster is 4800, and the inventory number of the hotspot data packet a in the cache area is 200.
In one possible implementation, when it is detected that the hot spot data packet satisfies the pull condition, the target pull amount of the inventory of the hot spot data packet is pulled from the storage cluster, and the pulled inventory is stored in the cache region. Wherein the pulling condition may include one or more of: the inventory surplus of the hot spot data packets in the storage cluster is larger than or equal to a first preset value, and the pressure value of the hot spot data packets to the storage cluster is larger than or equal to a second preset value. Optionally, the pulling condition may also be that the remaining inventory proportion of the hot spot data packet is greater than or equal to a third preset value, or may also be that a fragment pressure value of the hot spot data packet to the fragment is greater than or equal to a fourth preset value, which is not limited here. The preset values, such as the first, second, third, and fourth preset values, may be preset, or may be calculated based on statistical historical data, and so on.
In one possible embodiment, the method for pulling the inventory storage of the target pulling amount of the hot-point data packet from the storage cluster to the cache region may further include the following steps: determining the inventory surplus of the hotspot data packets in the storage cluster; and when the inventory remaining amount in the storage cluster is larger than or equal to a first preset value, storing the target pull quantity of the hot-point data packets from the storage cluster into the cache region. Specifically, the inventory remaining amount of the hotspot data packet in the storage cluster is used to indicate the inventory amount of the hotspot data packet in the storage cluster at the current time, where the current time may be a time when the inventory of the hotspot data packet is pulled from the storage cluster, or a time when the consumption speed of the hotspot data packet is obtained, and this is not limited here. For example, if the first preset value is 300, if the remaining stock amount of the hot spot data packets is 500, obviously, the remaining stock amount is greater than or equal to the first preset value, the stock of the target pull quantity of the hot spot data packets from the storage cluster is stored in the buffer area, and if the remaining stock amount of the hot spot data packets is 200, obviously, the remaining stock amount is less than the first preset value, the stock of the target pull quantity of the hot spot data packets from the storage cluster is not pulled.
Optionally, the consumption value of the hot spot data packet may be determined by recording the inventory amount deducted by each gift deduction request, so as to determine a remaining inventory proportion according to the inventory total amount configured for the hot spot data packet and the data packet consumption value, for example, if the consumption value of the hot spot data packet is Cost, the inventory total amount configured for the hot spot data packet is ALL, and the remaining inventory proportion may be ALL/Cost. After the remaining inventory proportion of the hot spot data packet is determined, when the remaining inventory proportion is larger than or equal to a third preset value, the inventory of the target pulling quantity of the hot spot data packet is pulled from the storage cluster and stored in the cache region. Wherein, the third preset value can also be called a consumption threshold value, and can be read from the configuration file.
In one possible embodiment, the method for pulling the inventory storage of the target pulling amount of the hot-point data packet from the storage cluster to the cache region may further include the following steps: when the pressure value of the hot spot data packet to the storage cluster is larger than or equal to a second preset value, the target pull quantity of the hot spot data packets pulled from the storage cluster is stored in the cache region. For example, the storage cluster pressure value of the hotspot data packet to the storage cluster is M, the second preset value is H, when the storage cluster pressure value M is greater than or equal to the second preset value H, the inventory of the target pull quantity of hotspot data packets pulled from the storage cluster is stored in the cache region, and when the storage cluster pressure value M is less than the second preset value H, the inventory of the target pull quantity of hotspot data packets not pulled from the storage cluster is not stored in the cache region. Optionally, when the storage cluster includes a plurality of segments, a segment pressure value corresponding to the segment storing the hotspot data packet by the hotspot data packet may also be detected, and when the segment pressure value is greater than or equal to a fourth preset value, the inventory of the target pulling number of the hotspot data packet pulled from the storage cluster is stored in the cache region.
In one possible embodiment, the method for pulling the inventory storage of the target pulling amount of the hot-point data packet from the storage cluster to the cache region may further include the following steps: determining the inventory surplus of the hotspot data packets in the storage cluster; and when the inventory surplus in the storage cluster is greater than or equal to a first preset value and the pressure value of the hot spot data packet to the storage cluster is greater than or equal to a second preset value, the target pull quantity of the inventory of the hot spot data packet is pulled from the storage cluster and stored in the cache region. For example, when the first preset value is 300 and the second preset value is M, if the inventory margin of the hot spot data packets is 500 and the pressure value of the storage cluster for the storage cluster is H (H is greater than M), obviously, the inventory margin is greater than or equal to the first preset value and the pressure value of the storage cluster is greater than or equal to the second preset value, the target pull amount of the hot spot data packets from the storage cluster is stored in the cache area; if the inventory margin of the hot spot data packets is 200 (less than the first preset value of 300), or the pressure value of the storage cluster is less than the second preset value of M, the inventory of the target pulling quantity of the hot spot data packets from the storage cluster is not stored in the cache region.
In a possible implementation manner, if it is detected that the type of the hotspot data packet is greater than or equal to the type preset value, the cache region may be divided into a plurality of cache spaces, and the inventory of the target pulling number of the hotspot data packets pulled from the storage cluster is stored in the target cache space in the cache region. For example, N hot spot data packets existing in the storage cluster are detected, and the stocks of the N types of hot spot data packets can be pulled from the storage cluster and stored in the cache region, the cache region is partitioned, if the cache region is isolated in a physical manner or the contents stored in each partitioned cache space are not interfered with each other in a software manner, numbers of the N partitioned cache spaces can be obtained, a modulus value corresponding to each cache space is obtained by calculating in a modulus manner, and the modulus value is the number of the cache space, so that the stocks of the hot spot data packets pulled from the storage cluster are stored in the corresponding cache spaces, and the N cache spaces can store the data packets of the N types of different types correspondingly.
According to the embodiment of the application, the hot spot data packets can be determined from the plurality of data packets, the target pulling quantity is determined according to the consumption speed of the hot spot data packets, and the inventory of the target pulling quantity for pulling the hot spot data packets from the storage cluster is stored in the cache region, so that the phenomenon that a large number of data packet deduction instructions deduct the inventory of the hot spot data packets from the storage cluster can be reduced, the load of the storage cluster can be reduced, and adverse effects brought by the hot spot data packets can be reduced.
Referring to fig. 6, fig. 6 is a schematic flowchart illustrating a hot spot data packet offloading method according to an embodiment of the present disclosure. The hot spot data packet distribution method can be executed by an electronic device. As shown in fig. 6, the hot spot packet offloading method may include the following steps S601-S605.
S601, determining a hot spot data packet from a plurality of data packets;
s602, determining target pulling quantity according to consumption speed of hot spot data packets, wherein the consumption speed is used for indicating deduction speed of inventory corresponding to each data packet;
s603, the target pull quantity of the inventory of the hot-point data packets is pulled from the storage cluster, and the pulled inventory is stored in the cache region.
Steps S601 to S603 can refer to steps S301 to S303, which are not described herein.
S604, receiving a data packet deduction instruction aiming at the hot spot data packet.
The data packet deduction instruction is used for indicating the stock of deduction data packets, and the data packet deduction instruction can be used for indicating the target deduction amount. It is to be understood that in some scenarios, the packet deduction instruction may not indicate the deduction amount, for example, the default deduction amount is 1, and the inventory of the hot packet is deducted from 1 every time the packet deduction instruction is received.
In a possible implementation manner, the packet deduction instruction indicates the deduction target deduction amount may be configured through a packet configuration interface as shown in fig. 5, for example, by configuring the single acquisition amount of the packet, it may be determined that the target deduction amount deducted by each packet deduction instruction is the configured single acquisition amount, in other words, by configuring the single acquisition amount, it may be implemented that the target deduction amount deducted by each packet deduction instruction is consistent, so that when the inventory of the packet is deducted, deduction may be directly performed according to the single acquisition amount, and the efficiency of deducting the inventory of the packet is improved.
And S605, deducting the inventory of the hot spot data packet from the cache region or the storage cluster according to the data packet deduction instruction.
The deduction of the inventory of the hot spot data packets from the cache region according to the data packet deduction instruction may be performed because the inventory number of the hot spot data packets in the cache region is greater than or equal to the target deduction number, or when the inventory number of the hot spot data packets in the cache region is less than the target deduction number, after the inventory of the hot spot data packets is pulled from the storage cluster, the inventory of the hot spot data packets is deducted from the cache region; according to the data packet deduction instruction, the deduction of the inventory of the hot spot data packet from the storage cluster may be due to an inventory failure of a target pulling quantity for pulling the hot spot data packet from the storage cluster, or that the hot spot data packet no longer satisfies a pulling condition of the hot spot data packet, or that the hot spot data packet no longer satisfies a hot spot data packet determination condition (i.e., is no hot spot data packet), which is not limited herein.
In one possible implementation, when it is detected that the inventory amount of the hotspot data packets in the cache area is greater than or equal to the target deduction amount, the inventory of the target deduction amount of the hotspot data packets is deducted from the cache area. The inventory number of the hotspot data packets in the cache region is used to indicate the corresponding number of the inventories of the hotspot data packets stored in the cache region, and the inventory of the hotspot data packets is the inventory pulled from the storage cluster in step S603. When the target deduction amount of the hot spot data packet is deducted from the cache region, the inventory corresponding to the target deduction amount of the hot spot data packet in the cache region is reduced, in other words, the inventory of the target pulling amount is deducted from the cache region in response to the data packet deduction instruction, so that the times of deducting the inventory of the hot spot data packet from the storage cluster is reduced, and the possibility of causing high load to the storage cluster is reduced. For example, the inventory of the hot data packets in the cache region is 100, and when it is detected that the target deduction amount indicated by the data packet deduction instruction is 5, obviously, if the inventory in the cache region is greater than the target deduction amount, the 5 inventories are deducted from the cache region, so that the inventory of the hot data packets in the cache region is 95.
In a possible implementation manner, when detecting that the inventory quantity of the hotspot data packets in the cache area is smaller than the target deduction quantity, detecting the inventory surplus quantity of the hotspot data packets in the storage cluster; when the inventory surplus in the storage cluster is larger than a threshold value, the inventory of the target pulling quantity of the hot-point data packets is pulled from the storage cluster and stored in a cache region; and deducting the target deduction amount of the hot spot data packet from the buffer area. The inventory remaining amount of the hotspot data packet in the storage cluster is used to indicate the inventory amount of the hotspot data packet in the storage cluster at the current time, where the current time may be a time of receiving a data packet deduction instruction, or a time of detecting the inventory remaining amount of the hotspot data packet in the storage cluster, and this is not limited here. When the inventory remaining amount in the storage cluster is greater than the first preset value, the inventory of the target pull quantity of the hot-point data packet is pulled from the storage cluster and stored in the cache region, and the specific description may refer to step S303, which is not described herein again. After pulling a target pull amount of inventory from the storage cluster, the amount of inventory in the cache area may be made greater than a target deduction amount, and then the target deduction amount of inventory is deducted from the cache area. For example, the inventory amount in the cache region is 3, the target deduction amount indicated by the packet deduction instruction is 5, obviously, the inventory amount in the cache region is smaller than the target deduction amount, the inventory remaining amount of the hot data packet in the storage cluster is detected, if the inventory remaining amount is greater than a threshold value, the inventory corresponding to the target pull amount is pulled from the storage cluster and stored in the cache region, if the target pull amount is 100, the inventory amount in the cache region is 103, and then the inventory of the target deduction amount 5 is deducted from the 103 inventories in the cache region.
Optionally, when it is detected that the inventory number of the hot-point data packets in the cache area is smaller than the target deduction number, before the inventory of the target pull number of the hot-point data packets is pulled from the storage cluster, the target pull number may be re-determined according to the relevant description of step S102, and then the inventory of the hot-point data packets is pulled from the storage cluster according to the re-determined target pull number. In other words, the target pull amount may be determined according to the consumption rate calculated when the stock is pulled from the storage cluster, that is, the target pull amount may be dynamically determined according to the actual consumption rate, so that the determination of the target pull amount is more realistic.
In one possible implementation, when the inventory of the target pull amount of the hotspot data packet from the storage cluster fails, if a data packet deduction instruction for the hotspot data packet is received, the inventory of the target deduction amount of the hotspot data packet is deducted from the storage cluster. The inventory failure of the target pull quantity of the hot-point data packets from the storage cluster may be because the remaining inventory in the storage cluster is less than the first preset value, or because the pull failure is caused by a system error, or because the memory in the cache area is insufficient, which is not limited herein. Because the target pull quantity inventory of the hot spot data packet from the storage cluster fails, the target deducted quantity inventory of the hot spot data packet is deducted from the storage cluster, so that the situation that when the cache region cannot pull the inventory from the storage cluster, a data packet deduction instruction still waits for deducting the target deducted quantity inventory from the cache region can be avoided, the time of waiting for response by a user is reduced, and the user experience is improved.
In one possible implementation, when the target deduction amount is greater than a preset deduction amount, determining an excess inventory amount for the hotspot data packet; deducting the inventory of the preset deduction quantity of the hot spot data packets from the buffer area; and returning prompt information to prompt that the excess stock quantity is unsuccessfully deducted. The preset deduction amount may be a value set by a relevant person, and the set value may be a maximum amount of the inventory of the data packets allowed to be deducted per unit time in the cache region, and if the preset deduction amount is exceeded, the data may be read from or written into the cache region. The excess inventory amount is used to indicate an inventory difference between the target deduction amount and a preset inventory amount, for example, if the target deduction amount for the hot spot data packet a is 100, and the preset deduction amount is 105, the excess inventory amount for the hot spot data packet may be 5. The prompt message is used for prompting the failure of the inventory deduction of the excess inventory quantity. Optionally, when inventory of a preset deduction quantity of the hot spot data packet is deducted from the cache region, if the preset deduction quantity is greater than the inventory quantity in the cache region, detecting an inventory remaining quantity of the hot spot data packet in the storage cluster; when the inventory surplus in the storage cluster is larger than a threshold value, the inventory of the target pulling quantity of the hot-point data packets is pulled from the storage cluster and stored in a cache region; and deducting the target deduction amount of the hot spot data packet from the buffer area. In other words, when the excess stock quantity is detected, the excess stock quantity is not deducted, only the part of the stock which is not excess is deducted, and the prompt message is returned. Therefore, the phenomenon that the waiting time of a user is too long due to the deduction of a large amount of inventory in unit time can be avoided, and the adverse effect caused by high load is reduced.
In a possible implementation manner, when a plurality of data packet deduction instructions are detected, the times of deducting the received data packet deduction instructions in unit time are determined, when the times of the received data packet deduction instructions in the unit time are greater than or equal to the preset instruction number, excess data packet deduction instructions for the hot data packet are determined, prompt information is returned according to the excess data packet deduction instructions to prompt that the data packet deduction fails, and inventory of the hot data packet is deducted from a cache region according to the data packet deduction instructions from which the excess data packet deduction instructions are removed. The preset instruction amount may be a value set by a relevant person, the set value may be a maximum amount of packet deduction instructions allowed to be executed in a unit time in the cache region, and if a packet deduction instruction exceeding the preset instruction amount is executed, a crash of reading and writing data from the cache region may be caused. The excess data packet order is a data packet deduction order which is received in unit time and is larger than a preset order number, the excess data packet deduction order can be determined randomly, or the target deduction number deducted by the data deduction order can be sorted, a data packet of which the sum of one or more target deduction numbers with the minimum target deduction number is equal to the excess inventory number is determined as an excess inventory data packet, or the target deduction numbers deducted by the data deduction order can be sorted, and a data packet of which the sum of one or more target deduction numbers with the maximum target deduction number is equal to the excess inventory number is determined as an excess inventory data packet, wherein the data packet is not limited here. The prompt message is used for prompting the initiating user of the excess data packet deduction instruction that the execution of the data packet deduction instruction fails. For example, 2000 deduction instructions for hot data packets a are detected in unit time, if the number of the preset instructions is 1500, 500 excess data packet deduction instructions are randomly determined, and prompt information is returned to prompt that the data packet deduction fails, so that the data packet deduction operation is performed only according to the data packet deduction instructions except the excess data packet deduction instructions, and therefore, by rejecting the data packet deduction instructions exceeding the number of the preset instructions, adverse effects caused by requests of a large number of data packet deduction instructions are reduced, and overlong time of waiting for response of a user due to deduction of a large number of stocks in unit time is avoided, and user experience is improved.
In one possible embodiment, if the cache region is divided into a plurality of cache spaces, the target deduction amount of the stock may be deducted from the cache space stored in the hot packet according to the packet deduction instruction. For example, if it is detected that the packet subtraction instruction indicates that the subtracted inventory of the hotspot packet is the inventory of the hotspot packet a, determining that the number of the cache space where the hotspot packet a is located is AC1101, accessing the cache space of the hotspot packet according to the number of the cache space, and if it is detected that the inventory number of the hotspot packet in the cache space is greater than or equal to the target subtraction number, subtracting the inventory of the target subtraction number of the hotspot packet from the cache space; when detecting that the inventory quantity of the hot spot data packets in the cache space is smaller than the target deduction quantity, detecting the inventory residual quantity of the hot spot data packets in the storage cluster, when the inventory residual quantity in the storage cluster is larger than a threshold value, pulling the inventory of the target pulling quantity of the hot spot data packets from the storage cluster to the cache space, and deducting the inventory of the target deduction quantity of the hot spot data packets from the cache space.
In a possible implementation manner, after the data packet deduction instruction is received, when the fact that the hot data packet inventory exists in the cache region is detected, the hot data packet inventory is deducted from the cache region. For example, if the single acquisition quantity is configured to be 1 time through the data packet configuration interface, each data packet deduction instruction indicates that the deducted stocks are 1, and when the cache region has the hot spot data packet stocks, that is, the hot spot data packet stocks are greater than or equal to 1, 1 stock is deducted from the cache region. Optionally, after receiving the deduction instruction of the data packet, if it is detected that the cache region has an inventory of the hotspot data packet, the inventory in the cache region is directly deducted, and if it is detected that the deduction instruction of the data packet indicates that the deducted target deduction amount is greater than all the inventories in the cache region, the inventory of the target deduction amount may be pulled from the storage cluster, so as to facilitate deduction of the inventory of the target deduction amount by the deduction instruction of the data packet, or deduction of the inventory of the remaining target deduction amounts from other cache regions in the data packet access system.
In one possible implementation, when the hot spot data packet no longer satisfies the hot spot data packet determination condition of the hot spot data packet, the hot spot data packet is not pulled from the storage cluster, and when the data packet deduction instruction is detected, the hot spot data packet target deduction amount of inventory is deducted from the storage cluster. In other words, since the inventory remaining amount of the hotspot packet gradually decreases when the hotspot packet no longer satisfies the hotspot packet determination condition, such as through consumption of the inventory of the hotspot packet during the activity process, the access request for the hotspot packet is gradually slowed down as shown in fig. 3, and the hotspot data is no longer the hotspot packet, the target deduction amount of the inventory of the hotspot packet can be directly deducted from the storage cluster when the packet deduction instruction is detected.
In one possible embodiment, the inventory of the hot-point data packet is pulled from the storage cluster at the time when the data packet can be deducted, and then the data packet deduction instruction is detected, and then the inventory is pulled from the storage cluster. For example, if an activity starts from time t, the hot spot data packet in the activity can be deducted only after time t, and after the activity starts, the data packet deduction instruction is received, and then the consumption speed of the hot spot data packet is calculated, so that the target amount of inventory is pulled from the storage cluster. Optionally, the time for pulling the inventory of the hotspot data packet from the storage cluster may be to pull the inventory from the storage cluster in advance by predicting the consumption rate of the hotspot data packet before the data packet can be deducted. For example, in a second killing activity, a large amount of packet deduction requests need to be completed within a certain time (i.e. a second killing in the limit case), and a certain time is needed to pull inventory from the storage cluster, the consumption speed of the hot-spot packets in the activity can be predicted by the activity and your baby (for example, the consumption speed of the hot-spot packets is 1.5 w/s within 10 seconds from the beginning of the activity), and the hot-spot packets are pulled to the cache area in advance.
For example, a data packet is taken as a gift packet. The method can adopt the idea of flow control, adopts an LRU mode to count the gift packet access type influence factor T, obtains the weight of each fragment in the storage cluster to the storage cluster (namely the service call information of the access local routing information weight) through access routing, and can obtain the pressure sequence of the access requests aiming at the first N gift packets (namely the first N data packets) to the fragments as { T1, T2, T3 … Tk … TN }, and the corresponding consumption speed is { C1, C2, … Ck … CN }. And then, accessing the Agent program through the Proxy program to obtain a weight value W remotely stored by local access, and calculating ALL weight synthesis W _ A remotely stored by communicating with the Agent, wherein the amplification value of the remote storage node is K ═ W _ A/W, the access speed of the corresponding top N gift packets is { T1K, T2K, … TN K }, a hot spot data packet (i.e. a hot Key) can be determined according to the access speed of the gift packets, and the remaining inventory proportion of the hot spot data packet is calculated, for example, the consumption value Cost of the hot spot data packet is returned every time the remote storage is accessed, the inventory proportion configured by the hot spot data packet is ALL (i.e. the configuration total distributed by the local program), and the remaining inventory proportion R of the hot spot data packet is ALL/Cost is calculated. And if the R value is greater than a third preset value R _ C (also referred to as a consumption threshold), and the slice pressure value of the hotspot data packet to a single slice in the storage cluster is calculated to be greater than a fourth preset value (also referred to as a single slice pressure early warning value), pulling a target pulling amount of inventory from the storage cluster, where the pulled inventory value may be F ═ Ck × Tm (Ck may be a consumption speed of the hotspot data packet, and Tm is a magic reading time). After the inventory value F is successfully pulled from the storage cluster, F is directly stored in a local shared memory (i.e., a cache region). Because each gift package or commodity (i.e. data package) has different access times, if the number of hot keys is large, the shared memory can be divided in a modulus mode to obtain a plurality of cache spaces, when a data package deduction instruction (i.e. a client has deduction action) is detected, the data package deduction instruction indicates an inventory for deducting the target pull quantity, whether the inventory of the cache space corresponding to the modulus value is larger than the target deduction quantity is checked, if the inventory of the cache space is larger than the target deduction quantity, the inventory of the target deduction quantity is directly deducted, if not, the corresponding R value (i.e. the residual margin proportion of the hot data package) is checked, and if the R value is larger than R _ C, the F value is deducted from the storage cluster and stored in the shared memory so as to be deducted by other data package deduction instructions.
For another example, please refer to fig. 7, fig. 7 is a schematic application flow diagram of a hot spot data packet offloading method provided in an embodiment of the present application, and as shown in fig. 7, an application process of the entire data packet offloading method mainly relates to a user cluster, an agent program, a work module, a cache area, a storage cluster, and the like. Specifically, after a client cluster initiates a data packet deduction instruction for a hot data packet, step 701 is executed, an agent program (such as a proxy program) receives the data packet deduction instruction for the hot data packet, then the agent program determines a storage position of the hot data packet by analyzing the data packet deduction instruction, step 702 is executed, a work module is called to execute the data packet deduction instruction, the work module is like a 'employee' under the agent program, and the work module specifically executes to deduct the inventory of the hot data packet from a cache area or a storage cluster; the working module performs step 703 to deduct the hotspot data packets from the cache region, and if the inventory of the hotspot data packets in the cache region is smaller than the target deduction amount indicated by the data packet deduction instruction, then performs step 704, where the electronic device pulls the inventory of the hotspot data packets from the storage cluster to store in the cache region, so that the working module deducts the inventory of the target deduction amount of the hotspot data packets from the cache region according to the data packet deduction instruction.
For another example, please refer to fig. 8, where fig. 8 is a schematic application flow diagram of a hot spot data packet offloading method according to an embodiment of the present application. If the inventory of the hot spot data packet is deducted from the cache region, the deduction process of the hot spot data packet may be as shown in fig. 8, and the whole process may refer to fig. 8, which relates to the user terminal, the agent program, the work module, the cache region, the storage cluster, and the like. Specifically, the user terminal initiates a data packet deduction instruction (step 801), the agent program checks the data packet deduction instruction, if the data packet deduction instruction check fails (step 802), if the network state of the user terminal is in a problem, or the data packet deduction instruction is rejected for reducing load according to the fact that the data packet deduction instruction of the hot data packet a is greater than a preset instruction number, the user terminal directly returns that the data packet deduction instruction fails to deduct the hot data packet; if the agent program passes the check, step 806 is executed, the work module is called to deduct the inventory of the hot spot data packet from the cache region, before that, the agent program may execute steps 803 and 804, count the access request information for the hot spot data packet, and determine the target pull number according to the consumption speed of the hot spot data packet, so that the cache region executes step 805, and the inventory of the target pull number for pulling the hot spot data packet from the storage cluster is stored in the cache region; when receiving an instruction distributed by the agent program, the working module accesses the cache area, deducts the inventory of the hotspot data packets if the inventory of the hotspot data packets in the cache area is larger than the target deduction quantity (step 807), and executes step 809 to update the inventory quantity of the hotspot data packets stored in the cache area; the working module feeds back the success of deduction of the inventory of the hotspot data packet to the agent program (step 808), and the agent program returns the success of deduction of the client data packet. If the target pull amount of the hot spot data packet from the storage cluster fails, the hot spot data packet is deducted from the storage cluster, step 810, so that the agent may feed back to the user end that the data packet deduction is successful. Referring to fig. 9, fig. 9 is a schematic view of an activity effect provided by an embodiment of the present application, where in fig. 9 (a) is a graph showing a variation value of access request times after applying the packet offloading method, and in fig. 901 is a graph showing a peak value of access request times of the hot spot packet, and as time goes on, the access request times for the hot spot packet gradually slow down. Fig. 9 (b) illustrates a pressure value change of a hot-spot data packet to a storage cluster after applying the data packet offloading method, and fig. 902 illustrates a time point when the data packet offloading method is executed. In addition, in the whole process of the scheme, the server executes corresponding operation according to the access request information, the consumption speed, the related preset value or the weight information and the like of the data packet, so that manual intervention is not needed, and the operation and maintenance cost is greatly reduced. Further, since the activity usually has a large number of accesses of only a small number of data packets, and a large number of accesses of data packets is small (i.e., "long tail effect"), the accuracy is high when querying the inventory corresponding to each data packet, because the data packets with the small number of accesses do not satisfy the pull condition of the inventory (e.g., the access speed of the storage cluster of the data packet is smaller than the speed threshold, the inventory margin is smaller than the first preset value, etc.), the data packet at the tail portion is not pulled to the cache region, and the inventory number in the storage cluster is the real-time inventory corresponding to the data packet, and the query inventory accuracy is high.
Referring to fig. 10, fig. 10 is a schematic application flow diagram of a hot spot data packet offloading method according to an embodiment of the present disclosure. Specifically, as shown in fig. 10, the present invention mainly relates to a user side, an agent program, a user dimension data packet, a storage cluster, and the like, where the user dimension data packet is a data packet storing relevant information of each user, the user dimension data packet has a corresponding value, and the value of the user dimension data packet is used to indicate that each user can obtain the number of the hot spot data packets. Specifically, the user side initiates a data packet deduction instruction (step 1001), the agent program checks the data packet deduction instruction, if the data packet deduction instruction check fails, step 1002 is executed, and the data packet deduction instruction is returned to the user side, and the hot data packet deduction failure is determined; if the packet command passes the check, the agent executes step 1003 to lock the value of the user dimension packet and the locking is successful (step 1001); the agent program executes step 1005, detecting the inventory quantity of the data packets in the storage cluster, and detecting that the inventory quantity of the data packets in the storage cluster is larger than the target deduction quantity (step 1001); the agent then proceeds to step 1007 where the value of the user dimension data packet is deducted and the value of the user dimension data packet is updated, and the agent proceeds to step 1010 where the inventory of the data packet is deducted from the storage cluster and the inventory of the data packet is updated. Optionally, when the deduction of the value of the data packet of the user dimension fails, rolling back to the data packet of the user dimension, checking the value of the data packet of the user dimension again, canceling the locking of the value of the user dimension, and rolling back to the user side the error of deducting the data packet inventory.
According to the embodiment of the application, the hot spot data packets can be determined from the plurality of data packets, the target pulling quantity is determined according to the consumption speed of the hot spot data packets, and the inventory of the target pulling quantity for pulling the hot spot data packets from the storage cluster is stored in the cache region, so that the hot spot data packets are shunted, and the hot spot data packets are deducted from the cache region or the storage cluster when the data packet deduction instruction is detected, so that the load of the storage cluster can be reduced, and the operation and maintenance cost is reduced.
Based on the description of the foregoing hot spot data packet offloading method embodiment, an embodiment of the present application further discloses a hot spot data packet offloading device, which may be configured in the foregoing electronic device, such as a server, for example, the device may be a computer program (including a program code) running in the electronic device. The apparatus may perform the method illustrated in fig. 11. Referring to fig. 11, the apparatus may operate as follows:
a hot spot data packet determining module 1101, configured to determine a hot spot data packet from a plurality of data packets;
the processing module 1102 is configured to determine a target pulling amount according to a consumption rate of the hotspot data packets, where the consumption rate is used to indicate a deduction rate of the inventory corresponding to each data packet;
the processing module 1102 is further configured to pull the target pull amount of the inventory of the hot-point data packet from the storage cluster, and store the pulled inventory in the cache region.
In an embodiment, when the processing module 1102 is configured to determine the target pull number according to the consumption rate of the hotspot packet, the processing module may specifically be configured to:
acquiring the consumption speed and the magic reading time of the hotspot data packets, wherein the magic reading time is used for indicating the time for generating magic reading of the inventory of each data packet;
and determining the target pulling quantity according to the consumption speed and the magic reading time.
In one embodiment, the processing module 1102 is further configured to receive a packet deduction instruction for the hotspot packet, where the packet deduction instruction indicates a target deduction number;
and when detecting that the inventory quantity of the hot spot data packets in the cache region is larger than or equal to the target deduction quantity, deducting the inventory of the target deduction quantity of the hot spot data packets from the cache region.
In one embodiment, the processing module 1102 is further configured to receive a packet deduction instruction for the hotspot packet, where the packet deduction instruction indicates a target deduction number;
when detecting that the inventory quantity of the hotspot data packets in the cache area is less than the target deduction quantity, detecting the inventory surplus of the hotspot data packets in the storage cluster;
when the inventory surplus in the storage cluster is larger than a threshold value, pulling the inventory of the hot-point data packet from the storage cluster and storing the inventory into a cache region;
and deducting the target deduction amount of the hot spot data packet from the buffer area.
In an embodiment, the processing module 1102 is further configured to pull an inventory of a target pull quantity of the hot-point data packet from the storage cluster, and store the pulled inventory in the cache region, specifically to:
when the hot spot data packet is detected to meet the pulling condition, the target pulling amount of the inventory of the hot spot data packet is pulled from the storage cluster, and the pulled inventory is stored in the cache region.
Wherein the pulling condition may include one or more of: the inventory surplus of the hot spot data packets in the storage cluster is larger than or equal to a first preset value or the pressure value of the hot spot data packets to the storage cluster is larger than or equal to a second preset value.
In one embodiment, the processing module 1102 is further configured to receive a packet deduction instruction for the hotspot packet, where the packet deduction instruction indicates a target deduction number;
when the target deduction quantity is larger than the preset deduction quantity, determining an excess inventory quantity for the hotspot data packet, wherein the excess inventory quantity is used for indicating an inventory difference value between the target deduction quantity and the preset inventory quantity;
deducting the inventory of the preset deduction quantity of the hot spot data packets from the buffer area;
and returning prompt information, wherein the prompt information is used for prompting that the excess inventory quantity is unsuccessfully deducted.
In an embodiment, when the processing module 1102 is configured to determine a hot spot data packet from a plurality of data packets, the processing module may specifically be configured to:
determining local access speeds of a plurality of data packets according to the access request information aiming at each data packet;
acquiring weight information, wherein the weight information is used for indicating the corresponding relation between the local access speed and the access speed of the storage cluster;
determining the access speed of the storage cluster corresponding to each data packet according to the local access speed and the weight information;
and determining the data packet with the access speed of the storage cluster meeting the preset condition as a hot spot data packet.
The functional modules in the embodiments of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules are integrated into one module. The integrated module may be implemented in a form of hardware, or may be implemented in a form of software functional module, which is not limited in this application.
According to the embodiment of the application, the hot spot data packets can be determined from the plurality of data packets, the target pulling quantity is determined according to the consumption speed of the hot spot data packets, and the inventory of the target pulling quantity for pulling the hot spot data packets from the storage cluster is stored in the cache region, so that the hot spot data packets are shunted, the load of the storage cluster can be reduced, and the operation and maintenance cost is reduced.
Referring to fig. 12 again, the electronic device according to an embodiment of the present disclosure is a schematic structural diagram, and the electronic device according to an embodiment of the present disclosure may include a processor 1201 and a storage device 1202. Optionally, the electronic device may also include a network interface 1203. Further optionally, the electronic device may further include a power supply module and the like. The processor 1201, the storage 1202, and the network interface 1203 may interact with each other to exchange data, the network interface 1203 is controlled by the processor to transmit and receive messages, the storage 1202 is used to store a computer program, the computer program includes program instructions, and the processor 1201 is used to execute the program instructions stored in the storage 1202. Wherein the processor 1201 is configured to invoke program instructions to perform the above-described methods.
The storage 1202 may include a volatile memory (volatile memory), such as a random-access memory (RAM); the storage device 1202 may also include a non-volatile memory (non-volatile memory), such as a flash memory (flash memory), a solid-state drive (SSD), or the like; the storage means 1202 may also comprise a combination of memories of the kind described above.
The processor 1201 may be a Central Processing Unit (CPU) 1201. In one embodiment, the processor 1201 may also be a Graphics Processing Unit (GPU) 1201. The processor 1201 may also be a combination of a CPU and a GPU.
In one embodiment, the storage 1202 is used to store program instructions. The processor 1201 may invoke program instructions to perform the following steps:
determining a hot spot data packet from a plurality of data packets;
determining a target pulling quantity according to the consumption speed of the hotspot data packets, wherein the consumption speed is used for indicating the deduction speed of the corresponding inventory of each data packet;
and pulling the target pulling amount of the inventory of the hot-point data packet from the storage cluster, and storing the pulled inventory into the cache region.
In an embodiment, when the processor 1201 determines the target pull number according to the consumption rate of the hotspot packet, it may specifically perform:
acquiring the consumption speed and the magic reading time of the hotspot data packets, wherein the magic reading time is used for indicating the time for generating magic reading of the inventory of each data packet;
and determining the target pulling quantity according to the consumption speed and the magic reading time.
In one embodiment, the processor 1201 may further perform: receiving a data packet deduction instruction aiming at the hotspot data packet, wherein the data packet deduction instruction indicates the target deduction number;
and when detecting that the inventory quantity of the hot spot data packets in the cache region is larger than or equal to the target deduction quantity, deducting the inventory of the target deduction quantity of the hot spot data packets from the cache region.
In one embodiment, the processor 1201 may further perform: receiving a data packet deduction instruction aiming at the hotspot data packet, wherein the data packet deduction instruction indicates the target deduction number;
when detecting that the inventory quantity of the hotspot data packets in the cache area is less than the target deduction quantity, detecting the inventory surplus of the hotspot data packets in the storage cluster;
when the inventory surplus in the storage cluster is larger than a threshold value, pulling the inventory of the hot-point data packet from the storage cluster and storing the inventory into a cache region;
and deducting the target deduction amount of the hot spot data packet from the buffer area.
In one embodiment, the processor 1201 specifically performs, when performing a process of pulling an inventory of a target pull amount of a hot-point data packet from a storage cluster and storing the pulled inventory in a cache region:
when the hot spot data packet is detected to meet the pulling condition, the target pulling amount of the inventory of the hot spot data packet is pulled from the storage cluster, and the pulled inventory is stored in the cache region.
Wherein the pulling condition may include one or more of: the inventory surplus of the hot spot data packets in the storage cluster is larger than or equal to a first preset value or the pressure value of the hot spot data packets to the storage cluster is larger than or equal to a second preset value.
In one embodiment, the processor 1201 may further perform:
receiving a data packet deduction instruction aiming at the hotspot data packet, wherein the data packet deduction instruction indicates the target deduction number;
when the target deduction quantity is larger than the preset deduction quantity, determining an excess inventory quantity for the hotspot data packet, wherein the excess inventory quantity is used for indicating an inventory difference value between the target deduction quantity and the preset inventory quantity;
deducting the inventory of the preset deduction quantity of the hot spot data packets from the buffer area;
and returning prompt information, wherein the prompt information is used for prompting that the excess inventory quantity is unsuccessfully deducted.
In an embodiment, when determining a hot spot packet from a plurality of packets, the processor 1201 specifically performs:
determining local access speeds of a plurality of data packets according to the access request information aiming at each data packet;
acquiring weight information, wherein the weight information is used for indicating the corresponding relation between the local access speed and the access speed of the storage cluster;
determining the access speed of the storage cluster corresponding to each data packet according to the local access speed and the weight information;
and determining the data packet with the access speed of the storage cluster meeting the preset condition as a hot spot data packet.
According to the embodiment of the application, the hot spot data packets can be determined from the plurality of data packets, the target pulling quantity is determined according to the consumption speed of the hot spot data packets, and the inventory of the target pulling quantity for pulling the hot spot data packets from the storage cluster is stored in the cache region, so that the hot spot data packets are shunted, and the hot spot data packets are deducted from the cache region or the storage cluster when the data packet deduction instruction is detected, so that the load of the storage cluster can be reduced, and the operation and maintenance cost is reduced.
In specific implementation, the apparatus, the processor 1201, the storage apparatus 1202 and the like described in this embodiment may perform the implementation described in the above method embodiment, and may also perform the implementation described in this embodiment, which is not described herein again.
A computer (readable) storage medium is provided in an embodiment of the present application, and a computer program is stored in the computer storage medium, and includes program instructions, and when the program instructions are executed by a processor, some or all of the steps executed in the above method embodiments may be executed. Alternatively, the computer storage media may be volatile or nonvolatile.
Embodiments of the present application also provide a computer program product or computer program comprising program instructions, which may be stored in a computer-readable storage medium. The processor of the computer device reads the program instructions from the computer-readable storage medium, and the processor executes the program instructions, so that the computer performs part or all of the steps performed in the method, which are not described herein again.
It will be understood by those skilled in the art that all or part of the processes of the methods of the above embodiments may be implemented by hardware instructions of a computer program, which may be stored in a computer storage medium, and the computer storage medium may be a computer readable storage medium, and when executed, the programs may include the processes of the above embodiments of the methods. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
While the present disclosure has been described with reference to particular embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure.

Claims (10)

1. A hot spot data packet shunting method is characterized by comprising the following steps:
determining a hot spot data packet from a plurality of data packets;
determining a target pulling quantity according to the consumption speed of the hotspot data packets, wherein the consumption speed is used for indicating the deduction speed of the corresponding inventory of each data packet;
and pulling the target pulling amount of the inventory of the hot spot data packets from the storage cluster, and storing the pulled inventory in a cache region.
2. The method of claim 1, wherein determining a target pull number according to the consumption rate of the hotspot packets comprises:
acquiring the consumption speed and the magic reading time of the hotspot data packets, wherein the magic reading time is used for indicating the time for generating magic reading of the stock of each data packet;
and determining a target pulling quantity according to the consumption speed and the magic reading time.
3. The method of claim 1, further comprising:
receiving a data packet deduction instruction for the hotspot data packet, wherein the data packet deduction instruction indicates a target deduction number;
when detecting that the inventory quantity of the hot spot data packets in the cache region is larger than or equal to the target deduction quantity, deducting the inventory of the target deduction quantity of the hot spot data packets from the cache region.
4. The method of claim 1, further comprising:
receiving a data packet deduction instruction for the hotspot data packet, wherein the data packet deduction instruction indicates a target deduction number;
when detecting that the inventory quantity of the hotspot data packets in the cache region is smaller than the target deduction quantity, detecting the inventory remaining quantity of the hotspot data packets in the storage cluster;
when the inventory surplus in the storage cluster is larger than a threshold value, pulling the inventory of the hot spot data packet from the storage cluster and storing the inventory into a cache region;
deducting the target deduction amount of inventory of the hotspot packets from the cache area.
5. The method of claim 1, further comprising:
receiving a data packet deduction instruction for the hotspot data packet, wherein the data packet deduction instruction indicates a target deduction number;
when the target deduction amount is larger than a preset deduction amount, determining an excess inventory amount for the hotspot data packet, wherein the excess inventory amount is used for indicating an inventory difference value between the target deduction amount and the preset inventory amount;
deducting the inventory of the preset deduction amount of the hotspot data packets from the cache region;
and returning prompt information, wherein the prompt information is used for prompting that the excess inventory quantity is unsuccessfully deducted.
6. The method of any of claims 1-5, wherein the pulling the target pull amount of inventory for the hotspot packet from the storage cluster and storing the pulled inventory in a cache area comprises:
when the hot spot data packet is detected to meet the pull condition, pulling the target pull quantity of the inventory of the hot spot data packet from the storage cluster, and storing the pulled inventory into a cache region;
wherein the pulling condition may comprise any one or more of: the inventory surplus of the hot spot data packet in the storage cluster is larger than or equal to a first preset value, and the pressure value of the hot spot data packet to the storage cluster is larger than or equal to a second preset value.
7. The method according to any one of claims 1-5, wherein said determining a hotspot packet from a plurality of packets comprises:
determining local access speeds of the plurality of data packets according to the access request information aiming at each data packet;
acquiring weight information, wherein the weight information is used for indicating the corresponding relation between the local access speed and the access speed of the storage cluster;
determining the access speed of the storage cluster corresponding to each data packet according to the local access speed and the weight information;
and determining the data packet with the access speed of the storage cluster meeting the preset speed condition as a hot spot data packet.
8. A hot spot data packet shunting device is characterized by comprising:
the hot spot data packet determining module is used for determining a hot spot data packet from a plurality of data packets;
the processing module is used for determining a target pulling quantity according to the consumption speed of the hotspot data packets, wherein the consumption speed is used for indicating the deduction speed of the corresponding inventory of each data packet;
the processing module is further configured to pull the inventory of the target pull quantity of the hotspot data packets from the storage cluster, and store the pulled inventory into a cache region.
9. An electronic device comprising a processor and a storage device, the processor being interconnected with the storage device, wherein the storage device is configured to store computer program instructions, and the processor is configured to execute the program instructions to implement the method of any one of claims 1-7.
10. A computer-readable storage medium, having stored thereon computer program instructions, which, when executed by a processor, are adapted to perform the method of any one of claims 1-7.
CN202110487441.9A 2021-04-30 2021-04-30 Hot spot data packet distribution method and device, electronic equipment and medium Active CN113225396B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110487441.9A CN113225396B (en) 2021-04-30 2021-04-30 Hot spot data packet distribution method and device, electronic equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110487441.9A CN113225396B (en) 2021-04-30 2021-04-30 Hot spot data packet distribution method and device, electronic equipment and medium

Publications (2)

Publication Number Publication Date
CN113225396A true CN113225396A (en) 2021-08-06
CN113225396B CN113225396B (en) 2022-08-12

Family

ID=77090778

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110487441.9A Active CN113225396B (en) 2021-04-30 2021-04-30 Hot spot data packet distribution method and device, electronic equipment and medium

Country Status (1)

Country Link
CN (1) CN113225396B (en)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110040848A1 (en) * 2009-08-17 2011-02-17 Yahoo! Inc. Push pull caching for social network information
WO2011157150A2 (en) * 2011-05-31 2011-12-22 华为技术有限公司 Data processing method, cache node, cooperating controller and system
CN105373347A (en) * 2015-10-23 2016-03-02 浪潮(北京)电子信息产业有限公司 Hot data identification and scheduling method and system for storage system
CN108683695A (en) * 2018-03-23 2018-10-19 阿里巴巴集团控股有限公司 Hot spot access processing method, cache access agent equipment and distributed cache system
CN109726074A (en) * 2018-08-31 2019-05-07 网联清算有限公司 Log processing method, device, computer equipment and storage medium
CN111432039A (en) * 2020-06-15 2020-07-17 北京金山云网络技术有限公司 Data request method, device and equipment in CDN and CDN node
CN112076464A (en) * 2020-09-04 2020-12-15 腾讯科技（深圳）有限公司 Data request processing method and device, computer equipment and storage medium
CN112115167A (en) * 2020-08-21 2020-12-22 苏宁云计算有限公司 Cache system hot spot data access method, device, equipment and storage medium
CN112615907A (en) * 2020-12-04 2021-04-06 北京齐尔布莱特科技有限公司 Data synchronization system and method

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110040848A1 (en) * 2009-08-17 2011-02-17 Yahoo! Inc. Push pull caching for social network information
WO2011157150A2 (en) * 2011-05-31 2011-12-22 华为技术有限公司 Data processing method, cache node, cooperating controller and system
CN105373347A (en) * 2015-10-23 2016-03-02 浪潮(北京)电子信息产业有限公司 Hot data identification and scheduling method and system for storage system
CN108683695A (en) * 2018-03-23 2018-10-19 阿里巴巴集团控股有限公司 Hot spot access processing method, cache access agent equipment and distributed cache system
CN109726074A (en) * 2018-08-31 2019-05-07 网联清算有限公司 Log processing method, device, computer equipment and storage medium
CN111432039A (en) * 2020-06-15 2020-07-17 北京金山云网络技术有限公司 Data request method, device and equipment in CDN and CDN node
CN112115167A (en) * 2020-08-21 2020-12-22 苏宁云计算有限公司 Cache system hot spot data access method, device, equipment and storage medium
CN112076464A (en) * 2020-09-04 2020-12-15 腾讯科技（深圳）有限公司 Data request processing method and device, computer equipment and storage medium
CN112615907A (en) * 2020-12-04 2021-04-06 北京齐尔布莱特科技有限公司 Data synchronization system and method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
王剑冰: "一种高并发分布式缓存服务架构", 《黑龙江科技信息》 *

Also Published As

Publication number Publication date
CN113225396B (en) 2022-08-12

Similar Documents

Publication Publication Date Title
CN105205014B (en) A kind of date storage method and device
US11075982B2 (en) Scaling hosts in distributed event handling systems
US9569742B2 (en) Reducing costs related to use of networks based on pricing heterogeneity
US20190220201A1 (en) Methods, apparatuses, and computer program products for controlling write requests in storage system
US10713223B2 (en) Opportunistic gossip-type dissemination of node metrics in server clusters
US10860245B2 (en) Method and apparatus for optimizing data storage based on application
US20160239229A1 (en) Hierarchical scale unit values for storing instances of data
US20140059163A1 (en) Distributed request processing
CN103312624A (en) Message queue service system and method
CN111737168A (en) Cache system, cache processing method, device, equipment and medium
US8868525B2 (en) Distributed information synchronization
CN107872517A (en) A kind of data processing method and device
CN106973091B (en) Distributed memory data redistribution method and system, and master control server
US10678192B1 (en) Optimization of production systems
CN110909978A (en) Resource processing method, device, server and computer readable storage medium
CN113225396B (en) Hot spot data packet distribution method and device, electronic equipment and medium
JP5661355B2 (en) Distributed cache system
CN108629653A (en) A kind of low delay inventory reduces service system and method
CN116089477B (en) Distributed training method and system
CN107295059A (en) The statistical system and method for service propelling amount
CN111905362A (en) Game guild loading method and server
CN103746839B (en) PaaS systems and PaaS are using the VM node scheduling methods in pond
JP6659229B2 (en) POS system, information processing method, and program
US11940923B1 (en) Cost based cache eviction
CN114844893A (en) Independent site distributed system, and independent site data management method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant