CN116155783A - Hot spot data detection method and device, storage medium and electronic equipment - Google Patents
Hot spot data detection method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN116155783A CN116155783A CN202211710359.9A CN202211710359A CN116155783A CN 116155783 A CN116155783 A CN 116155783A CN 202211710359 A CN202211710359 A CN 202211710359A CN 116155783 A CN116155783 A CN 116155783A
- Authority
- CN
- China
- Prior art keywords
- target data
- data
- request
- hot spot
- frequency
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/16—Threshold monitoring
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/147—Network analysis or design for predicting network behaviour
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present disclosure provides a method, apparatus, computer program product, non-transitory computer readable storage medium, and electronic device for detecting hotspot data. The method comprises the following steps: acquiring a request record of target data in a target time period; dividing a target time period into a plurality of time windows, and determining the request frequency of target data in the latest time window according to the request record; under the condition that the request frequency of the target data in the latest time window is larger than the first frequency threshold and smaller than the second frequency threshold, carrying out trend prediction on the request frequency of the target data in a plurality of time windows to obtain a corresponding trend prediction result; and determining whether the target data is hot spot data according to the trend prediction result. Embodiments of the present disclosure are capable of detecting hotspot data in advance and accurately.
Description
Technical Field
The present disclosure relates generally to the field of computer technology, and more particularly, to a method, apparatus, computer program product, non-transitory computer readable storage medium, and electronic device for detecting hotspot data.
Background
This section is intended to introduce a few aspects of the art that may be related to various aspects of the present disclosure that are described and/or claimed below. This section is believed to help provide background information to facilitate a better understanding of various aspects of the disclosure. It should therefore be understood that these statements are to be read in this light, and not as admissions of prior art.
In service scenes such as network live broadcast, hot spot phenomenon exists commonly. For example, in live broadcasting of a hot match, when a user enters a live broadcasting room, a large number of number package services (services for providing configuration and inquiry capabilities for group selection) are required to be called to divide crowd attributes, and the number package services form hot spot data.
Hot spot phenomena can present challenges to the operation of a service. For example, the access pressure of a large number of users is generally dispersed to each server, so as to solve the traffic flood peak problem. However, when hot spot is encountered, the pressure is concentrated on a part of servers, and the adverse effects such as system breakdown are caused after the bearing capacity of the servers is exceeded. For example, in a case of using rediss as a cache, data may exist on a certain redis fragment according to a hash rule, but because of the access of a short-time hot key (the access quantity of the key is far more than that of other keys in a period of time, so that a large amount of access traffic falls in a corresponding redis instance), the cluster paralysis of the redis fragment where the data is located may be instantaneously caused, and the stability of the whole system is further affected.
In known schemes, data is considered hot spot data and is buffered when its request frequency reaches a certain threshold. The scheme is good in stable flow flood peak, but in actual conditions, the flow unstably grows more frequently. If the threshold value of the hot spot is directly lowered, the hot spot can be cached in advance, but misjudgment of some hot spots and waste of the cache can be generated.
Therefore, it is necessary to propose a detection scheme for hot spot data, so as to alleviate or solve at least one of the above-mentioned technical problems.
Disclosure of Invention
The present disclosure is directed to a method, apparatus, computer program product, non-transitory computer readable storage medium, and electronic device for detecting hotspot data in advance and accurately.
According to a first aspect of the present disclosure, there is provided a method for detecting hotspot data, including: acquiring a request record of target data in a target time period; dividing the target time period into a plurality of time windows, and determining the request frequency of the target data in the latest time window according to the request record; under the condition that the request frequency of the target data in the latest time window is larger than a first frequency threshold and smaller than a second frequency threshold, carrying out trend prediction on the request frequency of the target data in a plurality of time windows to obtain a corresponding trend prediction result; and determining whether the target data is hot spot data according to the trend prediction result.
According to a second aspect of the present disclosure, there is provided a detection apparatus for hot spot data, including: the acquisition module is used for acquiring a request record of target data in a target time period; the dividing module is used for dividing the target time period into a plurality of time windows and determining the request frequency of the target data in the latest time window according to the request record; the prediction module is used for carrying out trend prediction on the request frequency of the target data in a plurality of time windows under the condition that the request frequency of the target data in the latest time window is larger than a first frequency threshold and smaller than a second frequency threshold, so as to obtain a corresponding trend prediction result; and the judging module is used for determining whether the target data is hot spot data according to the trend prediction result.
According to a third aspect of the present disclosure, there is provided a computer program product comprising program code instructions which, when the program product is executed by a computer, cause the computer to perform the method according to the first aspect of the present disclosure.
According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method according to the first aspect of the present disclosure.
According to a fifth aspect of the present disclosure, there is provided an electronic device comprising: a processor, a memory in electronic communication with the processor; and instructions stored in the memory and executable by the processor to cause the electronic device to perform the method according to the first aspect of the present disclosure.
In each embodiment provided by the disclosure, trend prediction is performed under the condition that the request frequency of the target data meets a specific condition, and whether the target data is hot spot data is judged according to the result of the trend prediction, so that the hot spot data can be detected in advance (for example, at least one time window in advance) compared with the existing mode, and the hot spot phenomenon can be dealt with in advance.
It should be understood that what is described in this section is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used solely to determine the scope of the claimed subject matter.
Drawings
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and other drawings may be obtained according to these drawings without inventive effort to a person of ordinary skill in the art. Throughout the drawings, identical reference numerals designate similar, but not necessarily identical, elements.
FIG. 1 illustrates a system architecture diagram of one embodiment of a method of detecting hotspot data in accordance with the present disclosure;
FIG. 2 illustrates a flow chart of one embodiment of a method of detecting hotspot data in accordance with the present disclosure;
FIG. 3A shows a schematic diagram of a time window of a method of detection of hotspot data according to the present disclosure;
FIG. 3B is a schematic diagram illustrating a determination process of a method of detecting hotspot data according to the present disclosure;
FIG. 3C is a schematic diagram illustrating a request handling process of a method of detecting hotspot data according to the present disclosure;
FIG. 3D is a schematic diagram illustrating one specific example of a method of detecting hotspot data according to the present disclosure;
FIG. 4 illustrates an exemplary block diagram of a detection apparatus for hotspot data according to an embodiment of the present disclosure;
fig. 5 shows a schematic diagram of an example electronic device 500 that may be used to implement embodiments of the present disclosure.
Detailed description of the preferred embodiments
The present disclosure will be described more fully hereinafter with reference to the accompanying drawings. However, the present disclosure may be embodied in many alternate forms and should not be construed as limited to the embodiments set forth herein. Thus, while the disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that there is no intention to limit the disclosure to the specific forms disclosed, but on the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the disclosure as defined by the appended claims.
It will be understood that, although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the teachings of the present disclosure.
Some examples are described herein in connection with block diagrams and/or flow charts, wherein each block represents a portion of circuit elements, module, or code that comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in other implementations, the functions noted in the blocks may occur out of the order noted. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
Reference herein to "an embodiment according to … …" or "in an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one implementation of the disclosure. The appearances of the phrase "in accordance with an embodiment" or "in an embodiment" in various places herein are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments necessarily mutually exclusive of other embodiments.
Fig. 1 illustrates an exemplary system architecture 100 to which embodiments of the methods, apparatuses, terminal devices, and storage media of the hotspot data detection of the present disclosure may be applied.
As shown in fig. 1, a system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The user may interact with the server 105 via the network 104 using the terminal devices 101, 102, 103 to receive or send messages or the like. Various communication client applications, such as a voice interaction type application, a video conference type application, a short video social type application, a web browser application, a shopping type application, a search type application, an instant messaging tool, a mailbox client, social platform software, etc., may be installed on the terminal devices 101, 102, 103.
The terminal devices 101, 102, 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, various electronic devices with microphones and speakers may be available, including but not limited to smartphones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, dynamic video expert compressed standard audio layer 3), MP4 (Moving Picture Experts Group Audio Layer IV, dynamic video expert compressed standard audio layer 4) players, portable computers and desktop computers, etc. When the terminal devices 101, 102, 103 are software, they can be installed in the above-listed electronic devices. Which may be implemented as a plurality of software or software modules, or as a single software or software module. The present invention is not particularly limited herein.
The server 105 may be a server providing various services, and for example, the server 105 may be a background server processing a detection request of hot spot data transmitted by the terminal devices 101, 102, 103.
In some cases, the method for detecting hot spot data provided by the present disclosure may be executed by the server 105, and accordingly, the device for detecting hot spot data may also be disposed in the server 105, where the system architecture 100 may not include the terminal devices 101, 102, 103.
In some cases, the method for detecting hot spot data provided by the present disclosure may be performed jointly by the terminal devices 101, 102, 103 and the server 105, and correspondingly, the detecting devices for the hot spot data may also be respectively provided in the terminal devices 101, 102, 103 and the server 105.
The server 105 may be hardware or software. When the server 105 is hardware, it may be implemented as a distributed server cluster formed by a plurality of servers, or as a single server. When server 105 is software, it may be implemented as a plurality of software or software modules (e.g., to provide distributed services), or as a single software or software module. The present invention is not particularly limited herein.
It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
FIG. 2 illustrates a flow chart of one embodiment of a method of detecting hotspot data according to the present disclosure. The method in this embodiment may be implemented by the server in fig. 1 or by the terminal device and the server in fig. 1 together.
In this embodiment, the hot spot phenomenon may refer to a phenomenon in which data (for example, a certain service or a certain method) is largely called in a short time. Hotspot data may refer to data that is accessed or invoked by a large number of requests in a short time.
As shown in fig. 2, the method 200 for detecting hot spot data includes the following steps:
In this embodiment, the target data may be data that is requested to be accessed or invoked. The request record for the target data within the target time period may be generated from the data request for the target data within the target time period. Wherein it may be determined whether a data request is a data request for target data based on whether a data identifier contained in the data request matches a data identifier of the target data.
In alternative embodiments, the target time period may be the last period of time, such as the last 10 seconds, the last 20 seconds, or the last 60 seconds. In an alternative embodiment, the target time period may be a time slice (the time-sharing operating system allocates a microscopic period of CPU time to each running process).
In this embodiment, the target period may be divided into a plurality of time windows according to the set time length, each time window having the above-described set time length. On this basis, the request frequency of the target data in each time window can be counted.
Fig. 3A shows a schematic diagram of a time window of a method of detection of hotspot data according to the present disclosure. As shown in fig. 3A, in this example, the time slice (i.e., the target time period) has a length of 10 seconds, and the time window has a set length of 2 seconds, so that the time slice may be divided into 5 time windows arranged from left to right in the figure, where the time window on the rightmost side is the nearest time window. On this basis, the request frequency of the target data in each time window is counted. For example, for target data with a data identification (i.e., id) of 20, the request frequency within each time window is 199, 100, 600, 400, and 500, respectively. For another example, for target data with a data identification (i.e., id) of 12, the request frequency within each time window is 20, 30, 50, 100, and 250, respectively.
In this embodiment, when the request frequency of the target data in the latest time window meets a preset condition, that is, is greater than a first frequency threshold (may also be referred to as a prediction threshold) and less than a second frequency threshold (may also be referred to as a hot spot threshold), trend prediction is performed on the target data. The second frequency threshold is used for preventing the request frequency from being too large, and ensuring that hot spot data are detected in advance before the request frequency reaches a peak value. The first frequency threshold is used for preventing the basic request frequency from being too small in trend prediction, so that the basic request frequency is combined with the upward trend to reasonably infer that the target data has a larger frequency peak value, and the accuracy of the detection result of the hot spot data is ensured. The first frequency threshold C1 is, for example, 200, and the second frequency threshold C2 is, for example, 400.
In fig. 3A, the data identifies (i.e., id) 12 target data, the request frequency in the most recent time window is 250, greater than the first frequency threshold 200 and less than the second frequency threshold 400, so that trend predictions can be made for the request frequency (i.e., time series 20, 30, 50, 100, and 250) of the target data in each time window.
In an alternative embodiment, in the case that the request frequency of the target data in the latest time window is greater than the second frequency threshold, the target data is directly determined to be hot spot data, and trend prediction is not needed. The method is beneficial to improving the operation efficiency of the detection method of the hot spot data.
In an alternative embodiment, in the case where the request frequency of the target data in the latest time window is less than the third frequency threshold (may also be referred to as the reference threshold), it is directly determined that the target data is not hot spot data, without trend prediction. Wherein the third frequency threshold is less than the first frequency threshold. The third frequency threshold C0 is, for example, 50. The method is beneficial to avoiding the waste of operation resources.
Specific values of the first frequency threshold, the second frequency threshold, and the third frequency threshold may be set in connection with actual situations, which is not limited in this disclosure.
In alternative embodiments, MK (Mann-Kendall ) trend prediction may be employed. MK trend prediction does not require samples to follow a certain distribution, nor is it disturbed by a few outliers. In MK trend prediction, H0 is originally assumed to be time series data (X1, …, xn), and is a sample with n independent random variables distributed in the same way; the alternative assumption H1 is a bilateral test, that for all k, j.ltoreq.n, and that the distribution of k.noteq.j, xk and Xj is not identical. If the original assumption is not acceptable, i.e., at the alpha confidence level, there is a significant upward or downward trend in the time series data. For statistic Z, it is an upward trend when it is greater than 0; below 0 there is a downward trend.
In other embodiments, other trend prediction methods may be employed, as the disclosure is not limited in this regard.
Fig. 3B is a schematic diagram illustrating a judgment process of the detection method of hot spot data according to the present disclosure. As shown in fig. 3B, for the request frequency of the target data in the latest time window, it is first determined whether the request frequency is greater than the reference threshold C0, if yes, a subsequent determination process is performed, and if not, it is directly determined that the target data is not hot spot data. And secondly, judging whether the request frequency is larger than a hot spot threshold C2, if so, directly determining that the target data is hot spot data, and if not, entering a subsequent judging flow. And then judging whether the request frequency is larger than a prediction threshold C1, if so, entering a subsequent judging flow, and if not, directly determining that the target data is not hot spot data. And finally, carrying out MK trend test on the time sequence of the request frequency corresponding to the target data, if the test result is an ascending trend, determining that the target data is hot spot data, and if the test result is not the ascending trend, determining that the target data is not hot spot data.
According to the method for detecting the hot spot data, the target data can be judged to be the hot spot data before the target data actually reaches the peak value, so that the hot spot data can be detected in advance.
In an alternative embodiment, it may be determined that the target data is hot spot data in the case that the trend prediction result is an upward trend. As can be appreciated, the frequency of the request of the basic target data in the latest time window is large enough (i.e. greater than the first frequency threshold), and meanwhile, the target data has an ascending trend, so that the target data can be reasonably estimated to have a large frequency peak value, and thus, the target data is hot spot data.
In this embodiment, trend prediction is performed when the request frequency of the target data meets a specific condition, and whether the target data is hot spot data is determined according to the result of the trend prediction, so that the hot spot data can be detected in advance (for example, at least one time window in advance) compared with the existing method, which is beneficial to coping with the hot spot phenomenon in advance. In addition, compared with a mode of directly lowering the threshold value of the hot spot, the embodiment can ensure the accuracy of the hot spot data detection result.
In an alternative embodiment, after step 240, the method further includes: and under the condition that the target data is hot spot data, caching the target data. In an alternative embodiment, the target data may be cached by a plurality of distributed nodes. In this case, an appropriate scheme may be selected to ensure cache data coherency.
By accurately caching hot spot data in advance, QPS of the downstream storage layer can be enabled
The query rate per second (Queries-per-second) tends to be more stable, so that the query pressure of hot spot data on a data layer is greatly reduced, and the usability of the system is improved.
In an alternative embodiment, after the target data is cached, the method further includes: receiving an access request for target data; and reading target data from the cache to feed back the access request.
Fig. 3C shows a schematic diagram of a request processing procedure of a detection method of hotspot data according to the present disclosure. As shown in fig. 3C, for a received data request, it is determined whether the data for which it is intended is hot spot data. In the event that hot spot data is not hit, the data is read from the storage layer and fed back. And under the condition of hitting the hot spot data, judging whether the hot spot data exists in the local cache. And under the condition that hot spot data exists in the local cache, reading the data from the cache and feeding back. And under the condition that hot spot data does not exist in the local cache, reading the data from the storage layer, feeding back the data, and writing the read data into the local cache.
Fig. 3D illustrates a schematic diagram of one specific example of a method of detecting hotspot data according to the present disclosure. As shown in fig. 3D, the interested party performs an interface embedding to count the request records of the target data in the target time period. The request records are reported to the hotspot data detection module in an asynchronous manner. The hot spot data detection module divides time windows of the target time period and counts the request frequency of the target data in each time window. On this basis, the hot spot data detection module performs heat calculation according to the judgment process of fig. 3B, for example, and outputs the identification of the heat data. And then, reading the data from the storage layer according to the identification of the heat data and caching the data. With distributed storage, data consistency needs to be maintained during caching. When a new data request hits the hot spot data, the data is read from the cache and fed back.
Fig. 4 illustrates an exemplary block diagram of a detection apparatus for hotspot data according to an embodiment of the present disclosure. As shown in fig. 4, the apparatus 400 for detecting hot spot data includes: an obtaining module 410, configured to obtain a request record of the target data in the target time period; a dividing module 420, configured to divide the target time period into a plurality of time windows, and determine a request frequency of the target data in a latest time window according to the request record; the prediction module 430 is configured to perform trend prediction on the request frequency of the target data in a plurality of time windows to obtain a corresponding trend prediction result when the request frequency of the target data in the nearest time window is greater than the first frequency threshold and less than the second frequency threshold; the determining module 440 is configured to determine whether the target data is hot spot data according to the trend prediction result.
It should be appreciated that the various modules of the apparatus 400 shown in fig. 4 may correspond to the various steps in the method 200 described with reference to fig. 2. Thus, the operations, features and advantages described above with respect to method 200 apply equally to apparatus 400 and the modules comprised thereby. For brevity, certain operations, features and advantages are not described in detail herein.
In an alternative embodiment, the apparatus 400 further includes a caching module configured to: and carrying out caching processing on the target data under the condition that the target data are hot spot data.
In an alternative embodiment, the cache module is further configured to: and caching the target data through a plurality of distributed nodes.
In an alternative embodiment, the determining module 440 is further configured to: and determining the target data as hot spot data under the condition that the request frequency of the target data in the latest time window is larger than a second frequency threshold value.
In an alternative embodiment, the determining module 440 is further configured to: and determining that the target data is not the hot spot data under the condition that the request frequency of the target data in the latest time window is smaller than a third frequency threshold, wherein the third frequency threshold is smaller than the first frequency threshold.
In an alternative embodiment, the prediction module 430 is further configured to: and carrying out MK trend prediction on the request frequency of the target data in a plurality of time windows.
In an alternative embodiment, the determining module 440 is further configured to: and determining that the target data is hot spot data under the condition that the trend prediction result is an ascending trend.
In an alternative embodiment, the apparatus 400 further comprises a request processing module for: receiving an access request for the target data; and reading the target data from the cache so as to feed back the access request.
Fig. 5 illustrates a schematic block diagram of an example electronic device 500 that may be used to implement embodiments of the present disclosure. Referring to fig. 5, a block diagram of an electronic device 500 that may be a server or a client of the present disclosure, which is an example of a hardware device that may be applied to aspects of the present disclosure, will now be described. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein. As shown in fig. 5, the electronic device 500 includes a computing unit 501 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 502 or a computer program loaded from a storage unit 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, ROM 502, and RAM 503 are connected to each other by a bus 504. An input/output (I/O) interface 505 is also connected to bus 504. Various components in the device 500 are connected to the I/O interface 505, including: an input unit 706 such as a keyboard, a mouse, etc.; an output unit 507 such as various types of displays, speakers, and the like; a storage unit 508 such as a magnetic disk, an optical disk, or the like; and a communication unit 509 such as a network card, modem, wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
The computing unit 501 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 501 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the detection method of hotspot data. For example, in some embodiments, the method of detecting hotspot data may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 500 via the ROM 502 and/or the communication unit 509. When the computer program is loaded into RAM 503 and executed by computing unit 501, one or more steps of the above-described method of detecting hotspot data may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the method of detection of hotspot data in any other suitable manner (e.g., by means of firmware).
The various illustrative logics, logical blocks, modules, circuits, and algorithm processes described in connection with the aspects disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. The interchangeability of hardware and software has been described generally in terms of functionality, and is illustrated in the various illustrative components, blocks, modules, circuits, and processes described above. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system.
The hardware and data processing apparatus used to implement the various illustrative logics, logical blocks, modules, and circuits described in connection with the aspects disclosed herein may be implemented or performed with a general purpose single or multi-chip processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor or any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. In some aspects, particular processes and methods may be performed by circuitry specific to a given function.
In one or more aspects, the functions described may be implemented in hardware, digital electronic circuitry, computer software, firmware (including the structures disclosed in this specification and their equivalents), or in any combination thereof. Aspects of the subject matter described in this specification can also be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on a computer storage medium for execution by, or to control the operation of, data processing apparatus.
If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. The processes of the methods or algorithms disclosed herein may be implemented in software modules executable by a processor, which may reside on a computer readable medium. Computer-readable media includes both computer storage media and communication media including any medium that can transfer a computer program from one place to another. A storage media may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Further, any connection is properly termed a computer-readable medium. Disk (Disk) and disc (Disk) as used herein include high-density optical discs (CDs), laser discs, optical discs, digital Versatile Discs (DVDs), floppy disks, and blu-ray discs where disks (disks) usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media. Additionally, the operations of a method or algorithm may be embodied as one or any combination or set of codes and instructions on a machine-readable medium and computer-readable medium, which may be incorporated into a computer program product.
The various embodiments in this disclosure are described in a related manner, and identical and similar parts of the various embodiments are all referred to each other, and each embodiment is mainly described in terms of differences from the other embodiments. In particular, for apparatus embodiments, device embodiments, computer-readable storage medium embodiments, and computer program product embodiments, the description is relatively simple, as relevant to the method embodiments in part.
Claims (12)
1. A method for detecting hot spot data comprises the following steps:
acquiring a request record of target data in a target time period;
dividing the target time period into a plurality of time windows, and determining the request frequency of the target data in the latest time window according to the request record;
under the condition that the request frequency of the target data in the latest time window is larger than a first frequency threshold and smaller than a second frequency threshold, carrying out trend prediction on the request frequency of the target data in a plurality of time windows to obtain a corresponding trend prediction result;
and determining whether the target data is hot spot data according to the trend prediction result.
2. The method of claim 1, wherein after the determining whether the target data is hot spot data based on the trend prediction result, the method further comprises:
and carrying out cache processing on the target data under the condition that the target data are hot spot data.
3. The method of claim 2, wherein the caching the target data comprises:
and caching the target data through a plurality of distributed nodes.
4. The method of claim 1, wherein after the determining the request frequency of the target data within the most recent time window from the request record, the method further comprises:
and under the condition that the request frequency of the target data in the latest time window is larger than a second frequency threshold value, determining that the target data is hot spot data.
5. The method of claim 4, wherein after the determining the request frequency of the target data within the most recent time window from the request record, the method further comprises:
and determining that the target data is not hot spot data under the condition that the request frequency of the target data in the latest time window is smaller than a third frequency threshold value, wherein the third frequency threshold value is smaller than the first frequency threshold value.
6. The method of claim 1, wherein trend predicting the frequency of requests for the target data over a plurality of time windows comprises:
and carrying out MK trend prediction on the request frequency of the target data in a plurality of time windows.
7. The method of claim 6, wherein the determining whether the target data is hot spot data based on the trend prediction result comprises:
and under the condition that the trend prediction result is an ascending trend, determining that the target data is hot spot data.
8. The method of claim 2, wherein after caching the target data, the method further comprises:
receiving an access request for the target data;
and reading the target data from the cache so as to feed back the access request.
9. A device for detecting hotspot data, comprising:
the acquisition module is used for acquiring a request record of target data in a target time period;
the dividing module is used for dividing the target time period into a plurality of time windows and determining the request frequency of the target data in the latest time window according to the request record;
the prediction module is used for carrying out trend prediction on the request frequency of the target data in a plurality of time windows under the condition that the request frequency of the target data in the latest time window is larger than a first frequency threshold and smaller than a second frequency threshold, so as to obtain a corresponding trend prediction result;
and the judging module is used for determining whether the target data is hot spot data according to the trend prediction result.
10. A computer program product comprising program code instructions which, when the program product is executed by a computer, cause the computer to carry out the method of at least one of claims 1-8.
11. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of at least one of claims 1-8.
12. An electronic device, comprising:
the processor may be configured to perform the steps of,
a memory in electronic communication with the processor; and
instructions stored in the memory and executable by the processor to cause the electronic device to perform the method according to at least one of claims 1-8.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211710359.9A CN116155783A (en) | 2022-12-29 | 2022-12-29 | Hot spot data detection method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211710359.9A CN116155783A (en) | 2022-12-29 | 2022-12-29 | Hot spot data detection method and device, storage medium and electronic equipment |
Publications (1)
Publication Number | Publication Date |
---|---|
CN116155783A true CN116155783A (en) | 2023-05-23 |
Family
ID=86353554
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202211710359.9A Pending CN116155783A (en) | 2022-12-29 | 2022-12-29 | Hot spot data detection method and device, storage medium and electronic equipment |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN116155783A (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116975585A (en) * | 2023-09-25 | 2023-10-31 | 中国人民解放军军事科学院国防科技创新研究院 | Method and device for formalized representation of computable instant advantage window |
-
2022
- 2022-12-29 CN CN202211710359.9A patent/CN116155783A/en active Pending
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116975585A (en) * | 2023-09-25 | 2023-10-31 | 中国人民解放军军事科学院国防科技创新研究院 | Method and device for formalized representation of computable instant advantage window |
CN116975585B (en) * | 2023-09-25 | 2023-12-15 | 中国人民解放军军事科学院国防科技创新研究院 | Method and device for formalized representation of computable instant advantage window |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109005056B (en) | CDN application-based storage capacity evaluation method and device | |
US7251649B2 (en) | Method for prioritizing content | |
CN109299144B (en) | Data processing method, device and system and application server | |
CN105205014A (en) | Data storage method and device | |
US10649905B2 (en) | Method and apparatus for storing data | |
CN110442762B (en) | Big data processing method based on cloud platform big data | |
CN108471385B (en) | Flow control method and device for distributed system | |
CN116155783A (en) | Hot spot data detection method and device, storage medium and electronic equipment | |
CN111984203B (en) | Data deduplication method and device, electronic equipment and storage medium | |
CN111131841A (en) | Live indirect access method and device, electronic equipment and storage medium | |
KR100671635B1 (en) | Service management using multiple service location managers | |
KR102225100B1 (en) | Method and apparatus for blocking obscene images based on DNA using artificial intelligence | |
CN109150792B (en) | Method and device for improving data storage security | |
AU2020226900B2 (en) | Adaptive retrieval of objects from remote storage | |
KR20220055661A (en) | Edge service processing system and control method thereof | |
CN117119209A (en) | Caching method and device for live broadcast back source data, storage medium and electronic equipment | |
CN116405726B (en) | Data storage control method, system and readable storage medium based on EMMC abrasion degree | |
CN117714410A (en) | Screenshot resource deployment method and device, storage medium and electronic equipment | |
CN106941451B (en) | A kind of file intelligent buffer method based on network aware and coverage rate threshold matrix | |
CN117119208A (en) | Live broadcast return source scheduling method and device, storage medium and electronic equipment | |
CN118631956A (en) | Video frame extraction method and device, storage medium and electronic equipment | |
CN116132701A (en) | Video bullet screen management method and device, storage medium and electronic equipment | |
CN114650252B (en) | Routing method and device based on enterprise service bus and computer equipment | |
CN118870081A (en) | Video processing method and device, storage medium and electronic equipment | |
CN117195849A (en) | Identification adding method and device, storage medium and electronic equipment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |