CN114528231A - Data dynamic storage method and device, electronic equipment and storage medium - Google Patents
Data dynamic storage method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114528231A CN114528231A CN202011197039.9A CN202011197039A CN114528231A CN 114528231 A CN114528231 A CN 114528231A CN 202011197039 A CN202011197039 A CN 202011197039A CN 114528231 A CN114528231 A CN 114528231A
- Authority
- CN
- China
- Prior art keywords
- data
- storage medium
- storage
- stored
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F12/0888—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches using selective caching, e.g. bypass
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/06—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols the encryption apparatus using shift registers or memories for block-wise or stream coding, e.g. DES systems or RC4; Hash functions; Pseudorandom sequence generators
- H04L9/0643—Hash functions, e.g. MD5, SHA, HMAC or f9 MAC
Abstract
The application discloses a data dynamic storage method and device, electronic equipment and a storage medium. Wherein, the method comprises the following steps: acquiring a data storage request, wherein the data storage request carries data to be stored; determining attribute information corresponding to the data to be stored; determining a target storage medium corresponding to the data to be stored from a storage medium set according to the attribute information, wherein the storage medium set comprises at least two candidate storage media of different types; and storing the data to be stored to the target storage medium. According to the technical scheme provided by the embodiment of the application, the target storage medium for storing the data to be stored is determined when the data storage request meets the preset condition, so that the service efficiency and the hit rate of each storage medium are improved.
Description
Technical Field
The present application relates to the field of data storage, and in particular, to a method, an apparatus, a system, an electronic device, and a storage medium for dynamically storing data.
Background
With the development of the internet, a cache device of a typical CDN (Content Delivery Network) generally includes 3 storage media (a memory, a mechanical hard disk, and a solid state disk), where the memory and the solid state disk have strong performance but small capacity, and are generally used for caching hot files and small files. However, in a special time node (for example, during a peak period of the flow), under the condition that the flow model is not changed, the overall magnitude of the increase may cause the mechanical hard disk to reach a bottleneck first, and the solid state hard disk and the memory to not reach the limit, so that the hit rate of the three cache media may not change too much, and the cache pressures of the three storage media may be too large, thereby affecting the service quality.
Disclosure of Invention
In order to solve the technical problems or at least partially solve the technical problems, the application provides a data dynamic storage method, a data dynamic storage device, an electronic device and a storage medium.
According to an aspect of an embodiment of the present application, there is provided a method for dynamically storing data, including:
acquiring a data storage request, wherein the data storage request carries data to be stored;
determining attribute information corresponding to the data to be stored;
determining a target storage medium corresponding to the data to be stored from a storage medium set according to the attribute information, wherein the storage medium set comprises at least two candidate storage media of different types;
and storing the data to be stored to the target storage medium.
Further, the storage medium set includes at least two candidate storage media of the same type, and when it is determined that the data to be stored corresponds to at least two target storage media of the same type, the storing the data to be stored to the target storage media includes:
acquiring a data identifier of the data to be stored;
performing hash calculation according to the data identification to obtain a target hash value;
acquiring a target storage address corresponding to the target hash value;
determining a storage address corresponding to the target storage medium;
and storing the data to be stored to a target storage medium with the storage address matched with the target storage address.
Further, the attribute information includes at least one of: the time information of the data to be stored, the data type of the data to be stored and the data volume of the data to be stored;
the determining the target storage medium corresponding to the data to be stored from the storage medium set according to the attribute information includes:
when the attribute information meets a preset condition, determining a target storage medium corresponding to the data to be stored from a storage medium set;
the attribute information meeting the preset condition comprises at least one of the following items:
the time information belongs to a target time period;
the data type belongs to a preset data type;
the data amount is greater than or equal to a preset data amount.
Further, the determining a target storage medium corresponding to the data to be stored from the storage medium set according to the attribute information includes:
acquiring a random function corresponding to the data to be stored;
calculating a random number corresponding to the data to be stored according to the random function;
and determining a target storage medium corresponding to the data to be stored from a storage medium set according to the random number.
Further, the determining, according to the random number, a target storage medium corresponding to the data to be stored from a storage medium set includes:
determining the upper limit of the storage amount of the first candidate storage medium in the target time period in unit time;
determining the stored data amount of the first candidate storage medium in the unit time in the target time period according to the random number;
when the stored data amount is smaller than the storage amount upper limit, determining the first candidate storage medium as the target storage medium;
when the stored data amount is larger than or equal to the storage amount upper limit, selecting a second candidate storage medium corresponding to the first candidate storage medium from the storage medium set as the target storage medium, wherein the second candidate storage medium is different from the first candidate storage medium in type.
Further, the determining an upper limit of the storage amount of the first candidate storage medium per unit time in the target time period includes:
acquiring the number and the storage proportion of the random numbers;
and calculating the upper limit of the storage amount of the first candidate storage medium in the target time period in unit time according to the number of the random numbers and the storage proportion.
Further, before determining an upper limit of the storage amount of the first candidate storage medium per unit time in the target time period, the method further includes:
acquiring a historical data storage record, wherein the historical data storage record comprises historical data storage amounts of a plurality of time periods;
and taking the time period when the historical data storage amount is greater than or equal to a preset threshold value as a target time period.
Further, the determining an upper limit of the storage amount of the first candidate storage medium per unit time in the target time period includes:
acquiring a target historical data storage amount of the unit time in the target time period based on the historical data storage record;
acquiring storage performance information of the first candidate storage medium;
and calculating the upper limit of the storage amount corresponding to the first candidate storage medium in unit time according to the target historical data storage amount and the storage performance information.
According to another aspect of the embodiments of the present application, there is also provided a data dynamic storage device, including:
the device comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring a data storage request which carries data to be stored;
the determining module is used for determining attribute information corresponding to the data to be stored;
the processing module is used for determining a target storage medium corresponding to the data to be stored from a storage medium set according to the attribute information, wherein the storage medium set comprises at least two candidate storage media of different types;
and the storage module is used for storing the data to be stored to the target storage medium.
According to another aspect of the embodiments of the present application, there is also provided a storage medium including a stored program that executes the above steps when the program is executed.
According to another aspect of the embodiments of the present application, there is also provided an electronic apparatus, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus; wherein: a memory for storing a computer program; a processor for executing the program stored in the memory to execute the steps of the method.
Embodiments of the present application further provide a computer program product containing instructions, which when executed on a computer, cause the computer to perform the steps of the above method.
Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages: according to the technical scheme provided by the embodiment of the application, the target storage medium for storing the data to be stored is determined when the data storage request meets the preset condition, so that the service efficiency and the hit rate of each storage medium are improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
Fig. 1 is a flowchart of a method for dynamically storing data according to an embodiment of the present application;
fig. 2 is a flowchart of a method for dynamically storing data according to another embodiment of the present application;
fig. 3 is a flowchart of a method for dynamically storing data according to another embodiment of the present application;
FIG. 4 is a block diagram of a dynamic storage device for data according to an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application, it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments, and the illustrative embodiments and descriptions thereof of the present application are used for explaining the present application and do not constitute a limitation to the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another similar entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
In the related art, a buffer is usually used to manage various storage media, such as a memory, a solid state disk, a mechanical hard disk, and the like.
Generally, the buffer is used for managing a plurality of first candidate storage media and a plurality of second candidate storage media, where the first candidate storage media are mechanical hard disks, and the second candidate storage media are solid state disks or memories.
The buffer is originally intended to mean a RAM with a faster Access speed than a general Random Access Memory (RAM), and generally it does not use a Dynamic Random Access Memory (DRAM) as a main Memory of a system, but uses an expensive but faster static Memory (SRAM). The buffer is a primary memory existing between the main memory and the CPU, and is composed of static memory chips (SRAM), and has a relatively small capacity but a much higher speed than the main memory, close to the speed of the CPU. The buffer is a high-speed small-capacity memory between the central processing unit and the main memory in the hierarchical structure of the computer memory system, and forms a primary memory together with the main memory. The scheduling and transfer of information between the buffer and the main memory is automated by hardware.
The embodiment of the application can be applied to the buffer, and the storage media are allocated to the data to be stored according to the attribute of the data to be stored in the data storage request, so that the service efficiency and the hit rate of each storage media are improved.
The embodiment of the application provides a data dynamic storage method and device, electronic equipment and a storage medium. The method provided by the embodiment of the invention can be applied to any required electronic equipment, for example, the electronic equipment can be electronic equipment such as a server and a terminal, and the method is not particularly limited herein, and is hereinafter simply referred to as electronic equipment for convenience in description.
According to an aspect of the embodiments of the present application, an embodiment of a method for dynamically storing data is provided, and fig. 1 is a flowchart of the method for dynamically storing data provided by the embodiments of the present application, as shown in fig. 1, the method includes the following steps S11 to S14:
step S11, acquiring a data storage request, wherein the data storage request carries data to be stored;
the data storage request in the embodiment of the present application may be generated by a client, and when data to be stored exists in the client, the data storage request is generated and sent to a Cache (Cache).
Step S12, determining attribute information corresponding to the data storage request;
in the embodiment of the application, after the data storage request is received, the data to be stored carried by the data storage request is analyzed to determine the attribute information of the data to be stored. The attribute information referred to in the embodiments of the present application may be time information of data to be stored, a data type of the data to be stored, a data amount of the data to be stored, and the like.
Step S13, determining a target storage medium corresponding to the data to be stored from the storage medium set according to the attribute information, wherein the storage medium set comprises at least two candidate storage media of different types;
in the embodiment of the application, when the attribute information meets the preset condition, a target storage medium corresponding to data to be stored is determined from the storage medium set. The attribute information satisfying the preset condition includes at least one of: the time information belongs to a target time period; the data type belongs to a preset data type; the data amount is greater than or equal to a preset data amount. It is understood that the target time period may be a traffic peak time period.
Step S14, store the data to be stored to the target storage medium.
In the embodiment of the present application, the storage medium set includes at least two candidate storage media of the same type, and when it is determined that the data to be stored corresponds to at least two target storage media of the same type, the data to be stored is stored in the target storage media, including the following steps a1-a 5:
step A1, acquiring a data identifier of data to be stored;
step A2, carrying out hash calculation according to the data identification to obtain a target hash value;
step A3, obtaining a target storage address corresponding to the target hash value;
step A4, determining a storage address corresponding to a target storage medium;
step A5, storing the data to be stored to the target storage medium with the storage address matching with the target storage address.
It is understood that the data identification may be a data name or the like of the data, or a code of the data. The hashing algorithm may be one or a combination of MD5, SHA-1, SHA256, and the like. And calculating the data name by adopting a hash algorithm to obtain a target hash value, taking the target hash value as a key name, and determining a target storage address corresponding to the target hash value from a corresponding relation table of a preset hash value and a storage medium address.
According to the technical scheme provided by the embodiment of the application, the target storage medium for storing the data to be stored is determined when the data storage request meets the preset condition, so that the service capacity of each storage medium is improved, and the hit rate of each storage medium is improved.
In the embodiment of the present application, determining a target storage medium corresponding to data to be stored from a storage medium set includes the following steps B1 to B3:
step B1, acquiring a random function corresponding to the data to be stored;
step B2, calculating a random number corresponding to the data to be stored according to the random function;
and step B3, determining a target storage medium corresponding to the data to be stored from the storage medium set according to the random number.
The method for acquiring the random function corresponding to the data to be stored comprises the following steps: determining a storage parameter corresponding to the data to be stored, and determining a random function corresponding to the storage parameter according to a function generation strategy.
It should be noted that there are many different ways to generate random numbers. These methods are called random number generators. The nature of a random number is that it is generated such that the number following it is irrelevant to the number preceding it.
The main principle of random number generation is to input the provided storage parameter to a random function, and output a single random number or a plurality of random numbers (i.e., a random number sequence) by the random function. In the embodiment of the present application, first, configuration information is obtained, where the configuration information includes: the storage parameter can be a fixed value or a value set by a worker according to the current storage data volume. The random function includes at least one of: a hybrid congruence function, an iterative coring function, a multiplication congruence function, or other linear or non-linear function.
The configuration information may be stored in the server in advance, and when the configuration information needs to be acquired, the configuration information may be directly read from the server, or the configuration information may be provided by a worker. In addition, the configuration information may further include other configuration data such as a random number, remark information, and the like, and this is not limited in this exemplary embodiment.
In this embodiment of the application, the determining, in the step a3, a target storage medium corresponding to data to be stored from a storage medium set according to a random number includes the following steps C1-C4:
step C1, determining the upper limit of the storage amount of the first candidate storage medium in the target time period per unit time;
in the embodiment of the present application, the upper limit of the storage amount per unit time may be determined by presetting the number of the random numbers and the storage ratio by an operator, for example: when the number of the random numbers is 100, the range of the random numbers can be selected from 0-99, the total amount of the storage amount of the unit time is 100, and then the upper limit of the storage amount of the first candidate storage medium is determined to be 20, and the upper limit of the storage amount of the second candidate storage medium is determined to be 80 according to the total amount of the storage amount and the storage proportion of 20%.
Step C2, determining the stored data amount of the first candidate storage medium per unit time in the target time period according to the random number.
In the embodiment of the present application, the stored data amount per unit time of the first candidate storage medium in the target time period is determined according to the number of generated random numbers per unit time, for example, the number of generated random numbers per unit time is 17, and the stored data amount per unit time of the first candidate storage medium in the target time period is 17.
And step C3, when the stored data amount is less than the upper storage amount limit, determining the first candidate storage medium as the target storage medium.
As an example, the target time period is 19:00-20:00, the unit time of the time period is minutes, the time information corresponding to the data to be stored and the number of generated random numbers in the unit time are determined, and the stored data amount is determined according to the number of generated random numbers, for example, in the time period 19:10-19:11, the number of generated random numbers is 19, that is, the stored data amount is 19, which is smaller than the upper limit of the storage amount, so the first candidate storage medium is used as the target storage medium.
And step C4, when the stored data amount is larger than or equal to the upper storage amount limit, selecting a second candidate storage medium corresponding to the first candidate storage medium from the storage medium set as a target storage medium, wherein the second candidate storage medium is different from the first candidate storage medium in type.
In the embodiment of the application, when the stored data amount is greater than or equal to the storage amount upper limit, the corresponding relation between a first candidate storage medium and a second candidate storage medium which are stored in advance is acquired, and the second candidate storage medium corresponding to the first candidate storage medium is determined according to the corresponding relation.
As an example, the target time period is 19:00 to 20:00, the unit time of the time period is minutes, the time information corresponding to the data to be stored and the number of generated random numbers in the unit time are determined, and the stored data amount is determined according to the number of generated random numbers, for example, in the time period 19:10 to 19:11, the number of generated random numbers is 59, that is, the stored data amount can be obtained as 59, and the stored data amount is larger than the upper limit of the storage amount, so the second candidate storage medium corresponding to the first candidate storage medium is used as the target storage medium.
According to the embodiment of the application, the storage pressure of the mechanical hard disk is reduced by setting the upper limit of the storage amount, the mechanical hard disk is prevented from being overloaded, and meanwhile, the data exceeding the upper limit of the storage amount is stored in the memory or the solid state disk, so that the service quality of a single device is ensured.
Fig. 2 is a flowchart of a data dynamic storage method according to another embodiment of the present application, and as shown in fig. 2, the embodiment of the present application further provides a process of implementing step S13: an embodiment of a method of determining a target time period may include the steps of:
step S21, obtaining a historical data storage record, wherein the historical data storage record comprises historical data storage amount of a plurality of time periods;
in the embodiment of the application, the history data storage records may be recorded according to a period, and the history records include the daily history data storage amount in a week, where the daily history data storage amount includes the data storage amount in each time period in the day.
In step S22, a time period in which the history data storage amount is greater than or equal to a preset threshold value is set as a target time period.
In the embodiment of the application, the time period that the data storage amount is greater than or equal to the preset threshold value is determined by analyzing the data storage amount of each day, then the times of occurrence of the time period that the historical data storage amount is greater than or equal to the preset threshold value in one week are counted, and the time period that is greater than the preset times is taken as the target time period. For example: and taking the time periods 19:00-22:00 and 12:00-13:00 as target time periods when the occurrence frequency of 19:00-22:00 and the occurrence frequency of 12:00-13:00 in one week are 6 times, 5 times and are both more than the preset frequency of 4 times.
Fig. 3 is a flowchart of a data dynamic storage method according to another embodiment of the present application, and as shown in fig. 3, the embodiment of the present application further provides a process of implementing step S13: in an embodiment of a method for determining an upper limit of storage per unit time for a first candidate storage medium for a target time period, the method may comprise the steps of:
step S31, obtaining the target historical data storage quantity of the target time section based on the historical data storage record;
step S32, acquiring storage performance information of the first candidate storage medium;
in step S33, an upper limit of the storage amount corresponding to the first candidate storage medium is calculated based on the target history data storage amount and the storage performance information.
In the embodiment of the present application, the first candidate storage medium is a mechanical hard disk, and the storage performance information of the mechanical hard disk may be data throughput of the mechanical hard disk. And determining an upper storage limit according to the historical data storage and the data throughput.
As an example, in the target time periods 19:00-22:00, the target historical data storage amount in the first time period 19:00-20:00 is a, the target historical data storage amount in the second time period 20:00-21:00 is B, and the target historical data storage amount in the third time period 20:00-21:00 is C, and then a target storage index is calculated according to the data storage amount and the data throughput of each time period to determine a storage ratio, for example: the storage ratio of the first period is 20%, the storage ratio of the second period is 25%, and the storage ratio of the third period is 15%. Since the preset total storage amount is 100, the upper storage limit of each time period is determined according to the storage proportion, wherein the upper storage limit of the first candidate storage medium in the first time period is 20, the upper storage limit of the second candidate storage medium in the second time period is 25, and the upper storage limit of the third candidate storage medium in the third time period is 15.
According to the embodiment of the application, the target storage index is set, and when the traffic peak time period is in, the data volume stored in the second candidate storage medium (the solid state disk or the memory) is calculated according to the target storage index and the current data volume, so that the service quality of the first candidate storage medium (the mechanical hard disk) is ensured.
Fig. 4 is a block diagram of a dynamic data storage device provided in an embodiment of the present application, where the device may be implemented as part of or all of an electronic device through software, hardware, or a combination of the two. As shown in fig. 4, the data dynamic storage device includes:
the obtaining module 41 is configured to obtain a data storage request, where the data storage request carries data to be stored;
a determining module 42, configured to determine attribute information corresponding to data to be stored;
a processing module 43, configured to determine a target storage medium corresponding to data to be stored from a storage medium set according to the attribute information, where the storage medium set includes at least two different types of candidate storage media;
and the storage module 44 is used for storing the data to be stored to the target storage medium.
Further, the storage medium set includes at least two candidate storage media of the same type, and when it is determined that the data to be stored corresponds to at least two target storage media of the same type, the storage module 44 is configured to obtain a data identifier of the data to be stored, perform hash calculation according to the data identifier to obtain a target hash value, obtain a target storage address corresponding to the target hash value, determine a storage address corresponding to the target storage medium, and store the data to be stored to the target storage medium whose storage address matches the target storage address.
Further, the attribute information includes at least one of: time information of data to be stored, data type of the data to be stored, and data volume of the data to be stored;
the processing module 43 is configured to determine, when the attribute information meets a preset condition, a target storage medium corresponding to the data to be stored from the storage medium set;
the attribute information satisfying the preset condition includes at least one of: the time information belongs to a target time period; the data type belongs to a preset data type; the data amount is greater than or equal to a preset data amount.
Further, the processing module 43 includes:
the acquisition submodule is used for acquiring a random function corresponding to the data to be stored;
the calculation submodule is used for calculating a random number corresponding to the data to be stored according to the random function;
and the query submodule is used for determining a target storage medium corresponding to the data to be stored from the storage medium set according to the random number.
Further, the query submodule includes:
the acquisition unit is used for determining the upper limit of the storage amount of the first candidate storage medium in unit time in the target time period;
an analysis unit configured to determine, based on the random number, a stored data amount per unit time of the first candidate storage medium within the target time period;
the processing unit is used for determining the first candidate storage medium as a target storage medium when the stored data amount is smaller than the upper limit of the storage amount;
and the processing unit is further used for selecting a second candidate storage medium corresponding to the first candidate storage medium from the storage medium set as a target storage medium when the stored data amount is greater than or equal to the storage amount upper limit, wherein the second candidate storage medium is different from the first candidate storage medium in type.
Further, the acquiring unit is used for acquiring the range and the storage proportion of the random number; and calculating the upper limit of the storage amount of the first candidate storage medium in the target time period in unit time according to the range of the random numbers and the storage proportion.
Further, an obtaining unit in the query submodule in the embodiment of the application is further configured to obtain a historical data storage record, where the historical data storage record includes historical data storage amounts of a plurality of time periods; and taking the time period when the historical data storage amount is greater than or equal to a preset threshold value as a target time period.
Further, an acquisition unit configured to acquire a target history data storage amount per unit time in a target time period based on the history data storage record; acquiring storage performance information of a first candidate storage medium; and calculating the upper limit of the storage amount corresponding to the first candidate storage medium in unit time according to the target historical data storage amount and the storage performance information.
An embodiment of the present application further provides an electronic device, as shown in fig. 5, the electronic device may include: a processor 1501, a communication interface 1502, a memory 1503 and a communication bus 1504, wherein the processor 1501, the communication interface 1502 and the memory 1503 complete communication with each other through the communication bus 1504.
A memory 1503 for storing a computer program;
the processor 1501 is configured to implement the steps of the above embodiments when executing the computer program stored in the memory 1503.
The communication bus mentioned in the above terminal may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the terminal and other equipment.
The Memory may include a Random Access Memory (RAM) or a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. Alternatively, the memory may be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the Integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component.
In another embodiment provided by the present application, a computer-readable storage medium is further provided, in which instructions are stored, and when the instructions are executed on a computer, the computer is caused to execute the data dynamic storage method described in any one of the above embodiments.
In yet another embodiment provided by the present application, there is also provided a computer program product containing instructions, which when run on a computer, causes the computer to execute the data dynamic storage method described in any of the above embodiments.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., coaxial cable, fiber optic, digital subscriber line) or wirelessly (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk), among others.
The above description is only for the preferred embodiment of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application are included in the protection scope of the present application.
The above description is merely exemplary of the present application and is presented to enable those skilled in the art to understand and practice the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (11)
1. A method for dynamically storing data is characterized by comprising the following steps:
acquiring a data storage request, wherein the data storage request carries data to be stored;
determining attribute information corresponding to the data to be stored;
determining a target storage medium corresponding to the data to be stored from a storage medium set according to the attribute information, wherein the storage medium set comprises at least two candidate storage media of different types;
and storing the data to be stored to the target storage medium.
2. The method according to claim 1, wherein the storage medium set includes at least two candidate storage media of the same type, and when it is determined that the data to be stored corresponds to at least two target storage media of the same type, the storing the data to be stored to the target storage media includes:
acquiring a data identifier of the data to be stored;
performing hash calculation according to the data identification to obtain a target hash value;
acquiring a target storage address corresponding to the target hash value;
determining a storage address corresponding to the target storage medium;
and storing the data to be stored to a target storage medium with the storage address matched with the target storage address.
3. The method of claim 1, wherein the attribute information comprises at least one of: the time information of the data to be stored, the data type of the data to be stored and the data volume of the data to be stored;
the determining the target storage medium corresponding to the data to be stored from the storage medium set according to the attribute information includes:
when the attribute information meets a preset condition, determining a target storage medium corresponding to the data to be stored from a storage medium set;
the attribute information meeting the preset condition comprises at least one of the following items:
the time information belongs to a target time period;
the data type belongs to a preset data type;
the data amount is greater than or equal to a preset data amount.
4. The method according to claim 3, wherein the determining the target storage medium corresponding to the data to be stored from the storage medium set according to the attribute information comprises:
acquiring a random function corresponding to the data to be stored;
calculating a random number corresponding to the data to be stored according to the random function;
and determining a target storage medium corresponding to the data to be stored from a storage medium set according to the random number.
5. The method according to claim 4, wherein the determining the target storage medium corresponding to the data to be stored from the storage medium set according to the random number comprises:
determining the upper limit of the storage amount of the first candidate storage medium in the target time period in unit time;
determining the stored data amount of the first candidate storage medium in the unit time in the target time period according to the random number;
when the stored data amount is smaller than the storage amount upper limit, determining the first candidate storage medium as the target storage medium;
when the stored data amount is larger than or equal to the storage amount upper limit, selecting a second candidate storage medium corresponding to the first candidate storage medium from the storage medium set as the target storage medium, wherein the second candidate storage medium is different from the first candidate storage medium in type.
6. The method of claim 5, wherein determining an upper storage limit per unit time for the first candidate storage medium in the target time period comprises:
acquiring the number and the storage proportion of the random numbers;
and calculating the upper limit of the storage amount of the first candidate storage medium in the target time period in unit time according to the number of the random numbers and the storage proportion.
7. The method of claim 5, wherein prior to determining an upper storage capacity limit per unit time for the first candidate storage medium over the target time period, the method further comprises:
acquiring a historical data storage record, wherein the historical data storage record comprises historical data storage amounts of a plurality of time periods;
and taking the time period when the historical data storage amount is greater than or equal to a preset threshold value as a target time period.
8. The method of claim 7, wherein determining an upper storage limit per unit time for the first candidate storage medium in the target time period comprises:
acquiring a target historical data storage amount of the unit time in the target time period based on the historical data storage record;
acquiring storage performance information of the first candidate storage medium;
and calculating the upper limit of the storage amount corresponding to the first candidate storage medium in unit time according to the target historical data storage amount and the storage performance information.
9. A dynamic data storage device, comprising:
the device comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring a data storage request which carries data to be stored;
the determining module is used for determining attribute information corresponding to the data to be stored;
the processing module is used for determining a target storage medium corresponding to the data to be stored from a storage medium set according to the attribute information, wherein the storage medium set comprises at least two candidate storage media of different types;
and the storage module is used for storing the data to be stored to the target storage medium.
10. A storage medium, characterized in that the storage medium comprises a stored program, wherein the program is operative to perform the method steps of any of the preceding claims 1 to 8.
11. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;
a memory for storing a computer program;
a processor for performing the method steps of any one of claims 1-8 by executing a program stored on a memory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011197039.9A CN114528231A (en) | 2020-10-30 | 2020-10-30 | Data dynamic storage method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011197039.9A CN114528231A (en) | 2020-10-30 | 2020-10-30 | Data dynamic storage method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114528231A true CN114528231A (en) | 2022-05-24 |
Family
ID=81618702
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011197039.9A Pending CN114528231A (en) | 2020-10-30 | 2020-10-30 | Data dynamic storage method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114528231A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114661249A (en) * | 2022-05-26 | 2022-06-24 | 深圳高灯计算机科技有限公司 | Data storage method and device, computer equipment and storage medium |
| CN115426322A (en) * | 2022-08-23 | 2022-12-02 | 绿盟科技集团股份有限公司 | Virtual storage method and device |
-
2020
- 2020-10-30 CN CN202011197039.9A patent/CN114528231A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114661249A (en) * | 2022-05-26 | 2022-06-24 | 深圳高灯计算机科技有限公司 | Data storage method and device, computer equipment and storage medium |
| CN114661249B (en) * | 2022-05-26 | 2022-09-27 | 深圳高灯计算机科技有限公司 | Data storage method and device, computer equipment and storage medium |
| CN115426322A (en) * | 2022-08-23 | 2022-12-02 | 绿盟科技集团股份有限公司 | Virtual storage method and device |
| CN115426322B (en) * | 2022-08-23 | 2023-09-19 | 绿盟科技集团股份有限公司 | Virtual storage method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107798038B (en) | Data response method and data response equipment | |
| WO2017016423A1 (en) | Real-time new data update method and device | |
| US20150256479A1 (en) | Weighted request rate limiting for resources | |
| CN106027595A (en) | Access log processing method and system for CDN node | |
| CN108932257A (en) | The querying method and device of multi-dimensional data | |
| CN114528231A (en) | Data dynamic storage method and device, electronic equipment and storage medium | |
| US20200301944A1 (en) | Method and apparatus for storing off-chain data | |
| JP2015530666A (en) | Data indexing method and apparatus | |
| CN111443899B (en) | Element processing method and device, electronic equipment and storage medium | |
| CN111563199B (en) | Data processing method and device | |
| CN109165096B (en) | Cache utilization system and method for web cluster | |
| CN111488377A (en) | Data query method and device, electronic equipment and storage medium | |
| CN111562884B (en) | Data storage method and device and electronic equipment | |
| CN111563109B (en) | Radix statistics method, apparatus, system, device, and computer-readable storage medium | |
| CN109213774B (en) | Data storage method and device, storage medium and terminal | |
| CN111382206A (en) | Data storage method and device | |
| CN107239542A (en) | A kind of data statistical approach, device, server and storage medium | |
| US9760836B2 (en) | Data typing with probabilistic maps having imbalanced error costs | |
| CN111190861B (en) | Hot spot file management method, server and computer readable storage medium | |
| CN109245943B (en) | Fault positioning method and device | |
| CN110708361A (en) | System, method and device for determining grade of digital content publishing user and server | |
| CN108574610B (en) | Pressure testing method and device, electronic equipment and medium | |
| WO2021244473A1 (en) | Frequency control method and apparatus | |
| CN109150819A (en) | A kind of attack recognition method and its identifying system | |
| CN110535853B (en) | Video request scheduling method, device, server and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |