CN109407995B - Cold and hot data management method and device, electronic equipment and computer readable medium - Google Patents
Cold and hot data management method and device, electronic equipment and computer readable medium Download PDFInfo
- Publication number
- CN109407995B CN109407995B CN201811307696.7A CN201811307696A CN109407995B CN 109407995 B CN109407995 B CN 109407995B CN 201811307696 A CN201811307696 A CN 201811307696A CN 109407995 B CN109407995 B CN 109407995B
- Authority
- CN
- China
- Prior art keywords
- heap
- data
- value
- cold
- hot
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
- G06F3/0649—Lifecycle management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0653—Monitoring storage devices or systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The disclosure provides a cold and hot data management method and device, electronic equipment and a computer readable medium, and belongs to the technical field of data storage. The cold and hot data management method comprises the following steps: acquiring cold and hot values of data in a memory and data in a hard disk; establishing a first top pile according to the cold and hot values of the data in the memory, and establishing a second top pile according to the cold and hot values of the data in the hard disk; and responding to the situation that the heap top value of the first top heap is smaller than the heap top value of the second top heap, and exchanging data corresponding to the heap top value and the heap top value in the first top heap and the second top heap. The method respectively constructs top heaps aiming at cold and hot values of data in the memory and the hard disk, exchanges the top heaps and the data by monitoring the top heaps and if the exchange conditions are met, realizes the bidirectional flow of the data in the memory and the hard disk, and ensures the high efficiency of data service.
Description
Technical Field
The present disclosure relates generally to the field of data storage technologies, and in particular, to a method and an apparatus for managing cold and hot data, an electronic device, and a computer-readable medium.
Background
Currently, in the data storage technology, for cold and hot data (a general term for cold data and hot data, data is layered based on a specified hit rate before being stored in a memory or a hard disk, data with a cold heat value higher than a reference value is hot data, and conversely, data with a cold heat value lower than the reference value is cold data), a processing mode of storing the hot data through the memory is generally that the hard disk stores the cold data, the memory writes the cold and hot data into the hard disk, and the data only flows from the memory to the hard disk in a single direction.
The above information disclosed in this background section is only for enhancement of understanding of the background of the disclosure and therefore it may contain information that does not constitute prior art that is already known to a person of ordinary skill in the art.
Disclosure of Invention
The present disclosure provides a method, an apparatus, an electronic device, and a computer-readable medium for cold and hot data management, which solve at least one of the above problems.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows, or in part will be obvious from the description, or may be learned by practice of the disclosure.
According to an aspect of the present disclosure, there is provided a cold and hot data management method, including: acquiring cold and hot values of data in a memory and data in a hard disk; establishing a first top pile according to the cold and hot values of the data in the memory, and establishing a second top pile according to the cold and hot values of the data in the hard disk; and responding to the situation that the heap top value of the first top heap is smaller than the heap top value of the second top heap, and exchanging data corresponding to the heap top value and the heap top value in the first top heap and the second top heap.
In an embodiment of the present disclosure, the obtaining the hot and cold values of the data in the memory and the data in the hard disk includes:
and respectively counting the hit times of the data in the memory and the data in the hard disk according to a user to obtain the cold heat value of the data in the memory and the cold heat value of the data in the hard disk, wherein the cold heat value of the data in the memory and the cold heat value of the data in the hard disk are both 0 when being initialized.
In one embodiment of the present disclosure, before responding to the top-of-heap value of the first top heap being less than the top-of-heap value of the second top heap, further comprising:
comparing the top of pile value of the first top pile with the top of pile value of the second top pile.
In one embodiment of the present disclosure, comparing the top of pile value of the first top pile and the top of pile value of the second top pile comprises:
and comparing the top value of the first top stack with the top value of the second top stack according to a preset period.
In an embodiment of the present disclosure, the hot and cold value storage data structure of the data in the memory and the hard disk is: { cooling value, key1 of data in memory, key2 of data in hard disk }, where key1 and key2 are a complex;
the exchanging data of the heap top value in the first top heap and the heap top value in the second top heap comprises:
acquiring corresponding value values through keys 1 and 2 in the cold heat value storage data structure; exchanging the value corresponding to the key1 with the value corresponding to the key 2; key1 and key2 are exchanged.
In an embodiment of the present disclosure, after exchanging the heap top value in the first top heap and the heap top value in the second top heap, the method further includes:
and adjusting the top pile according to the size of the cold heat value of the first top pile and the second top pile after the pile top value exchange.
According to still another aspect of the present disclosure, there is provided a cold and hot data management device including:
the cold and heat value acquisition module is configured to acquire cold and heat values of data in the memory and data in the hard disk; the top heap construction module is configured to establish a first top heap according to the cold and hot values of the data in the memory and establish a second top heap according to the cold and hot values of the data in the hard disk; and the exchange module is configured to respond that the heap top value of the first top heap is smaller than the heap top value of the second top heap, and exchange data corresponding to the heap top value and the heap top value in the first top heap and the second top heap.
In one embodiment of the present disclosure, further comprising: a comparison module configured to compare a top of pile value of the first top pile with a top of pile value of the second top pile.
According to yet another aspect of the present disclosure, there is provided an electronic device comprising a processor; a memory storing instructions for the processor to control the method steps as described above.
According to another aspect of the present disclosure, there is provided a computer-readable medium having stored thereon computer-executable instructions that, when executed by a processor, implement the method steps as described above.
According to the cold and hot data management method, the device, the electronic equipment and the computer readable medium provided by the embodiment of the disclosure, on one hand, a top heap is respectively constructed for the cold and hot values of data in a memory and a hard disk, the heap top value of the top heap is monitored, and if the exchange condition is met, the heap top value and the data are exchanged, so that the bidirectional flow of the data in the memory and the hard disk is realized, and the high efficiency of data service is ensured; on the other hand, the condition that the cold and hot data are distributed unevenly in the memory and the hard disk is avoided by exchanging the data in real time.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.
Fig. 1 is a flowchart illustrating a method for managing hot and cold data according to an embodiment of the disclosure.
Fig. 2 shows a flowchart of step S110 in fig. 1 according to an embodiment of the disclosure.
Fig. 3 is a flowchart illustrating a process of completing bidirectional exchange of data between a memory and a hard disk according to an embodiment of the present disclosure.
Fig. 4 is a schematic diagram illustrating a data flow of step S302 in fig. 3 according to an embodiment of the disclosure.
Fig. 5 shows a schematic diagram of step S307 in fig. 3 according to an embodiment of the disclosure.
Fig. 6 is a schematic diagram illustrating a cold and hot data management device provided in another embodiment of the disclosure.
Fig. 7 shows a schematic structural diagram of an electronic device suitable for implementing an embodiment of the present application, provided by an embodiment of the present disclosure.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and the like. In other instances, well-known structures, methods, devices, implementations, materials, or operations are not shown or described in detail to avoid obscuring aspects of the disclosure.
Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to specific embodiments and the accompanying drawings.
In the related embodiment of the invention, cold and hot data cannot be found intelligently and can flow bidirectionally between the memory and the hard disk on the premise of storing a large amount of data. The bidirectional flow of data between the memory and the hard disk cannot be intelligently realized according to the cold and hot values, but the cold data is simply stored in the hard disk, the hot data is stored in the memory, and a channel for efficient exchange of the cold and hot data is not established.
In view of the above problems, some embodiments of the present disclosure provide a hot and cold data management method and apparatus based on HeapSort (heap sorting), an electronic device, and a computer readable medium.
Fig. 1 is a flowchart illustrating a method for managing hot and cold data provided in an embodiment of the present disclosure, including the following steps:
as shown in fig. 1, in step S110, the cooling and heating values of the data in the memory and the data in the hard disk are obtained.
As shown in fig. 1, in step S120, a first top heap is established according to the hot and cold values of the data in the memory, and a second top heap is established according to the hot and cold values of the data in the hard disk.
As shown in fig. 1, in step S130, in response to that the heap top value of the first top heap is smaller than the heap top value of the second top heap, data corresponding to the heap top value and the heap top value in the first top heap and the second top heap is exchanged.
Through the cold and hot data management method in the exemplary embodiment, on one hand, a top heap is respectively constructed for the cold and hot values of data in the memory and the hard disk, the heap top value of the top heap is monitored, and if the exchange condition is met, the heap top value and the data are exchanged, so that the bidirectional flow of the data in the memory and the hard disk is realized, and the high efficiency of data service is ensured; on the other hand, the condition that the cold and hot data are distributed unevenly in the memory and the hard disk is avoided by exchanging the data in real time.
Next, each step in the cold and hot data management method in the embodiment of the present disclosure will be further described.
In step S110, the cold heat value of the data in the memory and the data in the hard disk is obtained. The specific step flow is shown in the following fig. 2.
In an embodiment of the present disclosure, the obtaining the hot and cold values of the data in the memory and the data in the hard disk includes:
respectively storing cold and hot data into an internal memory and a hard disk, and initializing cold and hot values of the data in the internal memory, wherein the cold and hot values of the data in the internal memory and the cold and hot values of the data in the hard disk are 0 when the data in the internal memory and the data in the hard disk are initialized; and then respectively counting according to the hit times of the user on the data in the memory and the data in the hard disk to obtain the cold heat value of the data in the memory and the cold heat value of the data in the hard disk.
Fig. 2 shows a flowchart of step S110 in fig. 1, which specifically includes the following steps:
as shown in fig. 2, in step S201, cold and hot data are allocated to a memory and a hard disk according to a preset reference value, and the cold and hot values of the data in the memory are initialized, where the cold and hot values of the data in the memory and the cold and hot values of the data in the hard disk are both 0 when initialized.
In an embodiment of the present disclosure, a specific value of the preset reference value may be changed according to a specific application scenario and a requirement, for example, for an application scenario where data access is more frequent, the preset reference value may be set to be slightly larger, and for an application scenario where data access is not very frequent, the preset reference value may be set to be slightly smaller.
As shown in fig. 2, in step S202, the cold heat value of the data in the memory and the cold heat value of the data in the hard disk are obtained according to the hit times of the data in the memory and the data in the hard disk counted by the user.
In step S120, a first top heap is established according to the hot and cold values of the data in the memory, and a second top heap is established according to the hot and cold values of the data in the hard disk.
In an embodiment of the present disclosure, in this step, the first top heap is a top heap for storing cold heat values of data in the memory, the second top heap is a top heap for storing cold heat values of data in the hard disk, and usually the number of hits of data in the memory is greater than the number of hits of data in the hard disk, that is, the number of cold heat values of data in the memory should be greater than the number of cold heat values of data in the hard disk, so that the first top heap established by the cold heat values of data in the memory is a small top heap (Min heapstart, that is, the minimum top heap value), and the second top heap of the cold heat values of data in the hard disk is a large top heap (Max heapstart, that is, the maximum top heap value).
In one embodiment of the present disclosure, the step of establishing the first top heap according to the hot and cold values of the data in the memory includes:
firstly, acquiring a value with the minimum cold and hot value of the data in the memory as a heap top value of the small top heap;
secondly, the cold and hot values of the data in the memory, which are larger than the heap top value, are respectively used as child nodes of the heap top value of the small top heap to construct and generate the small top heap.
In one embodiment of the present disclosure, the step of establishing the second top stack according to the hot and cold values of the data in the hard disk includes:
firstly, acquiring the maximum value of the cold and hot values of the data in the hard disk as the top value of the large top stack;
secondly, the cold and hot values of the data in the hard disk, which are smaller than the heap top value, are respectively used as child nodes of the large top heap to construct and generate the large top heap.
In step S120, a heapstart is respectively maintained in the memory and the hard disk according to the hot and cold values of the data, so that the hot and cold degrees of the data in the memory and the hard disk can be monitored in real time according to the heapstart, and the stack top value of the first top stack and the stack top value of the second top stack are compared.
In one embodiment of the present disclosure, the method specifically includes:
and comparing the top value of the first top stack with the top value of the second top stack according to a preset period.
The preset period may be set as required, and is not limited herein. The heap top values of the two top heaps are periodically checked so as to periodically exchange when an exchange mechanism is met, the reduction of the throughput of the whole system caused by real-time exchange can be avoided, and the timeliness of partial data is sacrificed to replace the stability of the throughput of the whole system.
Because the heap top value of the first top heap is the value with the minimum cold and hot degree of the data in the memory, and the heap top value of the second top heap is the value with the maximum cold and hot degree of the data in the hard disk, whether the data in the memory is cold (namely the hit rate is reduced) or not or whether the data in the hard disk is hot (namely the hit rate is increased) can be judged only by comparing the size of the two heap top values.
In step S130, in response to that the heap top value of the first top heap is smaller than the heap top value of the second top heap, data corresponding to the heap top value and the heap top value in the first top heap and the second top heap is exchanged.
In one embodiment of the present disclosure, the data exchange mechanism is configured as follows: when the Heapsort top value of the cold heat value of the memory data (the stack top value of the first top stack, namely the value with the minimum cold heat value in the memory) is smaller than the Heapsort top value of the cold heat value of the hard disk data (the stack top value of the second top stack, namely the value with the maximum cold heat value in the hard disk), the exchange mechanism is triggered.
The cold and hot value storage data structure of the data in the memory and the hard disk is as follows: { cold heat value, key1 of data in memory, key2 of data in hard disk }, where key1 and key2 are a complex, so that in step S140, once the exchange mechanism is triggered, the cold heat value itself is exchanged first, and then the data of the top value in the first top heap and the data of the top value in the second top heap are exchanged, where the process of data exchange specifically includes:
acquiring corresponding value values through keys 1 and 2 in the cold heat value storage data structure; exchanging the value corresponding to the key1 with the value corresponding to the key 2; key1 and key2 are exchanged.
In an embodiment of the present disclosure, after exchanging the heap top value in the first top heap and the heap top value in the second top heap, the method further includes:
and adjusting the top pile according to the size of the cold heat value by combining a pile algorithm for the first top pile and the second top pile after the pile top value exchange. For example, the two top heaps obtained after the heap top value swap may not both be able to comply with the rules in the respective top heaps, and if the heap top value in the first top heap after swap is not the minimum, the first top heap may need to be adjusted accordingly according to the rule of the minimum top heap.
Fig. 3 shows a flowchart of this embodiment for completing bidirectional exchange of data in the memory and the hard disk, which specifically includes the following steps:
as shown in fig. 3, in step S301, data is loaded and initialized, and the specific steps refer to step S201 and step S202 described above.
As shown in fig. 3, in step S302, data in the memory and/or the hard disk is hit according to the user request.
Fig. 4 is a schematic diagram illustrating a data flow of step S302 in fig. 3, and as shown in fig. 4, a user request is received, data in a memory and/or a hard disk is accessed according to the user request, and then the data is returned to the user.
As shown in fig. 3, in step S303, a cooling heat value is generated according to data hit by the user request.
The process specifically comprises the following steps:
the cold heat Value A1 of data in the memory is initialized to 0, and every time a Key1 hits, the Value of A1 is added with 1, and then the Key1 is stored in the memory as > Value 1. Sorting the cold and hot values of the data in the memory by using Min Heapsort, wherein the heap top value is represented by Q;
similarly, the cold heat Value B1 of data in the hard disk is initialized to 0, and each time Key2 hits, the Value of B1 is incremented by 1, and then Key2 ═ Value2 is stored in the hard disk (LevelDb). And sorting the cold and hot values of the data in the hard disk by using Max Heapsort, wherein the top value of the stack is represented by P.
As shown in fig. 3, in step S304, a timing scan is performed at a preset cycle.
As shown in FIG. 3, in step S305, it is determined whether the swap mechanism is triggered, wherein the swap mechanism is Q < P, if Q < P is true, then go to step S306, and if Q < P is false, then go to step S304 without triggering the swap mechanism.
As shown in fig. 3, in step S306, the exchange mechanism is triggered.
As shown in fig. 3, in step S307, the cold heat values corresponding to P and Q and the data corresponding to the cold heat values are exchanged.
Thus, the data exchange between the memory and the hard disk is completed, and the process returns to step S302.
Fig. 5 shows a schematic diagram of the principle of step S307, and as shown in fig. 5, the first top heap constructed in the memory uses a1 as a heap top Value, a1 corresponds to the hot and cold values Key1 and Value1 of the data in the memory, the lower child nodes are a2, A3, a4, a5, a6, and a7, respectively, the second top heap constructed in the hard disk uses B1 as the heap top Value, the lower child nodes are B2, B3, B4, B5, B6, and B7, and B1 corresponds to the hot and cold values Key2 and Value2 of the data in the memory. If the swap mechanism is triggered, top value swap, i.e. A1 and B1 swap, is performed, and the top heap is adjusted after swap, i.e. the top heap trees in the memory and the hard disk are adjusted respectively.
The technical problem to be solved by the technical scheme provided by the disclosure is how to efficiently transfer cold and hot data between an internal memory and a hard disk, and the real-time exchange of the data is completed by respectively maintaining a Heapsort for the internal memory and the hard disk data, wherein the cold heat value of the internal memory data maintains a small top stack (Min Heapsort), and the cold heat value of the hard disk data maintains a large top stack (Max Heapsort), and comparing the top values of the two stacks in real time.
In summary, in the cold and hot data management method provided in this embodiment, on one hand, top heaps are respectively constructed for cold and hot values of data in the memory and the hard disk, and the top heap value of the top heap is monitored, and if an exchange condition is satisfied, the top heap value and the data are exchanged, so that bidirectional flow of the data in the memory and the hard disk is realized, and high efficiency of data service is ensured; on the other hand, the condition that the cold and hot data are distributed unevenly in the memory and the hard disk is avoided by exchanging the data in real time.
Fig. 6 is a schematic diagram illustrating a cold and hot data management apparatus provided in another embodiment of the present disclosure, and as shown in fig. 6, the apparatus 600 includes: a cold heat value acquisition module 610, a top stack construction module 620, and an exchange module 630.
The cold heat value obtaining module 610 is configured to obtain the cold heat values of the data in the memory and the data in the hard disk; the top heap construction module 620 is configured to establish a first top heap according to the hot and cold values of the data in the memory, and establish a second top heap according to the hot and cold values of the data in the hard disk; the swapping module 630 is configured to, in response to the heap top value of the first top heap being smaller than the heap top value of the second top heap, swap data corresponding to the heap top value and the heap top value in the first top heap and the second top heap.
It should be noted that, the apparatus 600 further includes a comparing module configured to compare the top value of the first top stack with the top value of the second top stack, and send the comparison result to the exchanging module, so that the exchanging module can exchange the top values of the two top stacks when determining that the exchange mechanism is satisfied.
The functions of each module in the apparatus are described in the above method embodiments, and are not described again here.
In summary, in the cold and hot data management apparatus in this embodiment, on one hand, top heaps are respectively constructed for cold and hot values of data in the memory and the hard disk, and the top heap value of the top heap is monitored, and if an exchange condition is satisfied, the top heap value and the data are exchanged, so that bidirectional flow of the data in the memory and the hard disk is realized, and high efficiency of data service is ensured; on the other hand, the condition that the cold and hot data are distributed unevenly in the memory and the hard disk is avoided by exchanging the data in real time.
In another aspect, the present disclosure also provides an electronic device, including a processor and a memory, where the memory stores operating instructions for the processor to control the following method:
acquiring cold and hot values of data in a memory and data in a hard disk; establishing a first top pile according to the cold and hot values of the data in the memory, and establishing a second top pile according to the cold and hot values of the data in the hard disk; and correspondingly, the heap top value of the first top heap is smaller than the heap top value of the second top heap, and data corresponding to the heap top value and the heap top value in the first top heap and the second top heap are exchanged.
Referring now to FIG. 7, shown is a block diagram of a computer system 700 suitable for use in implementing the electronic device of an embodiment of the present application. The electronic device shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 7, the computer system 700 includes a Central Processing Unit (CPU)701, which can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. In the RAM703, various programs and data necessary for the operation of the system 700 are also stored. The CPU 701, the ROM 702, and the RAM703 are connected to each other via a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
The following components are connected to the I/O interface 705: an input portion 706 including a keyboard, a mouse, and the like; an output section 707 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 708 including a hard disk and the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. A drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a hard disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read out therefrom is mounted into the storage section 708 as necessary.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 709, and/or installed from the removable medium 711. The computer program executes the above-described functions defined in the system of the present application when executed by the Central Processing Unit (CPU) 701.
It should be noted that the computer readable medium shown in the present application may be a computer readable signal medium or a computer readable medium or any combination of the two. A computer readable medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present application may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor includes a transmitting unit, an obtaining unit, a determining unit, and a first processing unit. The names of these units do not in some cases constitute a limitation to the unit itself, and for example, the sending unit may also be described as a "unit sending a picture acquisition request to a connected server".
In another aspect, the present disclosure also provides a computer-readable medium, which may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to include the method steps of:
acquiring cold and hot values of data in a memory and data in a hard disk; establishing a first top pile according to the cold and hot values of the data in the memory, and establishing a second top pile according to the cold and hot values of the data in the hard disk; and responding to the situation that the heap top value of the first top heap is smaller than the heap top value of the second top heap, and exchanging data corresponding to the heap top value and the heap top value in the first top heap and the second top heap.
It should be clearly understood that this disclosure describes how to make and use particular examples, but the principles of this disclosure are not limited to any details of these examples. Rather, these principles can be applied to many other embodiments based on the teachings of the present disclosure.
Exemplary embodiments of the present disclosure are specifically illustrated and described above. It is to be understood that the present disclosure is not limited to the precise arrangements, instrumentalities, or instrumentalities described herein; on the contrary, the disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
Claims (10)
1. A method for managing cold and hot data, comprising:
acquiring cold and hot values of data in a memory and data in a hard disk;
establishing a first top heap according to the cold and hot values of data in the memory, wherein the first top heap is a small top heap; establishing a second top heap according to the cold and hot values of data in the hard disk, wherein the second top heap is a large top heap;
acquiring the minimum value of the cold and hot values of the data in the memory as the heap top value of the small top heap, and acquiring the maximum value of the cold and hot values of the data in the hard disk as the heap top value of the large top heap;
and responding to the situation that the heap top value of the first top heap is smaller than the heap top value of the second top heap, and exchanging data corresponding to the heap top value and the heap top value in the first top heap and the second top heap.
2. A method for managing cold and hot data according to claim 1, wherein the obtaining of the cold and hot values of the data in the memory and the data in the hard disk includes:
and respectively counting the hit times of the data in the memory and the data in the hard disk according to a user to obtain the cold heat value of the data in the memory and the cold heat value of the data in the hard disk, wherein the cold heat value of the data in the memory and the cold heat value of the data in the hard disk are both 0 when being initialized.
3. A method of cold-hot data management according to claim 1, in response to the heap top value of the first top heap being less than the heap top value of the second top heap, further comprising:
comparing the top of pile value of the first top pile with the top of pile value of the second top pile.
4. A method of cold-hot data management according to claim 3, wherein comparing the heap top value of the first top heap with the heap top value of the second top heap comprises:
and comparing the top value of the first top stack with the top value of the second top stack according to a preset period.
5. A cold and hot data management method according to claim 1, wherein the cold and hot value storage data structures of the data in the memory and the hard disk are: { cooling value, key1 of data in memory, key2 of data in hard disk }, where key1 and key2 are a complex;
the exchanging data of the heap top value in the first top heap and the heap top value in the second top heap comprises:
acquiring corresponding value values through keys 1 and 2 in the cold heat value storage data structure;
exchanging the value corresponding to the key1 with the value corresponding to the key 2;
key1 and key2 are exchanged.
6. A method for cold-hot data management according to claim 1, further comprising, after exchanging the heap top value in the first top heap and the heap top value in the second top heap:
and adjusting the top pile according to the size of the cold heat value of the first top pile and the second top pile after the pile top value exchange.
7. A cold-hot data management device, comprising:
the cold and heat value acquisition module is configured to acquire cold and heat values of data in the memory and data in the hard disk;
the top heap construction module is configured to establish a first top heap according to the cold and hot values of data in the memory, wherein the first top heap is a small top heap, and establish a second top heap according to the cold and hot values of the data in the hard disk, and the second top heap is a large top heap; acquiring the minimum value of the cold and hot values of the data in the memory as the heap top value of the small top heap, and acquiring the maximum value of the cold and hot values of the data in the hard disk as the heap top value of the large top heap;
and the exchange module is configured to respond that the heap top value of the first top heap is smaller than the heap top value of the second top heap, and exchange data corresponding to the heap top value and the heap top value in the first top heap and the second top heap.
8. A hot and cold data management apparatus according to claim 7, further comprising:
a comparison module configured to compare a top of pile value of the first top pile with a top of pile value of the second top pile.
9. An electronic device, comprising:
a processor;
memory storing instructions for the processor to control the method steps of any of claims 1-6.
10. A computer-readable medium having stored thereon computer-executable instructions, which when executed by a processor, perform the method steps of any one of claims 1-6.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811307696.7A CN109407995B (en) | 2018-11-05 | 2018-11-05 | Cold and hot data management method and device, electronic equipment and computer readable medium |
| CA3060855A CA3060855A1 (en) | 2018-11-05 | 2019-11-01 | Method and device for managing hot and cold data, electronic device, and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811307696.7A CN109407995B (en) | 2018-11-05 | 2018-11-05 | Cold and hot data management method and device, electronic equipment and computer readable medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109407995A CN109407995A (en) | 2019-03-01 |
| CN109407995B true CN109407995B (en) | 2020-03-27 |
Family
ID=65471750
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811307696.7A Active CN109407995B (en) | 2018-11-05 | 2018-11-05 | Cold and hot data management method and device, electronic equipment and computer readable medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN109407995B (en) |
| CA (1) | CA3060855A1 (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114138183B (en) * | 2021-10-29 | 2023-09-12 | 郑州云海信息技术有限公司 | Data layering method, device and equipment of storage system and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101533335A (en) * | 2009-04-09 | 2009-09-16 | 杭州华三通信技术有限公司 | Disk storage system and method for controlling disk storage and device thereof |
| CN102821113A (en) * | 2011-06-07 | 2012-12-12 | 阿里巴巴集团控股有限公司 | Cache method and system |
| CN103400598A (en) * | 2007-08-14 | 2013-11-20 | 三星电子株式会社 | Solid state memory, computer system including same, and method of operating same |
| CN104156317A (en) * | 2014-08-08 | 2014-11-19 | 浪潮(北京)电子信息产业有限公司 | Wiping and writing management method and system for non-volatile flash memory |
| CN105373350A (en) * | 2015-11-23 | 2016-03-02 | 联想(北京)有限公司 | Data management method and device |
| CN105447059A (en) * | 2014-09-29 | 2016-03-30 | 华为技术有限公司 | Data processing method and device |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150324447A1 (en) * | 2014-05-08 | 2015-11-12 | Altibase Corp. | Hybrid database management system and method of managing tables therein |
| CN107391301A (en) * | 2017-08-16 | 2017-11-24 | 北京奇虎科技有限公司 | Data managing method, device, computing device and the storage medium of storage system |
-
2018
- 2018-11-05 CN CN201811307696.7A patent/CN109407995B/en active Active
-
2019
- 2019-11-01 CA CA3060855A patent/CA3060855A1/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103400598A (en) * | 2007-08-14 | 2013-11-20 | 三星电子株式会社 | Solid state memory, computer system including same, and method of operating same |
| CN101533335A (en) * | 2009-04-09 | 2009-09-16 | 杭州华三通信技术有限公司 | Disk storage system and method for controlling disk storage and device thereof |
| CN102821113A (en) * | 2011-06-07 | 2012-12-12 | 阿里巴巴集团控股有限公司 | Cache method and system |
| CN104156317A (en) * | 2014-08-08 | 2014-11-19 | 浪潮(北京)电子信息产业有限公司 | Wiping and writing management method and system for non-volatile flash memory |
| CN105447059A (en) * | 2014-09-29 | 2016-03-30 | 华为技术有限公司 | Data processing method and device |
| CN105373350A (en) * | 2015-11-23 | 2016-03-02 | 联想(北京)有限公司 | Data management method and device |
Non-Patent Citations (2)
| Title |
|---|
| 基于一致性哈希的分层混合存储系统设计与实现;李晶;《万方数据库》;20180626;全文 * |
| 基于堆排序的片上Nor Flash损耗均衡硬件快速实现;徐书韬 等;《计算机工程与科学》;20171130;第39卷(第11期);第1971-1979页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CA3060855A1 (en) | 2020-05-05 |
| CN109407995A (en) | 2019-03-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110365752A (en) | Processing method, device, electronic equipment and the storage medium of business datum | |
| CN110365491A (en) | Method for processing business, device, equipment, storage medium and data-sharing systems | |
| CN111078436B (en) | Data processing method, device, equipment and storage medium | |
| US20230052935A1 (en) | Asynchronous accounting method and apparatus for blockchain, medium and electronic device | |
| CN112261094A (en) | Message processing method and proxy server | |
| US20230068770A1 (en) | Federated model training method and apparatus, electronic device, computer program product, and computer-readable storage medium | |
| CN112804675A (en) | Wireless communication core network and method for analyzing movement of user equipment therein | |
| CN113014452A (en) | Network flow testing method, device, testing end and storage medium | |
| CN109407995B (en) | Cold and hot data management method and device, electronic equipment and computer readable medium | |
| US20210132993A1 (en) | Rate limiting compliance assessments with multi-layer fair share scheduling | |
| US12020125B2 (en) | Method, electronic device, and computer program product for information processing | |
| CN106131162A (en) | A kind of method realizing network service agent based on IOCP mechanism | |
| CN113630476B (en) | Communication method and communication device applied to computer cluster | |
| CN115665239A (en) | Message subscription method, system, electronic device and storage medium based on Netty framework | |
| CN109583182A (en) | Start method, apparatus, electronic equipment and the computer storage medium of remote desktop | |
| CN115033904A (en) | Data processing method, apparatus, system, medium, and product | |
| US11070654B2 (en) | Sockets for shared link applications | |
| CN114172887A (en) | Intelligent center control system for exhibition hall | |
| CN107707383B (en) | Put-through processing method and device, first network element and second network element | |
| CN112040149A (en) | Input source switching method and device of screen projector, screen projector and storage medium | |
| CN111797169A (en) | Data storage method, equipment and storage medium | |
| CN113568936B (en) | Real-time stream data storage method, device and terminal equipment | |
| CN112508199B (en) | Feature selection method and device for cross-feature federal learning and related equipment | |
| CN113918603B (en) | Hash buffer generation method and device, electronic equipment and storage medium | |
| CN114629735B (en) | State interaction method, device, equipment and medium based on multiparty state channel |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |