WO2021184996A1 - Procédé et appareil de stockage de données pour une base de données - Google Patents

Procédé et appareil de stockage de données pour une base de données Download PDF

Info

Publication number
WO2021184996A1
WO2021184996A1 PCT/CN2021/075501 CN2021075501W WO2021184996A1 WO 2021184996 A1 WO2021184996 A1 WO 2021184996A1 CN 2021075501 W CN2021075501 W CN 2021075501W WO 2021184996 A1 WO2021184996 A1 WO 2021184996A1
Authority
WO
WIPO (PCT)
Prior art keywords
memory
data
database
log
sorting
Prior art date
Application number
PCT/CN2021/075501
Other languages
English (en)
Chinese (zh)
Inventor
熊刚
许友松
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2021184996A1 publication Critical patent/WO2021184996A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G06F16/219—Managing data history or versioning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors

Definitions

  • This application relates to the field of communication technology, and in particular to a data storage method and device for a database.
  • the files involved in the database such as the log that records data changes in the database, the sorted files generated by the execution of efflux, etc., are usually placed on the same disk, such as all on the local disk or on the cloud disk.
  • RDMA remote direct memory access
  • the present application provides a data storage method and device for a database, so as to provide an efficient and low-maintenance database data storage method.
  • the embodiments of the present application provide a data storage method for a database, which is executed by a data processing device.
  • the data processing device can record a first log when a data change in the database is detected.
  • a log is preferentially stored in the first memory.
  • the data processing device determines that the storage space in the first memory is insufficient, for example, less than the first threshold, it can switch to the second memory to continue recording the first log, the first memory and the second memory Deploy in different devices.
  • the log of the database is no longer limited to the local storage.
  • the storage space in the first storage is insufficient, it can be switched to other storage to continue to store the log, making the data storage method of the database more efficient and multiple storages can be deployed In different devices, large local disks are no longer needed, which can effectively reduce the disk maintenance cost of the database.
  • the sorting file generated by the database during the sorting process may also be stored in a similar manner.
  • the data processing device when the data processing device receives a database sorting request, it can sort the data in the database according to the received database sorting request; the data processing device can also generate a sorting file in the process of sorting the data.
  • the sorted file may be stored in the first memory, and when the data processing apparatus determines that the storage space of the first memory is less than the second threshold, it may switch to the second memory and continue to save the second memory.
  • the above method is only an example of receiving a database sorting request.
  • the data processing device will also sort the data in the database to generate a sorting file.
  • the sorting file can also be stored in the above-mentioned manner.
  • the sorting files generated during the data sorting process in the database can be stored in multiple memories, which improves the data storage efficiency of the database, and the multiple memories are distributed and deployed, reducing the maintenance cost of the local disk.
  • extended memory and memory can also be set.
  • the extended memory and memory can be used to store data with a higher read and write frequency in the database.
  • the extended memory and memory can be deployed in the first memory. It may also be another memory independent of the first memory.
  • the data processing device may store data in the database with a read and write frequency greater than the third threshold in the extended memory based on the read and write frequency of the data in the database; when the data processing device needs to read the data in the extended memory , The data can be read from the extended memory to the internal memory, and the data processing device can read the data from the internal memory.
  • the setting of the extended memory can effectively expand the storage space of the memory, so that more data can be stored in the extended memory, and ensure that these data can be efficiently read and written.
  • the data processing device can also eliminate the data with low read and write frequency in the extended memory in time.
  • the data processing device first detects the read and write frequency of the data in the extended memory; if the read and write frequency is stored in the extended memory When the data is lower than the third threshold, the data whose read and write frequency is lower than the third threshold in the extended memory can be eliminated.
  • the data processing device can eliminate the cold data in the extended memory in time, and effectively utilize the storage space in the extended memory.
  • the data processing device may also back up the first log stored in the first storage and the second storage. After the first log is backed up, the first log may be cleared; afterwards, if the database is If there is a data change, the data processing device may continue to record the second log of the database in the first memory according to the data change of the database.
  • the log when there is free storage space in the first storage (such as clearing the first log), the log can be continued to be stored in the first storage to effectively utilize the storage space of the first storage.
  • the data processing device may delete the sorting file after sorting the data in the database according to the sorting file.
  • the data processing device deletes the sorted files in time, which can ensure that there is free storage space in the first memory and the second memory so as to store other valid data.
  • the data processing apparatus may also record the correspondence between the data page and the data page identifier in the extended memory. For example, save the correspondence in the non-volatile memory, and when the device where the first memory is located is restarted Afterwards, the data processing device may reorganize the extended memory in the first memory according to the corresponding relationship.
  • the data processing device can quickly organize and expand the memory after the device is restarted.
  • the embodiments of the present application also provide a data processing device, and the beneficial effects can be referred to the description of the first aspect and will not be repeated here.
  • the device has the function of realizing the behavior in the method example of the first aspect described above.
  • the function can be realized by hardware, or by hardware executing corresponding software.
  • the hardware or software includes one or more modules corresponding to the above-mentioned functions.
  • the structure of the device includes a processing unit, a determining unit, and a switching unit. These units can perform the corresponding functions in the above-mentioned method example of the first aspect. For details, please refer to the detailed description in the method example. Do repeat.
  • an embodiment of the present application also provides a computing device.
  • the computing device includes a processor and a memory, and may also include a communication interface.
  • the processor executes the program instructions in the memory to execute the above-mentioned first aspect or
  • the memory is coupled with the processor and stores program instructions and data necessary to perform data synchronization.
  • the communication interface is used to communicate with other devices (such as client devices).
  • the present application provides a computing device system, which includes at least one computing device.
  • Each computing device includes a memory and a processor.
  • the processor of at least one computing device may be used to access the code in the memory to execute the first aspect or the method provided in any possible implementation manner of the first aspect.
  • the present application provides a non-transitory readable storage medium.
  • the non-transitory readable storage medium executes the foregoing first aspect or any of the first aspects. Possible implementation.
  • the storage medium stores the program.
  • the storage medium includes, but is not limited to, volatile memory, such as random access memory, non-volatile memory, such as flash memory, hard disk drive (HDD), and solid state drive (SSD).
  • the present application provides a computing device program product.
  • the computing device program product includes computer instructions. When executed by a computing device, the computing device can execute the foregoing first aspect or any possible aspect of the first aspect. Method to realize.
  • the computer program product may be a software installation package. In the case where the method provided in the foregoing first aspect or any possible implementation of the first aspect needs to be used, the computer program product may be downloaded and executed on a computing device. Program product.
  • Figure 1 is a schematic diagram of the architecture of a system provided by this application.
  • FIG. 2 is a schematic diagram of another system structure provided by this application.
  • FIG. 3 is a schematic diagram of another system structure provided by this application.
  • FIG. 4 is a schematic diagram of a data storage method for a database provided by this application.
  • FIG. 5 is a schematic structural diagram of a data processing device provided by this application.
  • FIG. 6 is a schematic diagram of a computing device provided by an embodiment of this application.
  • FIG. 7 is a schematic diagram of a computing device in a computing device system provided by an embodiment of the application.
  • FIG. 1 it is a schematic diagram of a system structure provided by an embodiment of this application.
  • the system includes a data processing apparatus 100, a first memory 200 and a second memory 300.
  • the first memory 200 may include a volatile memory (volatile memory), such as a random access memory (random access memory, RAM).
  • volatile memory such as a random access memory (random access memory, RAM).
  • the first memory may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (solid disk drive). -state drive, SSD).
  • NVM non-volatile memory
  • ROM read-only memory
  • flash memory flash memory
  • HDD hard disk drive
  • solid state drive solid disk drive
  • SSD solid disk drive
  • the first memory may also include a combination of the above types.
  • the first memory 200 is similar to the second memory 300.
  • the first storage 200 and the second storage 300 are deployed in different data centers and located in different devices.
  • the first storage 200 is a local disk
  • the second storage 300 is a cloud disk.
  • the first memory 200 and the second memory 300 are used to store data in the database and related files of the database, such as logs (such as physical logs and logical logs), sort files, and so on.
  • logs such as physical logs and logical logs
  • sort files such as sort files, and so on.
  • the first storage 200 may also include an extended memory and a memory, and the extended memory and the memory are used to store data with a relatively high read and write frequency in the database.
  • the data processing device 100 may be a hardware device, such as a server, a terminal computing device, etc., or a software device, specifically a set of software systems running on a hardware computing device.
  • the embodiment of the present application does not limit the location where the data processing device 100 is deployed. Exemplarily, as shown in FIG.
  • the data processing apparatus 100 may run in a cloud computing device system (including at least one cloud computing device, such as a server, etc., in the embodiment of the present application, the cloud computing device system is a data center), It can also run on an edge computing device system (including at least one edge computing device, such as a server, a desktop computer, etc., the edge computing device system in the embodiment of this application is a data center), or it can run on various terminal computing devices, For example: notebook computers, personal desktop computers, etc.
  • a cloud computing device system including at least one cloud computing device, such as a server, etc., in the embodiment of the present application, the cloud computing device system is a data center
  • an edge computing device system including at least one edge computing device, such as a server, a desktop computer, etc., the edge computing device system in the embodiment of this application is a data center
  • various terminal computing devices For example: notebook computers, personal desktop computers, etc.
  • the data processing device 100 can also be a device composed of multiple parts logically.
  • the data processing device 100 can include a processing unit, a determining unit, and a switching unit.
  • Each component of the data processing device 100 can be deployed in different systems. Or in the server.
  • each part of the device can run in three environments of cloud computing equipment system, edge computing equipment system or terminal computing equipment respectively, and can also run in any two of these three environments. middle.
  • the cloud computing equipment system, the edge computing equipment system and the terminal computing equipment are connected by a communication path, which can communicate and transmit data with each other.
  • the data storage method for the database provided by the embodiment of the present application is executed by the combined parts of the data processing apparatus 100 running in three environments (or any two of the three environments).
  • the first memory 200 can be deployed in the same system or in the same hardware device as the data processing device 100, so that the data processing device 100 can read data from the first memory 200 more efficiently.
  • This application does not limit the second memory 300.
  • the deployment location only needs to ensure that the first storage 200 and the second storage 300 are deployed in different data centers.
  • the data processing device 100 can generate a log according to the change status of the data in the database.
  • the free storage space in the first storage 200 is greater than the first threshold, the generated log is preferentially stored in the first storage 200 As the log is generated and stored, the storage space in the first storage 200 gradually decreases.
  • the data processing device 100 switches to the second storage 300 to continue storing the log. It can be seen from the above that the logs can be stored in at least two different first memories 200, which can effectively improve the utilization of different memories and ensure that the logs of the database can be completely saved.
  • the following is a data storage method for a database provided by an embodiment of the present application with reference to FIG. 4, and the method includes:
  • Step 401 The data processing device 100 detects the data change of the database.
  • Data changes in the database include but are not limited to: insert, delete, and update of data. Among them, data insertion refers to adding new data to the database, data deletion refers to deleting data in the database, and data updating refers to changing one data in the database to another data.
  • Step 402 The data processing device 100 generates a first log according to the data change of the database.
  • the log is used to record the data changes sent by the database of the recorder, such as data insertion, deletion, and update.
  • the data processing device 100 will generate a corresponding log after checking the data change. In this embodiment of the application, it is distinguished from the log generated by the subsequent data processing device 100.
  • the generated log is called the first log.
  • Step 403 When the free storage space in the first memory 200 is not less than the first threshold, the data processing apparatus 100 stores the first log in the first memory 200.
  • Step 404 When the data processing device 100 determines that the free storage space in the first storage 200 is less than the first threshold, it switches to the second storage 300 to continue storing the first log.
  • the storage order of the logs is the first storage 200 -> the second storage 300.
  • the data processing device 100 preferentially saves the first log in the first memory 200, and when the free storage space in the first memory 200 is insufficient (for example, less than the first threshold), it switches to the second memory 300 to continue storing the first log.
  • a fixed file is configured for storing logs.
  • the file can be called a log file.
  • the size of the log file is fixed, usually a preset value.
  • the data processing device 100 generates When the size of the first log reaches the preset value, the database cannot continue to make data changes. At this time, the first log needs to be backed up. After the backup is completed, the first log can be cleared, which means that the first log stored in the first storage 200 and the second storage 300 are deleted.
  • the data processing device 100 can continue to generate a log.
  • the log generated by the data processing device 100 is referred to herein as the second log, which is similar to the first log.
  • the data processing device 100 preferentially saves the second log in the first memory 200, and when the free storage space of the first memory 200 is insufficient, it switches to the second memory 300 to continue to store the second memory 300.
  • the storage method of the log in the first storage 200 and the second storage 300 is introduced.
  • the data processing device 100 may also use a similar storage method for other data in the database (such as sorted files).
  • the data processing device 100 may sort the data in the database according to the received request to feed back the sorted data.
  • a sorting file is generated, and the sorting file records the sequence of the sorted data.
  • the sort file can also record the storage location of the data and the index number of the data.
  • the sorted file can be stored in the first storage 200 first. As the sorted file is stored in the first storage 200, the free storage space in the first storage 200 becomes less and less. When the free storage space in the first storage 200 is insufficient (for example, less than the second threshold), it is switched to the second storage 300 to continue storing the sorted files.
  • the sorting file can be deleted.
  • the storage order of the sorted files is the first memory 200 -> the second memory 300, and the first memory 200 is first stored in the first memory 200, and the second memory 300 is second.
  • the data processing device 100 when the data processing device 100 acrobatics receives a data query request, in order to be able to query the data processing device 100 more quickly, it can also query the data in the database.
  • the sorting is performed to generate a sorting file.
  • the storage method of the sorting file is similar to the foregoing method, and will not be repeated here.
  • the data processing device 100 can store logs and sorted files in the first memory 200 and the second memory 300, and this data storage method is more flexible for data storage and effectively expands the data storage space.
  • the first storage 200 may be a local disk of the database
  • the second storage 300 is deployed in a cloud computing device system, that is, the second storage 300 is a cloud disk
  • related files or data of the database may be extended to Cloud storage
  • the files involved in the database can be distributed on local disks and cloud disks.
  • This storage method combines the scalability of cloud disks and can also effectively save maintenance costs.
  • the embodiment of the present application only takes two memories included in the system as an example. In some scenarios, a larger number of memories may be included, and the location of data storage is more flexible, and the utilization rate of the memory can also be effectively improved. .
  • the embodiment of the present application can also expand the memory (buffer pool) in the device where the database is deployed, for example, an extended memory (extend buffer pool) is set in the first memory.
  • the read and write frequency of data in the database is usually different.
  • the memory can place the read and write frequency in the database. Higher data.
  • the frequency of reading and writing data in the memory is relatively high, and the data processing device 100 can obtain data from the memory first, which can effectively improve the read and write data of the data, and further improve the processing efficiency of the data processing device 100.
  • the memory space is usually limited, and it is not possible to store all data with higher read and write frequency in the memory.
  • the data stored in the memory may also be updated, which will eliminate some data with higher read and write frequency, resulting in data processing devices.
  • 100 needs to load these data from the second memory 300, and for this purpose, an extended memory can be added.
  • the extended memory can store data whose data read/write frequency is greater than the third threshold, in order to make the extended memory store data whose data read/write frequency is greater than the third threshold.
  • the data processing device 100 can periodically update the data in the extended memory, store data with a data read and write frequency greater than the third threshold in the extended memory, and can also eliminate data stored in the extended memory with a read and write frequency lower than the third threshold.
  • the data processing device 100 migrates data into the extended memory, it can first locate the data to be migrated.
  • the data processing device 100 may use the data in the internal memory as the data to be migrated in.
  • the data in the database is usually organized in fixed-size data units, such as data pages.
  • the size of data stored in each data page (page) is the same, and each data page is configured with an identity (ID).
  • ID is configured with an identity (ID).
  • the data processing device 100 can realize rapid positioning of the data page through the identification of each data page in the memory.
  • a hash map may be used to organize the identification of the data page and the data page, and the data processing apparatus 100 may be based on the hash table and according to the data The identification of the page locates to the data page.
  • the data processing device 100 When the data processing device 100 eliminates data from the extended memory, it can first locate the data to be moved out. For example, the data processing apparatus 100 can use a page replacement algorithm, such as least recently used (LRU), to determine which data page that the extended memory finds is less used, and the data in the data page is the data to be migrated out.
  • LRU least recently used
  • the data processing device 100 When the data processing device 100 needs to read data in the database, it can first obtain the data from the memory or the extended memory.
  • the extended memory further expands the amount of data that the memory can store, ensuring more read and write frequencies.
  • the data of can be stored in the first memory, which can improve the efficiency of data reading and writing.
  • the data processing apparatus 100 may also record the corresponding relationship between the data page in the extended memory and the identifier of the data page, and store the corresponding relationship in the non-volatile memory.
  • the first device restarts At the time, you can directly call the previously saved correspondence to reorganize the extended memory so that you can quickly locate the data page stored in the extended memory.
  • an embodiment of the present application also provides a data processing device, which is configured to execute the method performed by the data processing device 100 in the method embodiment shown in FIG. 4.
  • the data processing device 500 includes a processing unit 501, a determining unit 502, and a switching unit 503, and the aforementioned modules may be software modules.
  • a connection is established between units through a communication path.
  • the processing unit 501 is configured to record the first log of the database in the first memory according to the data change of the database.
  • the processing unit 501 may execute steps 401 to 403 as shown in FIG. 4.
  • the determining unit 502 is configured to determine that the storage space of the first memory is less than a first threshold.
  • the determining unit 502 may execute the method of determining that the storage space of the first memory is less than the first threshold in step 404 shown in FIG. 4.
  • the switching unit 503 is configured to switch the processing unit 501 to the first log of the second memory recording database after the determining unit 502 determines that the storage space of the first memory is less than the first threshold, where the first memory and the second memory are deployed and In different devices.
  • the switching unit 503 may execute the method of exactly switching to the second memory to record the first log in step 404 as shown in FIG. 4.
  • the device further includes a sorting unit 504.
  • the sorting unit 504 can sort the data in the database according to the received database sorting request; the processing unit 501 saves the sorting file generated during the sorting process in the first memory;
  • the switching unit 503 may cause the processing unit 501 to switch to the second memory to save the second memory.
  • the first memory includes extended memory and memory
  • the processing unit 501 can store the data in the database with a read and write frequency greater than the third threshold in the read and write frequency of the data in the database, and read the data in the extended memory to the memory.
  • the processing unit 501 may also eliminate data whose read/write frequency is lower than the third threshold in the extended memory.
  • the processing unit 501 may also clear the first log after the first log is backed up; according to the data change of the database, the first log is changed in the first memory. Record the second log in the database.
  • the sorting file may be deleted after the sorting unit 504 finishes sorting the data in the database according to the sorting file.
  • the processing unit 501 may also record the corresponding relationship between the data page and the data identifier in the extended memory, and reorganize the extended memory according to the corresponding relationship after the device where the first memory is located is restarted.
  • the division of modules in the embodiments of this application is illustrative, and it is only a logical function division. In actual implementation, there may be other division methods.
  • the functional modules in the various embodiments of this application can be integrated into one process. In the device, it can also exist alone physically, or two or more modules can be integrated into one module.
  • the above-mentioned integrated modules can be implemented in the form of hardware or software functional modules.
  • the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of the present application essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , Including several instructions to enable a terminal device (which may be a personal computer, a mobile phone, or a network device, etc.) or a processor to execute all or part of the steps of the method in each embodiment of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disks or optical disks and other media that can store program codes. .
  • the computing device 600 is shown in FIG. 6.
  • the computing device 600 includes a bus 601, a processor 602, a communication interface 603, and a memory 604.
  • the processor 602, the memory 604, and the communication interface 603 communicate through a bus 601.
  • the processor 602 may be a central processing unit (CPU).
  • the memory 604 may include a volatile memory (volatile memory), such as a random access memory (random access memory, RAM).
  • the memory 604 may also include a non-volatile memory (non-volatile memory), such as read-only memory (ROM), flash memory, HDD or SSD.
  • Executable code is stored in the memory, and the processor 602 executes the executable code to execute the aforementioned data storage method (the method shown in FIG. 4).
  • the memory 604 may also include an operating system and other software modules required for running processes.
  • the operating system can be LINUX TM , UNIX TM , WINDOWS TM etc.
  • the memory 604 stores the modules in the aforementioned data processing apparatus 500.
  • the memory 604 may also include other software modules required for running processes such as an operating system.
  • the operating system can be LINUX TM , UNIX TM , WINDOWS TM etc.
  • the present application also provides a computing device system.
  • the computing device system includes at least one computing device 700 as shown in FIG. 7.
  • the computing device 700 includes a bus 701, a processor 702, a communication interface 703, and a memory 704.
  • the processor 702, the memory 704, and the communication interface 703 communicate through a bus 701.
  • At least one computing device 700 in the computing device system communicates with each other through a communication path.
  • the processor 702 may be a CPU.
  • the memory 704 may include volatile memory, such as random access memory.
  • the memory 704 may also include non-volatile memory, such as read-only memory, flash memory, HDD or SSD.
  • Executable code is stored in the memory 704, and the processor 702 executes the executable code to perform any part or all of the aforementioned data synchronization method.
  • the memory may also include an operating system and other software modules required for running processes.
  • the operating system can be LINUX, UNIX, WINDOWS TM, etc.
  • any one or any multiple modules of the aforementioned data processing apparatus 500 are stored in the memory 704.
  • the memory 704 may also include other software modules required for running processes, such as an operating system.
  • the operating system can be LINUX, UNIX, WINDOWS TM, etc.
  • At least one computing device 700 in the computing device system establishes communication with each other through a communication network, and each computing device runs any one or any multiple units in the data processing apparatus 500. At least one computing device 700 collectively performs the aforementioned data storage operation.
  • the computer program product for data synchronization includes one or more computer program instructions for data synchronization.
  • the data synchronization program according to the embodiment of the present invention is generated in whole or in part. Process or function.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • the computer instructions may be transmitted from a website, computer, server, or data center. Transmission to another website, computer, server, or data center via wired (such as coaxial cable, optical fiber, digital subscriber line, or wireless (such as infrared, wireless, microwave, etc.)).
  • the computer-readable storage medium includes A readable storage medium of computer program instructions for classification model training.
  • the computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center integrated with one or more available media.
  • the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, an SSD).

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Quality & Reliability (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

L'invention concerne un procédé et un appareil de stockage de données pour une base de données, ceux-ci étant utilisés pour fournir un moyen de stockage de données, qui est efficace et a de faibles coûts de maintenance, pour une base de données. Le procédé comprend les étapes suivantes : lorsqu'il est détecté que des données dans une base de données changent (401), un appareil de traitement de données enregistre un premier journal (402) ; le premier journal est de préférence stocké dans une première mémoire (403) ; et lorsque l'appareil de traitement de données détermine que l'espace de stockage dans la première mémoire est insuffisant, par exemple, inférieur à une première valeur de seuil, commuter vers une seconde mémoire pour continuer à enregistrer le premier journal (404), la première mémoire et la seconde mémoire étant déployées dans différents dispositifs. Un journal d'une base de données n'est plus confiné dans une mémoire locale, et lorsque l'espace de stockage dans une première mémoire est insuffisant, une commutation vers une autre mémoire peut être effectuée pour continuer à stocker le journal, de telle sorte que la manière dont la base de données stocke des données est plus efficace. De multiples mémoires peuvent être déployées dans différents dispositifs, et aucun disque local de grande taille n'est nécessaire, de telle sorte que les coûts de maintenance de disque de la base de données peuvent être efficacement réduits.
PCT/CN2021/075501 2020-03-20 2021-02-05 Procédé et appareil de stockage de données pour une base de données WO2021184996A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010203414.X 2020-03-20
CN202010203414.XA CN113495883A (zh) 2020-03-20 2020-03-20 一种针对数据库的数据存储方法及装置

Publications (1)

Publication Number Publication Date
WO2021184996A1 true WO2021184996A1 (fr) 2021-09-23

Family

ID=77769685

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/075501 WO2021184996A1 (fr) 2020-03-20 2021-02-05 Procédé et appareil de stockage de données pour une base de données

Country Status (2)

Country Link
CN (1) CN113495883A (fr)
WO (1) WO2021184996A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115391098A (zh) * 2022-08-25 2022-11-25 北京有竹居网络技术有限公司 大数据分析方法、装置、边缘节点及云端服务器
CN115658328A (zh) * 2022-12-07 2023-01-31 摩尔线程智能科技(北京)有限责任公司 用于管理存储空间的装置及方法、计算设备、芯片
CN116744168A (zh) * 2022-09-01 2023-09-12 荣耀终端有限公司 日志存储方法及相关装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080148270A1 (en) * 2006-12-15 2008-06-19 International Business Machines Corporation Method and implementation for storage provisioning planning
CN103838521A (zh) * 2014-02-28 2014-06-04 华为技术有限公司 一种数据处理方法及装置
CN104539669A (zh) * 2014-12-17 2015-04-22 中国电子科技集团公司第十五研究所 一种基于移动终端的数据同步方法
US9465899B2 (en) * 2013-03-15 2016-10-11 Freescale Semiconductor, Inc. Method for provisioning decoupling capacitance in an integrated circuit
CN110413684A (zh) * 2018-04-25 2019-11-05 武汉海康存储技术有限公司 一种数据库同步方法、装置及系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080148270A1 (en) * 2006-12-15 2008-06-19 International Business Machines Corporation Method and implementation for storage provisioning planning
US9465899B2 (en) * 2013-03-15 2016-10-11 Freescale Semiconductor, Inc. Method for provisioning decoupling capacitance in an integrated circuit
CN103838521A (zh) * 2014-02-28 2014-06-04 华为技术有限公司 一种数据处理方法及装置
CN104539669A (zh) * 2014-12-17 2015-04-22 中国电子科技集团公司第十五研究所 一种基于移动终端的数据同步方法
CN110413684A (zh) * 2018-04-25 2019-11-05 武汉海康存储技术有限公司 一种数据库同步方法、装置及系统

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115391098A (zh) * 2022-08-25 2022-11-25 北京有竹居网络技术有限公司 大数据分析方法、装置、边缘节点及云端服务器
CN116744168A (zh) * 2022-09-01 2023-09-12 荣耀终端有限公司 日志存储方法及相关装置
CN116744168B (zh) * 2022-09-01 2024-05-14 荣耀终端有限公司 日志存储方法及相关装置
CN115658328A (zh) * 2022-12-07 2023-01-31 摩尔线程智能科技(北京)有限责任公司 用于管理存储空间的装置及方法、计算设备、芯片
CN115658328B (zh) * 2022-12-07 2023-10-03 摩尔线程智能科技(北京)有限责任公司 用于管理存储空间的装置及方法、计算设备、芯片

Also Published As

Publication number Publication date
CN113495883A (zh) 2021-10-12

Similar Documents

Publication Publication Date Title
WO2021184996A1 (fr) Procédé et appareil de stockage de données pour une base de données
CN108319654B (zh) 计算系统、冷热数据分离方法及装置、计算机可读存储介质
US9268711B1 (en) System and method for improving cache performance
CN108268219B (zh) 一种处理io请求的方法及装置
US10372687B1 (en) Speeding de-duplication using a temporal digest cache
US10839016B2 (en) Storing metadata in a cuckoo tree
CN108733306B (zh) 一种文件合并方法及装置
US11513996B2 (en) Non-disruptive and efficient migration of data across cloud providers
US20150142755A1 (en) Storage apparatus and data management method
US9268693B1 (en) System and method for improving cache performance
US9367256B2 (en) Storage system having defragmentation processing function
WO2019001521A1 (fr) Procédé de stockage de données, dispositif de stockage, client et système
CN110618789B (zh) 一种重复数据的删除方法及装置
US8572338B1 (en) Systems and methods for creating space-saving snapshots
US9268696B1 (en) System and method for improving cache performance
CN112684975B (zh) 一种数据存储方法及装置
US11593312B2 (en) File layer to block layer communication for selective data reduction
US11513739B2 (en) File layer to block layer communication for block organization in storage
US10614036B1 (en) Techniques for de-duplicating data storage systems using a segmented index
US10795596B1 (en) Delayed deduplication using precalculated hashes
CN113625952A (zh) 一种对象存储方法、装置、设备及存储介质
KR20200078426A (ko) 복구 서버 및 컴퓨터 프로그램
US8886883B1 (en) System and method for improving cache performance
US9208098B1 (en) System and method for improving cache performance
WO2021185059A1 (fr) Procédé et appareil de migration de données pour une base de données

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21770494

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21770494

Country of ref document: EP

Kind code of ref document: A1