CN117009438A - Universal database node capacity expansion method and device based on k8s - Google Patents

Universal database node capacity expansion method and device based on k8s Download PDF

Info

Publication number
CN117009438A
CN117009438A CN202311070000.4A CN202311070000A CN117009438A CN 117009438 A CN117009438 A CN 117009438A CN 202311070000 A CN202311070000 A CN 202311070000A CN 117009438 A CN117009438 A CN 117009438A
Authority
CN
China
Prior art keywords
data
capacity expansion
database
current
backup
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311070000.4A
Other languages
Chinese (zh)
Inventor
李慧玲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Yunyingsheng Data Co ltd
Original Assignee
Hangzhou Yunyingsheng Data Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Yunyingsheng Data Co ltd filed Critical Hangzhou Yunyingsheng Data Co ltd
Priority to CN202311070000.4A priority Critical patent/CN117009438A/en
Publication of CN117009438A publication Critical patent/CN117009438A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application discloses a universal database node capacity expansion method and device based on k8s, which relate to the technical field of databases and comprise the following steps: creating a storage according to a database node capacity expansion target, and determining whether to copy data in the storage according to the type of the database; when copying data in storage, determining a data copying mode according to the current environment of the database; and carrying out data copying according to the data copying mode, and creating a computing node on the storage of which the data copying is completed so as to complete node capacity expansion. The application provides a set of universal database node capacity expansion realization scheme, so that any database can realize the node capacity expansion function through the configuration mode.

Description

Universal database node capacity expansion method and device based on k8s
Technical Field
The application relates to the technical field of databases, in particular to a k8 s-based universal database node capacity expansion method and device.
Background
The database is a stateful application, when the capacity of a node is expanded, only the computing node is often insufficient, the storage is also required to be synchronously increased, and the data of a database cluster is required to be copied in advance in the storage, otherwise, a large amount of data cannot be copied into the newly-added node in a short time, but the original k8s node expansion only solves the problems of the capacity expansion and the storage of the computing node, the expansion of the computing node is not realized, and meanwhile, the operation of copying the data by the existing database operation and maintenance management tool is often an independent realization mode for the capacity expansion node, and several kinds of copy logics are required to be realized by the database, so that the main defects are that: 1. corresponding copy logic is required to be added for each database, so that development cost is increased; 2. the specification and standardization cannot be established, and errors are easy to occur.
Disclosure of Invention
The application provides a general database node capacity expansion method based on k8s, which aims to solve the problem that the operation logic of data copying cannot be used commonly when the database node capacity is expanded in the prior art.
In order to achieve the above purpose, the present application adopts the following technical scheme:
the application discloses a universal database node capacity expansion method based on k8s, which comprises the following steps:
creating a storage according to a database node capacity expansion target, and determining whether to copy data in the storage according to the type of the database;
when the data is copied in the storage, determining a data copying mode according to the current environment of the database;
and carrying out data copying according to the data copying mode, and creating a computing node on the storage of which the data copying is completed so as to complete node capacity expansion.
Preferably, the determining the data copying mode according to the current environment of the database includes:
and determining whether the current environment of the database supports disk snapshots, if so, taking the disk snapshots as a data copying mode, and if not, restoring the backup as the data copying mode.
Preferably, the method further comprises the steps of:
judging whether the data copying is completed or not, if not, determining the current backup state of the data;
judging the current recovery state of the data when the current backup state is completed;
and when the current recovery state is finished, cleaning all temporary data generated in the data backup and recovery process.
Preferably, the current backup state includes not created, being backed up or being completed, and when the current backup state is not created, the backup is created, and when the current backup state is being backed up, the backup is waited for completion.
Preferably, the current recovery state includes not started, in recovery or completed, and when the current recovery state is not started, recovery is executed, and when the current recovery state is in recovery, recovery is waited for completion.
A k8 s-based universal database node capacity expansion device, comprising:
the creation module is used for creating a storage according to the database node capacity expansion target and determining whether to copy data in the storage according to the type of the database;
the determining module is used for determining the data copying mode according to the current environment of the database when the data is copied in the storage;
and the capacity expansion module is used for carrying out data copying according to the data copying mode, and creating a computing node on the storage of which the data copying is completed so as to complete node capacity expansion.
Preferably, the determining module includes:
and the selection unit is used for determining whether the current environment of the database supports disk snapshots, if so, taking the disk snapshots as a data copy mode, and if not, restoring the backup as the data copy mode.
Preferably, the capacity expansion module includes:
the first judging unit is used for judging whether the data copying is finished or not, and if not, determining the current backup state of the data;
the second judging unit is used for judging the current recovery state of the data when the current backup state is completed;
and the cleaning unit is used for cleaning all temporary data generated in the data backup and recovery process when the current recovery state is completed.
An electronic device comprising a memory and a processor, the memory to store one or more computer instructions, wherein the one or more computer instructions are executable by the processor to implement a k8 s-based universal database node expansion method as in any of the above.
A computer readable storage medium storing a computer program which, when executed by a computer, implements a k8 s-based universal database node expansion method as claimed in any one of the preceding claims.
The application has the following beneficial effects:
the application provides a set of universal database node capacity expansion realization scheme, so that any database can realize the node capacity expansion function through the configuration mode.
Drawings
In order to more clearly illustrate the embodiments of the application or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the description below are only some embodiments of the application, and that other drawings can be obtained according to these drawings without inventive faculty for a person skilled in the art.
FIG. 1 is a flow chart of a universal database node capacity expansion method based on k8 s;
FIG. 2 is a schematic diagram of a universal database node capacity expansion device based on k8 s;
fig. 3 is a schematic diagram of an electronic device for implementing a k8 s-based universal database node capacity expansion method.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The terms "first," "second," and the like in the claims and the description of the application, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order, and it is to be understood that the terms so used may be interchanged, if appropriate, merely to describe the manner in which objects of the same nature are distinguished in the embodiments of the application by the description, and furthermore, the terms "comprise" and "have" and any variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of elements is not necessarily limited to those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
As shown in fig. 1, the present embodiment provides a general database node capacity expansion method based on k8s, which includes the following steps:
s110, creating a storage according to a database node capacity expansion target, and determining whether to copy data in the storage according to the type of the database;
s120, when the data is copied in the storage, determining a data copying mode according to the current environment of the database;
and S130, performing data copying according to the data copying mode, and creating a computing node on the storage of which the data copying is completed so as to complete node capacity expansion.
The Kubernetes, namely k8s is a Google open-source container cluster management system, provides a series of complete functions such as deployment operation, resource scheduling, service discovery, dynamic expansion and the like for containerized application on the basis of a Docker technology, improves convenience of large-scale container cluster management, and native k8s node expansion can finish storage creation and computing node expansion.
Specifically, a storage is created according to a database node capacity expansion target, whether the data of the database needs to be copied in the created storage or not is determined according to the type of the database, whether the data needs to be copied is determined from the configuration of database characteristics or not, most of newer databases do not need to copy the data, some more classical relational databases such as mysql and postgresql need to copy the data, whether the data needs to be copied is simplified into a mark, a database manufacturer configures the data according to the type of the database, if the data does not need to be copied, a computing node is directly created on the storage, so that node capacity expansion is realized, otherwise, the optimal data copying mode is determined according to the current environment of the database, and the optional data copying modes are two, namely a disk snapshot mode and a backup recovery mode.
Further, judging whether the data copying is completed or not, if not, determining the current backup state of the data;
judging the current recovery state of the data when the current backup state is completed;
and when the current recovery state is finished, cleaning all temporary data generated in the data backup and recovery process.
Then, according to the determined data copying mode, data copying is carried out, whether the data copying is completed or not is judged, because of reentrant, the data copying can be completed when entering the step, if yes, a computing node is directly created on a storage, otherwise, the current backup state of the data is judged, and three conditions exist in the current backup state: if the backup is in the state of not being created, calling a backup API configured by the database to create the backup, if the backup is in the state of being backed up, waiting for the backup to be completed, and if the backup is in the state of being completed, judging the current recovery state of the data, wherein the current recovery state also comprises three possible: if the state is not started, the recovery API configured by the database starts to recover, if the state is not started, the recovery API is configured by the database to wait for the recovery to complete, if the state is completed, all temporary data generated in the data backup and recovery processes are cleaned, and then the data copy is completed, because the backup recovery is completed and the data copy is not completely equivalent to the completion, for example, when the snapshot is expanded, the recovery process is completed when the snapshot state becomes available and is referenced by one storage, but the data copy is not completed yet, the storage state is required to be changed to be a binding state copy and is completed really, and finally a computing node is created on the storage of which the data copy is completed and the computing node is expanded.
The embodiment provides a set of universal database node capacity expansion implementation scheme, so that any database can realize the node capacity expansion function through the configuration mode.
As shown in fig. 2, this embodiment further provides a general database node capacity expansion device based on k8s, including:
a creating module 10, configured to create a storage according to a database node capacity expansion target, and determine whether to copy data in the storage according to a type of the database;
a determining module 20, configured to determine a data copying manner according to a current environment of the database when copying data in the storage;
and the capacity expansion module 30 is used for carrying out data copying according to the data copying mode and creating a computing node on the storage of which the data copying is completed so as to complete node capacity expansion.
One embodiment of the above device may be: the creation module 10 creates a storage according to a database node capacity expansion target and determines whether to copy data in the storage according to the type of the database; determining a data copying mode according to the current environment of the database when the determining module 20 copies the data in the storage; the capacity expansion module 30 performs data copying according to the data copying mode, and creates a computing node on the storage of which the data copying is completed to complete node capacity expansion.
Further, the determining module 20 further includes:
and the selection unit 21 is configured to determine whether the current environment of the database supports disk snapshots, if yes, take the disk snapshots as a data copy mode thereof, and if no, restore the backup as the data copy mode thereof.
Further, the capacity expansion module 30 further includes:
a first judging unit 31, configured to judge whether the copying of the data is completed, and if not, determine a current backup state of the data;
a second judging unit 32 for judging the current recovery state of the data when the current backup state is completed;
and a cleaning unit 33, configured to clean all temporary data generated during the data backup and recovery process when the current recovery state is completed.
As shown in fig. 3, the present embodiment further provides an electronic device, including a memory 301 and a processor 302, where the memory 301 is configured to store one or more computer instructions, and the one or more computer instructions are executed by the processor 302 to implement a k8 s-based universal database node capacity expansion method.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the electronic device described above may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.
A computer readable storage medium storing a computer program which, when executed by a computer, causes the computer to implement a k8 s-based universal database node capacity expansion method as described above.
By way of example, a computer program may be divided into one or more modules/units stored in the memory 301 and executed by the processor 302 and completed by the input interface 305 and the output interface 306 to complete the present application, and one or more modules/units may be a series of computer program instruction segments capable of performing specific functions for describing the execution of the computer program in a computer device.
The computer device may be a desktop computer, a notebook computer, a palm computer, a cloud server, or the like. The computer device may include, but is not limited to, a memory 301, a processor 302, it will be understood by those skilled in the art that the present embodiment is merely an example of a computer device and is not limiting of a computer device, may include more or fewer components, or may combine certain components, or different components, e.g., a computer device may also include an input 307, a network access device, a bus, etc.
The processor 302 may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors 302, digital signal processors 302 (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. The general purpose processor 302 may be a microprocessor 302 or the processor 302 may be any conventional processor 302 or the like.
The memory 301 may be an internal storage unit of the computer device, such as a hard disk or a memory of the computer device. The memory 301 may also be an external storage device of a computer device, such as a plug-in hard disk, smart Media Card (SMC), secure Digital (SD) Card, flash memory Card (Flash Card) or the like, which is provided on a computer device, and further, the memory 301 may also include an internal storage unit of a computer device and an external storage device, the memory 301 may also be used to store computer programs and other programs and data required by a computer device, the memory 301 may also be used to temporarily store the programs and data in the output 308, and the aforementioned storage Media include a U disk, a removable hard disk, a read-only memory ROM303, a random access memory RAM304, a disk or an optical disk and other various Media that can store program codes.
The foregoing is merely illustrative of specific embodiments of the present application, and the scope of the present application is not limited thereto, but any changes or substitutions within the technical scope of the present application should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. The universal database node capacity expansion method based on k8s is characterized by comprising the following steps of:
creating a storage according to a database node capacity expansion target, and determining whether to copy data in the storage according to the type of the database;
when the data is copied in the storage, determining a data copying mode according to the current environment of the database;
and carrying out data copying according to the data copying mode, and creating a computing node on the storage of which the data copying is completed so as to complete node capacity expansion.
2. The k8 s-based universal database node capacity expansion method according to claim 1, wherein the determining the data copy mode according to the current environment of the database comprises:
and determining whether the current environment of the database supports disk snapshots, if so, taking the disk snapshots as a data copying mode, and if not, restoring the backup as the data copying mode.
3. The k8 s-based universal database node capacity expansion method according to claim 1, further comprising, after the data copying according to the data copying mode:
judging whether the data copying is completed or not, if not, determining the current backup state of the data;
judging the current recovery state of the data when the current backup state is completed;
and when the current recovery state is finished, cleaning all temporary data generated in the data backup and recovery process.
4. A method for expanding a universal database node based on k8s according to claim 3, wherein the current backup state comprises not created, being backed up or completed, creating a backup when the current backup state is not created, and waiting for the backup to be completed when the current backup state is in backup.
5. A method of expanding a generic database node based on k8s according to claim 3, wherein the current recovery state comprises not started, in recovery or completed, and wherein recovery is performed when the current recovery state is not started, and recovery is waited for completion when the current recovery state is in recovery.
6. A k8 s-based universal database node capacity expansion device, comprising:
the creation module is used for creating a storage according to the database node capacity expansion target and determining whether to copy data in the storage according to the type of the database;
the determining module is used for determining the data copying mode according to the current environment of the database when the data is copied in the storage;
and the capacity expansion module is used for carrying out data copying according to the data copying mode, and creating a computing node on the storage of which the data copying is completed so as to complete node capacity expansion.
7. The k8 s-based universal database node capacity expansion device of claim 6, wherein the determining module comprises:
and the selection unit is used for determining whether the current environment of the database supports disk snapshots, if so, taking the disk snapshots as a data copy mode, and if not, restoring the backup as the data copy mode.
8. The k8 s-based universal database node capacity expansion device of claim 6, wherein the capacity expansion module comprises:
the first judging unit is used for judging whether the data copying is finished or not, and if not, determining the current backup state of the data;
the second judging unit is used for judging the current recovery state of the data when the current backup state is completed;
and the cleaning unit is used for cleaning all temporary data generated in the data backup and recovery process when the current recovery state is completed.
9. An electronic device comprising a memory and a processor, the memory configured to store one or more computer instructions, wherein the one or more computer instructions are executable by the processor to implement a k8 s-based universal database node expansion method as recited in any one of claims 1-5.
10. A computer-readable storage medium storing a computer program, wherein the computer program when executed causes a computer to implement a k8 s-based universal database node capacity expansion method as claimed in any one of claims 1 to 5.
CN202311070000.4A 2023-08-23 2023-08-23 Universal database node capacity expansion method and device based on k8s Pending CN117009438A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311070000.4A CN117009438A (en) 2023-08-23 2023-08-23 Universal database node capacity expansion method and device based on k8s

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311070000.4A CN117009438A (en) 2023-08-23 2023-08-23 Universal database node capacity expansion method and device based on k8s

Publications (1)

Publication Number Publication Date
CN117009438A true CN117009438A (en) 2023-11-07

Family

ID=88569057

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311070000.4A Pending CN117009438A (en) 2023-08-23 2023-08-23 Universal database node capacity expansion method and device based on k8s

Country Status (1)

Country Link
CN (1) CN117009438A (en)

Similar Documents

Publication Publication Date Title
CN107391628B (en) Data synchronization method and device
CN103729442A (en) Method for recording event logs and database engine
CN111338662A (en) Firmware upgrading method and device for slave station and terminal
CN110442480B (en) Mirror image data cleaning method, device and system
CN115562911B (en) Virtual machine data backup method, device, system, electronic equipment and storage medium
CN106339458B (en) Stage partitioning method and terminal based on elastic distributed data set
CN114090179A (en) Migration method and device of stateful service and server
CN111913793A (en) Distributed task scheduling method, device, node equipment and system
CN111046004B (en) Data file storage method, device, equipment and storage medium
CN113468143A (en) Data migration method, system, computing device and storage medium
CN112433812A (en) Method, system, equipment and computer medium for virtual machine cross-cluster migration
CN108121514B (en) Meta information updating method and device, computing equipment and computer storage medium
CN115952185B (en) Data processing method and device, equipment and storage medium
CN104268089A (en) Information updating method, device and system
CN111831408A (en) Asynchronous task processing method and device, electronic equipment and medium
CN117009438A (en) Universal database node capacity expansion method and device based on k8s
CN108073460B (en) Global lock preemption method and device in distributed system and computing equipment
EP4318211A1 (en) Method for inspecting code under weak memory order architecture, and corresponding device
CN114564210A (en) Copy deployment method, device, system, electronic equipment and storage medium
CN114443302A (en) Container cluster capacity expansion method, system, terminal and storage medium
CN111431951B (en) Data processing method, node equipment, system and storage medium
CN114968753A (en) Equipment upgrading test method, medium, electronic equipment and test system
CN109002343B (en) Method and device for realizing batch creation of virtual machines
CN117520019B (en) Component communication method and device
US10862958B1 (en) Flexible associativity in multitenant clustered environments

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication