CN114356886A - Database node instance online capacity reduction method and device and electronic equipment - Google Patents
Database node instance online capacity reduction method and device and electronic equipment Download PDFInfo
- Publication number
- CN114356886A CN114356886A CN202111652714.7A CN202111652714A CN114356886A CN 114356886 A CN114356886 A CN 114356886A CN 202111652714 A CN202111652714 A CN 202111652714A CN 114356886 A CN114356886 A CN 114356886A
- Authority
- CN
- China
- Prior art keywords
- capacity reduction
- database
- state
- system table
- instance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 230000009467 reduction Effects 0.000 title claims abstract description 265
- 238000000034 method Methods 0.000 title claims abstract description 79
- 238000012544 monitoring process Methods 0.000 claims abstract description 19
- 230000008569 process Effects 0.000 claims abstract description 19
- 238000005096 rolling process Methods 0.000 claims description 16
- 238000004422 calculation algorithm Methods 0.000 claims description 10
- 238000003860 storage Methods 0.000 claims description 9
- 238000011946 reduction process Methods 0.000 abstract description 11
- 238000010586 diagram Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 7
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000002360 preparation method Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000005520 cutting process Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 230000005012 migration Effects 0.000 description 2
- 238000013508 migration Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011038 discontinuous diafiltration by volume reduction Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/217—Database tuning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides an online capacity reduction method and device for a database node instance and electronic equipment, wherein the method comprises the following steps: acquiring an example capacity reduction instruction of a database node; calling a corresponding instance capacity reduction script according to the instance capacity reduction instruction; monitoring the capacity reduction state of the database node in the process of running the example capacity reduction script; when the capacity reduction state of the database node enters a system table updating starting state, locking the system table of the database, deleting the system table information of the to-be-deleted example indicated by the example capacity reduction instruction, and updating the system table information of all the examples in the system table; and after the system table information updating work of all the examples is completed, releasing the system table of the database. The online capacity reduction of the database node instance is realized by calling the instance capacity reduction script, the system table is locked, and the system table information of the database is updated in time, so that the consistency of the system table is ensured, and the database can still provide services to the outside in the capacity reduction process.
Description
Technical Field
The present application relates to the field of database management technologies, and in particular, to an online capacity reduction method and apparatus for a database node instance, and an electronic device.
Background
In the era of cloud computing and big data, greenplus is a new generation big data platform as an open source, is compatible with SQL standard, has the capacity of mass data analysis, and can provide a big data engine with strong and big functions of real-time processing, elastic capacity expansion, mixed load, parallel storage, computing and the like.
However, the greenplus database does not support the function of deleting the data node instance, when a user needs to reduce the scale of the existing greenplus cluster, only relevant schemes can be made manually, and the data can be removed manually according to the schemes in a certain sequence according to the schemes offline, so that the operation is complex, and the consumed time is long. Moreover, in the process of manually removing the node instance, the greenplus database needs to be stopped, and the service cannot be provided to the outside in the period.
Disclosure of Invention
The application provides an online capacity reduction method and device for a database node instance and electronic equipment, and aims to overcome the defects that in the prior art, when the capacity reduction of the database node instance is carried out online, the database cannot provide services for a long time and the like.
The first aspect of the present application provides an online capacity reduction method for a database node instance, including:
acquiring an example capacity reduction instruction of a database node;
calling a corresponding instance capacity reduction script according to the instance capacity reduction instruction;
monitoring the capacity reduction state of the database node in the process of running the example capacity reduction script;
when the capacity reduction state of the database node enters a system table updating starting state, locking the system table of the database, deleting the system table information of the to-be-deleted example indicated by the example capacity reduction instruction, and updating the system table information of all the examples in the system table;
and releasing the system table of the database after the system table information updating work of all the examples is finished.
Optionally, the method further includes:
and generating a capacity reduction state record file for the to-be-deleted example indicated by the example capacity reduction instruction according to the monitoring result of the capacity reduction state of the database node.
Optionally, the method further includes:
judging whether the to-be-deleted instance has the reduced volume state recording file or not;
if yes, reading the capacity reduction state recording file to obtain the historical capacity reduction progress of the example to be deleted;
and running the instance capacity reduction script by taking the latest capacity reduction state corresponding to the historical capacity reduction progress as a starting state.
Optionally, the method further includes:
receiving an instance rollback instruction for the database node;
extracting a capacity reduction state record file of the to-be-rolled back example indicated by the example rollback instruction;
judging whether the to-be-rolled back instance meets a preset rolling back requirement or not according to the capacity reduction state recording file;
and when the example to be rolled back meets the rolling back requirement, rolling back the example to be rolled back according to the reduced capacity state record file so as to restore the example to be rolled back to the original state.
Optionally, after the update of the system table information of all the instances is completed, the capacity reduction state of the database node enters a capacity reduction completion state, and the method further includes:
and deleting the capacity reduction state record file of the deleted example after the capacity reduction state of the database node enters a capacity reduction completion state.
Optionally, the capacity reduction state of the database node includes a capacity reduction redistribution starting state, and the method further includes:
and when the capacity reduction state of the database node enters a capacity reduction redistribution starting state, migrating the table data of the example to be deleted to other database nodes according to the node configuration information of the database based on a preset consistent Hash algorithm.
Optionally, before invoking the corresponding instance capacity reduction script according to the instance capacity reduction instruction, the method further includes:
acquiring state information of the database;
judging whether the database is in a normal operation mode or not according to the state information of the database;
and if so, executing the instance capacity reduction instruction and calling the corresponding instance capacity reduction script.
A second aspect of the present application provides an online capacity reduction apparatus for a database node instance, including:
the acquisition module is used for acquiring an example capacity reduction instruction of the database node;
the script calling module is used for calling the corresponding instance capacity-reducing script according to the instance capacity-reducing instruction;
the monitoring module is used for monitoring the capacity reduction state of the database node in the process of running the example capacity reduction script;
the system table management module is used for locking the system table of the database, deleting the system table information of the to-be-deleted example indicated by the example capacity reduction instruction and updating the system table information of all the examples in the system table when the capacity reduction state of the database node enters a system table updating starting state;
and the system table releasing module is used for releasing the system table of the database after the system table information updating work of all the examples is finished.
Optionally, the apparatus further comprises:
and the capacity reduction state record file generation module is used for generating a capacity reduction state record file for the example to be deleted indicated by the example capacity reduction instruction according to the monitoring result of the capacity reduction state of the database node.
Optionally, the script invoking module is further configured to:
judging whether the to-be-deleted instance has the reduced volume state recording file or not;
if yes, reading the capacity reduction state recording file to obtain the historical capacity reduction progress of the example to be deleted;
and running the instance capacity reduction script by taking the latest capacity reduction state corresponding to the historical capacity reduction progress as a starting state.
Optionally, the apparatus further comprises:
a rollback module to receive an instance rollback instruction for the database node; extracting a capacity reduction state record file of the to-be-rolled back example indicated by the example rollback instruction; judging whether the to-be-rolled back instance meets a preset rolling back requirement or not according to the capacity reduction state recording file; and when the example to be rolled back meets the rolling back requirement, rolling back the example to be rolled back according to the reduced capacity state record file so as to restore the example to be rolled back to the original state.
Optionally, after the updating of the system table information of all the instances is completed, the capacity reduction state of the database node enters a capacity reduction completion state, and the apparatus further includes:
and the deleting module is used for deleting the capacity reduction state record file of the currently deleted example after the capacity reduction state of the database node enters a capacity reduction completion state.
Optionally, the capacity reduction state of the database node includes a capacity reduction redistribution starting state, and the apparatus further includes:
and the data migration module is used for migrating the table data of the example to be deleted to other database nodes according to the node configuration information of the database based on a preset consistent Hash algorithm when the capacity reduction state of the database nodes enters a capacity reduction redistribution starting state.
Optionally, the apparatus further comprises:
the state detection module is used for acquiring state information of the database; judging whether the database is in a normal operation mode or not according to the state information of the database; and if so, executing the instance capacity reduction instruction and calling the corresponding instance capacity reduction script.
A third aspect of the present application provides an electronic device, comprising: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executes computer-executable instructions stored by the memory to cause the at least one processor to perform the method as set forth in the first aspect above and in various possible designs of the first aspect.
A fourth aspect of the present application provides a computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a processor, implement a method as set forth in the first aspect and various possible designs of the first aspect.
This application technical scheme has following advantage:
the application provides an online capacity reduction method and device for a database node instance and electronic equipment, wherein the method comprises the following steps: acquiring an example capacity reduction instruction of a database node; calling a corresponding instance capacity reduction script according to the instance capacity reduction instruction; monitoring the capacity reduction state of the database node in the process of running the example capacity reduction script; when the capacity reduction state of the database node enters a system table updating starting state, locking the system table of the database, deleting the system table information of the to-be-deleted example indicated by the example capacity reduction instruction, and updating the system table information of all the examples in the system table; and after the system table information updating work of all the examples is completed, releasing the system table of the database. The method provided by the scheme realizes the online capacity reduction of the database node instance by calling the instance capacity reduction script, and ensures the consistency of the system table by locking the system table and timely updating the system table information of the database so as to ensure that the database can still provide services to the outside in the capacity reduction process.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art according to these drawings.
Fig. 1 is a schematic structural diagram of an online capacity reduction system based on a database node example according to an embodiment of the present application;
fig. 2 is a schematic flow chart of an online capacity reduction method for a database node example provided in the embodiment of the present application;
FIG. 3 is a schematic diagram of a consistent hashing algorithm provided by an embodiment of the present application;
fig. 4 is a schematic overall flow chart of an online capacity reduction method for a database node example provided in the embodiment of the present application;
fig. 5 is a schematic structural diagram of an online capacity reduction device for a database node example provided in the embodiment of the present application;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
With the above figures, there are shown specific embodiments of the present application, which will be described in more detail below. These drawings and written description are not intended to limit the scope of the disclosed concepts in any way, but rather to illustrate the concepts of the disclosure to those skilled in the art by reference to specific embodiments.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Furthermore, the terms "first", "second", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. In the description of the following examples, "plurality" means two or more unless specifically limited otherwise.
The Greenplus database does not support the function of deleting the data node instance, when a user needs to reduce the scale of the existing Greenplus cluster, only relevant schemes can be made manually, the data are removed manually according to the schemes in a certain sequence, the operation is complex, the data are easy to lose, and the risk is high. In the process of manually removing the node instance, the greenplus database needs to be stopped, and the service cannot be provided to the outside in the period, so that the node instance capacity reduction cannot be performed quickly and efficiently without affecting other services.
In order to solve the above problems, the method, the device and the electronic device for online capacity reduction of the database node instance provided by the embodiment of the application obtain the instance capacity reduction instruction of the database node; calling a corresponding instance capacity reduction script according to the instance capacity reduction instruction; monitoring the capacity reduction state of the database node in the process of running the example capacity reduction script; when the capacity reduction state of the database node enters a system table updating starting state, locking the system table of the database, deleting the system table information of the to-be-deleted example indicated by the example capacity reduction instruction, and updating the system table information of all the examples in the system table; and after the system table information updating work of all the examples is completed, releasing the system table of the database. The method provided by the scheme realizes the online capacity reduction of the database node instance by calling the instance capacity reduction script, and ensures the consistency of the system table by locking the system table and timely updating the system table information of the database so as to ensure that the database can still provide services to the outside in the capacity reduction process.
The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present invention will be described below with reference to the accompanying drawings.
First, a structure of an online capacity reduction system based on a database node instance is described:
the database node instance online capacity reduction method, the database node instance online capacity reduction device and the electronic equipment are suitable for a data node instance online capacity reduction scene of a Greenplus database cluster. As shown in fig. 1, the structural schematic diagram of the database node instance online capacity reduction system according to the embodiment of the present application mainly includes a database cluster and a database node instance online capacity reduction device, where the database cluster includes a plurality of database nodes, and specifically, the database node instance online capacity reduction device receives an instance capacity reduction instruction of a database node sent by a user, and then performs instance capacity reduction on the database node in the database cluster according to the instance capacity reduction instruction.
The embodiment of the application provides an online capacity reduction method for a database node instance, which is used for performing online capacity reduction on the data node instance of a Greenplus database cluster. The execution subject of the embodiment of the present application is an electronic device, such as a server, a desktop computer, a notebook computer, a tablet computer, and other electronic devices that can be used for performing online capacity reduction of database node instances.
As shown in fig. 2, a schematic flow chart of an online capacity reduction method for a database node instance provided in the embodiment of the present application is shown, where the method includes:
Wherein, the instance capacity shrink instruction at least identifies the ID of the instance to be deleted, such as the unique ID (dbid) of the instance and the ID (content) of the main-standby instance group.
It should be noted that the example capacity reduction script that can be called may be a script that is preset in the database system, and the automatic operation of the example capacity reduction operation may be realized by calling the corresponding example capacity reduction script according to the example capacity reduction instruction.
It should be noted that the example capacity reduction process includes multiple stages, such as: the method comprises a capacity reduction mode setting stage, a capacity reduction mode preparing stage, a capacity reduction data redistribution stage and a system table updating stage. Accordingly, the capacity reduction state of the database node is divided into: starting the setting of the capacity reduction mode, finishing the setting of the capacity reduction mode, starting the preparation of the capacity reduction mode, finishing the preparation of the capacity reduction mode, starting the redistribution of capacity reduction data, finishing the redistribution of the capacity reduction data, starting the updating of a system table, finishing the updating of the system table and finishing the capacity reduction.
And step 204, when the capacity reduction state of the database node enters a system table updating starting state, locking the system table of the database, deleting the system table information of the to-be-deleted example indicated by the example capacity reduction instruction, and updating the system table information of all the examples in the system table.
It should be noted that, when a user needs to read or store data in the database cluster, a target database node needs to be located based on the system table, so as to complete subsequent data reading or storing operations.
Specifically, when the capacity reduction state of the database node enters the system table update starting state, it may be determined that the capacity reduction data redistribution operation has been completed, and the to-be-deleted instance is deleted next, so as to avoid that the user just stores data to the to-be-deleted instance next, the system table may be temporarily locked first, then the system table information (ID identifier) of the to-be-deleted instance is deleted, and the system table information of all instances is updated in the system table, and it is ensured that there is no interval between the system table information of other instances.
And step 205, releasing the system table of the database after the system table information updating work of all the examples is completed.
Specifically, when the system table of the database is released, the user can perform data reading or storing operation based on the new system table. Accordingly, during the database system table locking, all query services which are running by the user are not interrupted. The user can not define or change the structure of the data table, the updating time of the system table is generally within 1 second, and the response speed cannot be greatly influenced.
On the basis of the foregoing embodiment, as a practical manner, on the basis of the foregoing embodiment, in an embodiment, the method further includes:
step 301, generating a reduced capacity state record file for the to-be-deleted instance indicated by the instance reduced capacity instruction according to the monitoring result of the reduced capacity state of the database node.
The capacity-reduction state recording file records capacity-reduction progress recording information such as the starting time, the duration and the ending time of each capacity-reduction state.
Further, in an embodiment, before the instance capacity reduction, it may be determined whether the instance to be deleted has a capacity reduction state record file; if yes, reading the capacity reduction state recording file to obtain the historical capacity reduction progress of the example to be deleted; and running the instance capacity reduction script by taking the latest capacity reduction state corresponding to the historical capacity reduction progress as a starting state.
Specifically, if the current database system stores the capacity reduction state record file of the to-be-deleted instance, it may be determined that an exception may have occurred in the process of instance capacity reduction before, which may cause interruption of the instance capacity reduction process. Therefore, in order to improve the efficiency of the instance capacity reduction, the starting state of the capacity reduction process of the current instance can be determined according to the historical capacity reduction progress represented by the capacity reduction state recording file of the instance to be deleted, and then the instance capacity reduction script is operated according to the previous historical capacity reduction progress, so that the operation time of the script at this time is shortened.
On the basis of the foregoing embodiment, in order to further enrich the role of the database node instance provided in the embodiment of the present application in the capacity reduction method, as an implementable manner, on the basis of the foregoing embodiment, in an embodiment, the method further includes:
step 401, receiving an instance rollback instruction for a database node;
step 402, extracting a reduced volume state record file of the example to be rolled back, which is indicated by the example rolling back instruction;
step 403, judging whether the to-be-rolled back instance meets a preset rolling back requirement or not according to the reduced volume state recording file;
and step 404, when the to-be-rolled back instance meets the rolling back requirement, rolling back the to-be-rolled back instance according to the reduced capacity state record file so as to restore the to-be-rolled back instance to the original state.
It should be noted that, when an error occurs in the program or script during the running process, the instance can be restored to the previous correct state through the rollback technique.
Specifically, whether or not rollback is possible may be determined from the content of the gpreduced. Status does not exist, indicating that no data needs to be rolled back. Status file record is system table UPDATE complete (CATALOG _ UPDATE _ DONE), indicating that the retraction is complete and rollback cannot be performed. Otherwise, the original state of the database cluster is restored. And deleting other volume reduction process record files such as the gpredue.
Further, in an embodiment, after the update of the system table information of all the instances is completed, the capacity reduction state of the database node enters the capacity reduction completion state, and accordingly, the capacity reduction state record file of the currently deleted instance may be deleted after the capacity reduction state of the database node enters the capacity reduction completion state, or other capacity reduction process record files such as the tables status (capacity reduction state table) and status _ detail table may also be deleted simultaneously as in the above embodiments.
On the basis of the foregoing embodiments, in order to improve the efficiency of capacity reduction redistribution, as an implementable manner, on the basis of the foregoing embodiments, in an embodiment, the capacity reduction state of the database node includes a capacity reduction redistribution starting state, and when the capacity reduction state of the database node enters the capacity reduction redistribution starting state, the table data of the example to be deleted may be migrated to another database node according to the node configuration information of the database based on a preset consistent hash algorithm.
It should be noted that, if all the instances on a certain database node are deleted, all the table data on the node needs to be migrated to other nodes, and the database node is also deleted. The node configuration information of the database comprises position information of each database node in the hash ring and the like.
Fig. 3 is a schematic diagram of a consistent hashing algorithm according to an embodiment of the present disclosure. The consistent hash algorithm is also called a ring-cutting method, and the ring-cutting method organizes the whole hash space into a virtual ring which is connected end to end, and is generally set as [0,2^32-1], namely, the hash value is a 32-bit unsigned integer. For example, the number or IP of the node N is subjected to hash function hash (N) calculation, the result is modulo 2^32 and mapped on the ring, and then the data is mapped on the ring according to the same hash function hash (k), so that the data is stored on the ring to traverse the first node found in the clockwise direction. When the nodes expand or contract, the data in the affected interval is redistributed to the adjacent nodes according to the clockwise principle, so that the purpose of incremental updating is achieved, the algorithm cannot re-hash all the data, and the efficiency of the data redistribution process is improved.
On the basis of the foregoing embodiment, in order to ensure that the database node instance can normally implement the online capacity reduction process, as an implementable manner, on the basis of the foregoing embodiment, in an embodiment, before invoking a corresponding instance capacity reduction script according to an instance capacity reduction instruction, the method further includes:
and 503, if yes, executing the instance capacity reduction instruction, and calling the corresponding instance capacity reduction script.
It should be noted that the database state information includes various database health indicators for determining that the database is normal, and the embodiment of the present application is not limited to which database health indicators are specifically included.
And conversely, if the database is in the abnormal mode, exiting the online capacity reduction process of the database node instance.
Exemplarily, as shown in fig. 4, an overall flow diagram of an online capacity reduction method for a database node instance provided in the embodiment of the present application is shown, where the flow is as follows:
step 1, when an instance capacity reduction instruction is issued, firstly, checking the state of a database, and if the database runs normally, executing step 2; otherwise, executing step 3;
step 3, entering an exit module, wherein the module exits the current capacity reduction operation;
step 4, entering a capacity reduction configuration file generation module, generating a capacity reduction configuration file in the module, and then executing step 5;
step 5, entering a database configuration acquisition module, acquiring relevant information of the current database cluster in the module, wherein the relevant information comprises a main example, a mirror image example, a system table and the like, and then executing step 6;
step 6, entering a capacity reduction state acquisition module, reading the contents of a gpredue status file and a status table in the module, and then executing step 7;
step 7, entering a capacity reduction state judging module, and executing step 8 if a rollback command parameter is issued in the module; if the cleaning command parameter is issued, executing the step 9; if the status table does not exist, the gpredue.status file does not exist, and the specified capacity reduction configuration file parameter exists, executing step 10; if the latest status of the status table is "capacity reduction setup complete", "capacity reduction redistribution stop", or "capacity reduction redistribution start", then step 11 is executed; if the status table is "capacity reduction redistribution is completed" or "online update system table is started", executing step 12;
step 8, entering a rollback module, wherein the module is restored to the original state of the cluster, and deleting the status table, the status _ detail table and the gpredue.
Step 9, entering a capacity information clearing module, and deleting the status table, the status _ detail table and the gpredue.
Step 10, entering a capacity reduction preparation module, creating a status table and a status _ detail table and a gpredue file in the module, updating the contents of the status table and the status _ detail table and the gpredue file, and then executing step 11;
step 11, entering a data redistribution module, wherein data redistribution is carried out, values of a status table, a status _ detail table and a gpredue.status file are updated, and then step 12 is executed;
and step 12, entering a system table online updating module, wherein the system table is locked, and the information of the instance to be deleted is removed from the system table.
The method for online capacity reduction of the database node instance provided by the embodiment of the application obtains the instance capacity reduction instruction of the database node; calling a corresponding instance capacity reduction script according to the instance capacity reduction instruction; monitoring the capacity reduction state of the database node in the process of running the example capacity reduction script; when the capacity reduction state of the database node enters a system table updating starting state, locking the system table of the database, deleting the system table information of the to-be-deleted example indicated by the example capacity reduction instruction, and updating the system table information of all the examples in the system table; and after the system table information updating work of all the examples is completed, releasing the system table of the database. According to the method provided by the scheme, the online capacity reduction of the database node instance is realized by calling the instance capacity reduction script, and the consistency of the system table is ensured by locking the system table and timely updating the system table information of the database, so that the database can still provide services to the outside in the capacity reduction process, and the reliability of the database system is improved. And moreover, by using the consistent Hash algorithm, the data amount redistributed in the capacity reduction process is reduced, and the data redistribution efficiency is improved. The unnecessary workload that the effectual people's maloperation that has reduced produced to and the loss that can't compensate that causes, promoted work efficiency.
The embodiment of the application provides an online capacity reduction device for a database node instance, which is used for executing the online capacity reduction method for the database node instance provided by the embodiment.
Fig. 5 is a schematic structural diagram of a database node example online capacity reduction device according to an embodiment of the present disclosure. The database node instance online capacity reduction device 50 includes: an acquisition module 501, a script calling module 502, a monitoring module 503, a system table management module 504 and a system table release module 505.
The system comprises an acquisition module, a capacity reduction module and a capacity reduction module, wherein the acquisition module is used for acquiring an instance capacity reduction instruction of a database node; the script calling module is used for calling the corresponding instance capacity-reducing script according to the instance capacity-reducing instruction; the monitoring module is used for monitoring the capacity reduction state of the database node in the process of running the example capacity reduction script; the system table management module is used for locking the system table of the database, deleting the system table information of the to-be-deleted example indicated by the example capacity reduction instruction and updating the system table information of all the examples in the system table when the capacity reduction state of the database node enters the system table updating starting state; and the system table releasing module is used for releasing the system table of the database after the system table information updating work of all the examples is completed.
Specifically, in one embodiment, the apparatus further comprises:
and the capacity reduction state record file generation module is used for generating a capacity reduction state record file for the example to be deleted indicated by the example capacity reduction instruction according to the monitoring result of the capacity reduction state of the database node.
Specifically, in an embodiment, the script calling module is further configured to:
judging whether the to-be-deleted instance has a capacity reduction state recording file or not;
if yes, reading the capacity reduction state recording file to obtain the historical capacity reduction progress of the example to be deleted;
and running the instance capacity reduction script by taking the latest capacity reduction state corresponding to the historical capacity reduction progress as a starting state.
Specifically, in one embodiment, the apparatus further comprises:
the rollback module is used for receiving an example rollback instruction aiming at the database node; extracting a reduced volume state record file of the example to be rolled back, which is indicated by the example rollback instruction; judging whether the to-be-rolled back instance meets a preset rolling back requirement or not according to the capacity reduction state recording file; and when the example to be rolled back meets the rolling back requirement, rolling back the example to be rolled back according to the reduced volume state record file so as to restore the example to be rolled back to the original state.
Specifically, in an embodiment, after the system table information updating work of all the instances is completed, the capacity reduction state of the database node enters a capacity reduction completion state, and the apparatus further includes:
and the deleting module is used for deleting the capacity reduction state record file of the currently deleted example after the capacity reduction state of the database node enters a capacity reduction completion state.
Specifically, in an embodiment, the capacity reduction state of the database node includes a capacity reduction redistribution starting state, and the apparatus further includes:
and the data migration module is used for migrating the table data of the example to be deleted to other database nodes according to the node configuration information of the database based on a preset consistent Hash algorithm when the capacity reduction state of the database nodes enters the capacity reduction redistribution starting state.
Specifically, in one embodiment, the apparatus further comprises:
the state detection module is used for acquiring state information of the database; judging whether the database is in a normal operation mode or not according to the state information of the database; if yes, executing the instance capacity reduction instruction and calling the corresponding instance capacity reduction script.
The specific manner in which each module performs operations in the database node instance on the scalable apparatus in this embodiment has been described in detail in the embodiment related to the method, and will not be described in detail here.
The online capacity reduction device for the database node instance provided in the embodiment of the present application is used to execute the online capacity reduction method for the database node instance provided in the foregoing embodiment, and an implementation manner and a principle thereof are the same and are not described again.
The embodiment of the application provides electronic equipment for executing the database node instance online capacity reduction method provided by the embodiment.
Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device 60 includes: at least one processor 61 and memory 62;
the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored by the memory, causing the at least one processor to perform the database node instance online capacity reduction method provided by the above embodiments.
The electronic device provided in the embodiment of the present application is configured to execute the database node instance online capacity reduction method provided in the foregoing embodiment, and an implementation manner and a principle of the electronic device are the same and are not described again.
The embodiment of the present application provides a computer-readable storage medium, where computer-executable instructions are stored in the computer-readable storage medium, and when a processor executes the computer-executable instructions, the method for online capacity reduction of a database node instance provided in any of the above embodiments is implemented.
The storage medium including the computer-executable instructions of the embodiments of the present application may be used to store the computer-executable instructions of the database node instance online capacity reduction method provided in the foregoing embodiments, and an implementation manner and a principle thereof are the same, and are not described again.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
It is obvious to those skilled in the art that, for convenience and simplicity of description, the foregoing division of the functional modules is merely used as an example, and in practical applications, the above function distribution may be performed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to perform all or part of the above described functions. For the specific working process of the device described above, reference may be made to the corresponding process in the foregoing method embodiment, which is not described herein again.
Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.
Claims (10)
1. An online capacity reduction method for a database node instance is characterized by comprising the following steps:
acquiring an example capacity reduction instruction of a database node;
calling a corresponding instance capacity reduction script according to the instance capacity reduction instruction;
monitoring the capacity reduction state of the database node in the process of running the example capacity reduction script;
when the capacity reduction state of the database node enters a system table updating starting state, locking the system table of the database, deleting the system table information of the to-be-deleted example indicated by the example capacity reduction instruction, and updating the system table information of all the examples in the system table;
and releasing the system table of the database after the system table information updating work of all the examples is finished.
2. The method of claim 1, further comprising:
and generating a capacity reduction state record file for the to-be-deleted example indicated by the example capacity reduction instruction according to the monitoring result of the capacity reduction state of the database node.
3. The method of claim 2, further comprising:
judging whether the to-be-deleted instance has the reduced volume state recording file or not;
if yes, reading the capacity reduction state recording file to obtain the historical capacity reduction progress of the example to be deleted;
and running the instance capacity reduction script by taking the latest capacity reduction state corresponding to the historical capacity reduction progress as a starting state.
4. The method of claim 2, further comprising:
receiving an instance rollback instruction for the database node;
extracting a capacity reduction state record file of the to-be-rolled back example indicated by the example rollback instruction;
judging whether the to-be-rolled back instance meets a preset rolling back requirement or not according to the capacity reduction state recording file;
and when the example to be rolled back meets the rolling back requirement, rolling back the example to be rolled back according to the reduced capacity state record file so as to restore the example to be rolled back to the original state.
5. The method according to claim 2, wherein after completing the system table information updating work of all the instances, the capacity reduction state of the database node enters a capacity reduction completed state, and the method further comprises:
and deleting the capacity reduction state record file of the deleted example after the capacity reduction state of the database node enters a capacity reduction completion state.
6. The method of claim 1, wherein the reduced capacity state of the database node comprises a reduced capacity redistribution start state, the method further comprising:
and when the capacity reduction state of the database node enters a capacity reduction redistribution starting state, migrating the table data of the example to be deleted to other database nodes according to the node configuration information of the database based on a preset consistent Hash algorithm.
7. The method of claim 1, wherein prior to invoking the corresponding instance capacity reduction script in accordance with the instance capacity reduction instruction, the method further comprises:
acquiring state information of the database;
judging whether the database is in a normal operation mode or not according to the state information of the database;
and if so, executing the instance capacity reduction instruction and calling the corresponding instance capacity reduction script.
8. An online capacity reduction device for database node instances, comprising:
the acquisition module is used for acquiring an example capacity reduction instruction of the database node;
the script calling module is used for calling the corresponding instance capacity-reducing script according to the instance capacity-reducing instruction;
the monitoring module is used for monitoring the capacity reduction state of the database node in the process of running the example capacity reduction script;
the system table management module is used for locking the system table of the database, deleting the system table information of the to-be-deleted example indicated by the example capacity reduction instruction and updating the system table information of all the examples in the system table when the capacity reduction state of the database node enters a system table updating starting state;
and the system table releasing module is used for releasing the system table of the database after the system table information updating work of all the examples is finished.
9. An electronic device, comprising: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executing the computer-executable instructions stored by the memory causes the at least one processor to perform the method of any of claims 1-7.
10. A computer-readable storage medium having computer-executable instructions stored thereon which, when executed by a processor, implement the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111652714.7A CN114356886A (en) | 2021-12-30 | 2021-12-30 | Database node instance online capacity reduction method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111652714.7A CN114356886A (en) | 2021-12-30 | 2021-12-30 | Database node instance online capacity reduction method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114356886A true CN114356886A (en) | 2022-04-15 |
Family
ID=81104239
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111652714.7A Withdrawn CN114356886A (en) | 2021-12-30 | 2021-12-30 | Database node instance online capacity reduction method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114356886A (en) |
-
2021
- 2021-12-30 CN CN202111652714.7A patent/CN114356886A/en not_active Withdrawn
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110569305B (en) | Block synchronization method, device, medium and computing equipment | |
| CN107885758B (en) | Data migration method of virtual node and virtual node | |
| CN108121782B (en) | Distribution method of query request, database middleware system and electronic equipment | |
| CN109643310B (en) | System and method for redistribution of data in a database | |
| JP2007241486A (en) | Memory system | |
| CN111917834A (en) | Data synchronization method and device, storage medium and computer equipment | |
| CN112231108A (en) | Task processing method and device, computer readable storage medium and server | |
| CN111930716A (en) | Database capacity expansion method, device and system | |
| CN111666266A (en) | Data migration method and related equipment | |
| CN109726211B (en) | Distributed time sequence database | |
| JP6828253B2 (en) | Backup control device, backup control method and program | |
| CN112579550B (en) | Metadata information synchronization method and system of distributed file system | |
| CN114020721A (en) | Data management method, device, equipment and storage medium for time sequence database cluster | |
| JP5691306B2 (en) | Information processing system | |
| CN108121514B (en) | Meta information updating method and device, computing equipment and computer storage medium | |
| CN114356886A (en) | Database node instance online capacity reduction method and device and electronic equipment | |
| CN116089518A (en) | Data model extraction method and system, terminal and medium | |
| CN107894942B (en) | Method and device for monitoring data table access amount | |
| US8630976B2 (en) | Fast search replication synchronization processes | |
| CN114116723A (en) | Snapshot processing method and device and electronic equipment | |
| CN113806309A (en) | Metadata deleting method, system, terminal and storage medium based on distributed lock | |
| CN108984343B (en) | Virtual machine backup and storage management method based on content analysis | |
| CN109582330B (en) | Data model upgrading method, device, equipment and readable storage medium | |
| CN107590286B (en) | Method and device for managing transaction information in cluster file system | |
| CN115114258A (en) | Data copying method and device, electronic equipment and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20220415 |
|
| WW01 | Invention patent application withdrawn after publication |