CN115269352A

CN115269352A - Database performance determination method and device, electronic equipment and storage medium

Info

Publication number: CN115269352A
Application number: CN202210926168.XA
Authority: CN
Inventors: 李奇伟; 彭卫华; 刘嘉健; 陈立璜
Original assignee: Industrial and Commercial Bank of China Ltd ICBC
Current assignee: Industrial and Commercial Bank of China Ltd ICBC
Priority date: 2022-08-03
Filing date: 2022-08-03
Publication date: 2022-11-01

Abstract

The disclosure provides a database performance determination method and device, electronic equipment and a storage medium, relates to the technical field of databases, and can be applied to the technical field of finance. The database performance determination method comprises the following steps: under the condition that the service data of the source database is migrated to the target database and the target standby database, recording a first format database operation language running in the source database; converting the first format database operation language into a second format database operation language matched with the target database; performing a first playback operation on the service data in the target standby database by using the first format database operation language to obtain first playback data, and performing a second playback operation on the service data in the target database by using the second format database operation language to obtain second playback data; and determining the performance of the target database after the data migration according to the first playback data and the second playback data.

Description

Database performance determination method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of database technologies, and in particular, to a method, an apparatus, a device, a medium, and a program product for determining database performance.

Background

An information system constructed based on a traditional database faces greater uncertainty, a database is replaced in a financial institution, and in the process of migrating from the traditional database to a novel database, due to the fact that different databases have greater differences in database object characteristics, DML syntax, function parameter entering and parameter exiting formats and the like, the situation that business execution errors occur after migration frequently according to a conventional migration method.

Disclosure of Invention

In view of the foregoing, the present disclosure provides a database performance determination method, apparatus, device, medium, and program product.

In one aspect of the present disclosure, a database performance determining method is provided, including:

under the condition that service data of a source database is migrated to a target database and a target standby database, recording a first format database operation language running in the source database, wherein the first format database operation language is used for executing preset data operation on the service data in the source database, the source database and the target database are different databases, and the source database and the target standby database are the same database;

converting the first format database operation language into a second format database operation language matched with the target database;

performing a first playback operation on the service data in the target standby database by using the first format database operation language to obtain first playback data, and performing a second playback operation on the service data in the target database by using the second format database operation language to obtain second playback data;

and determining the performance of the target database after the data migration according to the first playback data and the second playback data.

According to an embodiment of the present disclosure, wherein:

the service data comprises forward migration full data and forward migration incremental data, the forward migration full data is the transaction data generated by the application server connected with the source database and providing services to the outside before the time period of executing the data migration, and the forward migration incremental data is the transaction data generated by the application server connected with the source database and providing services to the outside in the time period of executing the data migration.

According to an embodiment of the present disclosure, further comprising:

under the condition that the performance of the target database after data migration meets the preset performance judgment condition, migrating the service data of the source database to the target database again;

and switching the service of the application server which provides the service to the outside from the source database to the target database.

According to the embodiment of the present disclosure, after switching the traffic of the application server providing the service to the outside from the source database to the target database:

reversely migrating the reverse migration incremental data of the target database to the source database, wherein the reverse migration incremental data are as follows: after the service of the application server is switched to the target database from the source database, the application server connected with the target database provides transaction data generated by the service to the outside;

and under the condition that the performance of the target database does not meet the preset performance judgment condition, the service of the application server providing the service externally is switched back to the source database from the target database.

According to an embodiment of the present disclosure, wherein switching traffic of an application server providing a service to outside from a source database to a target database comprises:

switching part of services of the application server from a source database to a target database;

and under the condition that the execution result of part of the services of the application server related to the target database is correct, switching all the services of the application server from the source database to the target database.

According to the embodiment of the present disclosure, wherein the switching part of the traffic of the application server from the source database to the target database comprises:

and switching part of the read-only transaction service of the first class service of the application server from the source database to the target database.

According to the embodiment of the present disclosure, wherein switching part of the traffic of the application server from the source database to the target database comprises:

dividing a second class of service of the application server into a plurality of unit sub-services, wherein each unit sub-service is associated with one data subset in the service data;

and switching the target unit sub-service in the plurality of unit sub-services from the target data subset in the source database to the target data subset in the target database.

According to the embodiment of the present disclosure, before migrating the service data of the source database to the target database and the target backup database, the method further includes:

converting the format of a database object of the source data into an object format matched with a target database;

and installing the database object after the format conversion in the target database.

Another aspect of the present disclosure provides a database performance determining apparatus including a recording module, a first converting module, a playback module, and a determining module.

The recording module is used for recording a first format database operation language running in a source database under the condition that service data of the source database is migrated to a target database and a target standby database, wherein the first format database operation language is used for executing preset data operation on the service data in the source database, the source database and the target database are different databases, and the source database and the target standby database are the same database;

the first conversion module is used for converting the first format database operation language into a second format database operation language matched with the target database;

the playback module is used for executing a first playback operation on the service data in the target standby database by using the first format database operation language to obtain first playback data and executing a second playback operation on the service data in the target database by using the second format database operation language to obtain second playback data;

and the determining module is used for determining the performance of the target database after the data migration according to the first playback data and the second playback data.

According to the embodiment of the disclosure, the service data includes forward migration full amount data and forward migration incremental data, the forward migration full amount data is transaction data generated by an application server connected with a source database and providing services to the outside before a time period of performing data migration, and the forward migration incremental data is transaction data generated by an application server connected with the source database and providing services to the outside within the time period of performing data migration.

According to the embodiment of the disclosure, the system further comprises a migration module and a first switching module.

The migration module is used for migrating the service data of the source database to the target database again under the condition that the performance of the target database after the data migration meets the preset performance judgment condition;

the first switching module is used for switching the service of the application server which provides the service to the outside from the source database to the target database.

According to the embodiment of the disclosure, the system further comprises an anti-migration module and a second switching module.

The anti-migration module is configured to, after switching a service of an application server that provides a service to the outside from a source database to a target database, anti-migrate reverse migration incremental data of the target database to the source database, where the reverse migration incremental data is: after the service of the application server is switched to the target database from the source database, the application server connected with the target database provides transaction data generated by the service to the outside;

and the second switching module is used for switching back the service of the application server which provides the service to the outside from the target database to the source database under the condition that the performance of the target database does not meet the preset performance judgment condition.

According to the embodiment of the disclosure, the first switching module comprises a first switching unit and a second switching unit.

The first switching unit is used for switching part of services of the application server from a source database to a target database;

and the second switching unit is used for switching all the services of the application server from the source database to the target database under the condition that the execution result of part of the services of the application server related to the target database is correct.

According to the embodiment of the disclosure, the first switching unit includes a first switching subunit, configured to switch a part of the read-only transaction traffic of the first class of traffic of the application server from the source database to the target database.

According to an embodiment of the present disclosure, wherein the first switching unit includes a dividing subunit and a second switching subunit.

The dividing subunit is used for dividing the second type of service of the application server into a plurality of unit sub-services, wherein each unit sub-service is associated with one data subset in the service data;

and the second switching subunit is used for switching the target unit sub-service in the plurality of unit sub-services from the target data subset in the source database to the target data subset in the target database.

According to the embodiment of the disclosure, the device further comprises a second conversion module and an installation module.

The second conversion module is used for converting the format of the database object of the source data into an object format matched with the target database before the service data of the source database is migrated to the target database and the target standby database;

and the installation module is used for installing the database object after the format conversion in the target database.

Another aspect of the present disclosure provides an electronic device including: one or more processors; a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the database performance determination method described above.

Another aspect of the present disclosure also provides a computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the above-described database performance determination method.

Another aspect of the present disclosure also provides a computer program product comprising a computer program which, when executed by a processor, implements the above-described database performance determination method.

Drawings

The foregoing and other objects, features and advantages of the disclosure will be apparent from the following description of embodiments of the disclosure, which proceeds with reference to the accompanying drawings, in which:

FIG. 1 schematically illustrates an application scenario diagram of a database performance determination method, apparatus, device, medium, and program product according to embodiments of the disclosure;

FIG. 2 schematically illustrates a flow chart of a database performance determination method according to an embodiment of the disclosure;

FIG. 3 schematically illustrates a system diagram of a database performance determination method according to an embodiment of the disclosure;

fig. 4 schematically shows a block diagram of the structure of a database performance determination apparatus according to an embodiment of the present disclosure; and

fig. 5 schematically illustrates a block diagram of an electronic device adapted to implement a database performance determination method according to an embodiment of the present disclosure.

Detailed Description

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is illustrative only and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.

In those instances where a convention analogous to "at least one of A, B, and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B, and C" would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.).

Because an information system constructed based on a traditional database faces greater uncertainty, a database is replaced in a financial institution, and in the process of migrating from the traditional database (such as ORACLE) to a novel database (such as GaussDB), due to the fact that different databases have greater differences in database object characteristics, DML syntax, function entry and exit formats and the like, a situation that a service execution error occurs after migration frequently according to a conventional migration method.

For example, there are some cases where the approximate syntax can be compiled and run in both libraries, but the actual execution result is different, which eventually results in a service error, taking ORACLE and GaussDB as an example, and using a digital intercept system function trunc, where the two entries of the function are 164.67 and 1, respectively, and where the two database run results (out-entries) are the same and are both 164.6, if the second entry is-1.6, the ORACLE run result is 160, and the GaussDB run result is 100. The same function name and entry parameter can be seen, the results are very different in two database part scenes, and if the problem of correcting the type cannot be found in advance, the business accounting problem can be caused finally.

For the problems, in the related art, the problems are found by manually constructing test cases and performing performance test and comparison on the continuous databases, but the manual case construction cannot cover the actual production scene, a large number of problems can only be found after production, and the production risk is high.

For another example, the optimizer may have a large difference, and there are cases where the partial SQL has good or bad, and even there may be a performance gap of more than ten times, which affects the stable operation of the post-migration industrial business system, and performance deterioration may cause transaction timeout, and even may cause the database connection to be used up and unable to serve outside.

In view of this, an embodiment of the present disclosure provides a database performance determining method, including:

Fig. 1 schematically illustrates an application scenario diagram of a database performance determination method, apparatus, device, medium, and program product according to embodiments of the present disclosure.

As shown in fig. 1, the application scenario 100 according to this embodiment may include a terminal device 101, a service system 102, a source database 103, and a target database 104, where the terminal device 101, the service system 102, the source database 103, and the target database 104 may communicate with each other through a network, and the network may include various connection types, such as a wired connection, a wireless communication link, a fiber optic cable, and so on.

A user may use the terminal device 101 to interact with a server of the business system 102 over a network to receive or send messages or the like. Various messaging client applications, such as shopping applications, web browser applications, search applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only) may be installed on the terminal device 101.

The terminal device 101 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.

The business system 102 may include one or more servers, which may be servers providing various services, such as a background management server (for example only) providing support for websites browsed by users using the terminal devices 101. The background management server may analyze and perform other processing on the received data such as the user request, and feed back a processing result (e.g., a webpage, information, or data obtained or generated according to the user request) to the terminal device.

In an application scenario of the embodiment of the present disclosure, during the service provided by the service system 102 to the outside, service migration and data migration between the source database 103 and the target database 104 need to be performed. Before formally executing service migration, in order to know the performance of the target database 104, in the process of executing the data migration, a standby database of a source database is set, and under the condition that service data of the source database 103 is migrated to the target database 104 and the target standby database, a first format database operation language running in the source database 103 is recorded and converted into a second format database operation language matched with the target database 104; performing a first playback operation on the service data in the target standby database by using the first format database operation language to obtain first playback data, and performing a second playback operation on the service data in the target database 104 by using the second format database operation language to obtain second playback data; finally, the performance of the target database 104 after the data migration is determined according to the first playback data and the second playback data.

It should be understood that the number of terminal devices, service servers, database servers in fig. 1 is merely illustrative. Any number of terminal devices, service servers, database servers may be provided, as desired for implementation.

It should be noted that the method and the apparatus for determining database performance according to the embodiments of the present disclosure may be applied to the technical field of databases, may also be applied to the technical field of finance, and may also be applied to any fields other than the technical field of databases and the field of finance.

The database performance determination method of the disclosed embodiment will be described in detail below with reference to fig. 2 to 5 based on the scenario described in fig. 1.

FIG. 2 schematically shows a flow chart of a database performance determination method according to an embodiment of the present disclosure.

As shown in fig. 2, the database performance determination method of this embodiment includes operations S201 to S204.

In operation S201, under the condition that the service data of the source database is migrated to the target database and the target backup database, a first format database operation language (application SQL) running in the source database is recorded, where the first format database operation language is used to perform a predetermined data operation on the service data in the source database, the source database and the target database are different databases, and the source database and the target backup database are the same database;

converting the first format database operation language into a second format database operation language (application SQL) matched with the target database in operation S202;

in operation S203, performing a first playback operation on the service data in the target backup library by using the first format database operation language to obtain first playback data, and performing a second playback operation on the service data in the target database by using the second format database operation language to obtain second playback data;

in operation S204, performance of the target database after the data migration is determined according to the first playback data and the second playback data.

In an application scenario of the heterogeneous database migration in the embodiment of the present disclosure, during a period in which a service system provides a service to the outside, service migration and data migration between a source database and a target database need to be performed, where the source database and the target database are different databases, the source database is, for example, an ORACLE database, and the target database is a GaussDB database.

Before formally executing the service migration, in order to determine the performance of the target database, in the embodiment of the present disclosure, through the above operations S201 to S204, the recording, conversion, and playback comparison of the application SQL are performed to obtain the performance of the target database, and in a case where there is no problem in determining the performance of the target database, the service migration is executed again to avoid a situation where the service execution is incorrect after the migration.

According to the embodiment of the present disclosure, since the source database is connected to the service system that provides the service to the outside before migration is performed, the source database cannot perform SQL playback in the source database during the service providing to the outside, and therefore, in the embodiment of the present disclosure, by setting the backup database that is the same as the source database, that is, the target backup database and the target backup database perform SQL playback respectively and compare the playback results, and the results are used as comparison results of SQL performance in the source database and the target database.

According to the embodiment of the disclosure, during the period of providing the service to the outside through the source database, firstly, the service data of the source database is migrated to the target database and the target standby database, so as to prepare the data for the subsequent SQL playback. Then, in operation S201, the SQL executed in the source database is recorded. The SQL text running on the source database is recorded through network recording, the recorded SQL is played back on the target standby database and the target database through JAVA programs respectively, two playback path data are consistent before playback is started, and the SQL is sequenced according to the occurrence time of a source end during playback.

According to the embodiment of the present disclosure, since different databases may have respective SQL dialects (or database dialects, which refer to variants of structured query languages for accessing databases, different dialects may also be supported according to different specific database systems, that is, an operation language of a certain database may not only support the SQL standard, but also have some own unique syntax, which is called dialects), for example, the CONNECT BY statement may only be used in ORACLE. Therefore, in order to avoid the situation that the SQL cannot be played back or the playback error occurs in the target database, in operation S202, the first format database operation language is converted into the second format database operation language matching the target database, so that the problem that the SQL format does not match the database can be solved.

According to the embodiment of the present disclosure, in operation S204, the performance of the target database after the data migration is determined according to the first playback data and the second playback data, which may be respectively determining the service performance and the SQL operation performance of the target database after the data migration.

Specifically, the service performance may be determined by comparing the service execution results respectively represented by the first playback data and the second playback data to obtain the service performance good and bad results, and avoid the execution result error, for example, taking ORACLE and GaussDB as an example, using a digital interception system function trunc, where two entries of the function are 164.67 and 1, respectively, and if the two entries (out) are the same, both are 164.6, if the second entry is-1.6, the ORACLE operation result is 160, and the GaussDB operation result is 100. The same function name and entry parameter can be seen, the result is very different in two database partial scenes, and the problems need to be discovered and corrected in advance through SQL playback. The purpose of analyzing the service performance is to eliminate some change points (such as time stamps, and the operation results are different every time), compare the change points with the operation results of the same SQL twice, verify the migration correctness by using the actual production service scene, and reduce the service risk caused by migration transformation. In the previous example, when the entry references of the system function trunc are 164.67 and-1.6, the difference exists between the running results of the two libraries, so that problems can be found during the comparison of playback analysis functions, a function difference list is generated, and the application development system is submitted for modification.

Specifically, the determination of the SQL operation performance may be to compare response times of the SQL execution of the first playback operation and the second playback operation, for example, when the same SQL query statement is executed, a corresponding time of executing the SQL on the source database is 2 seconds, a corresponding time of executing the SQL on the target database is 10 seconds, and a difference between the two is large, which may be considered that the SQL operation performance of the target database is poor.

According to the embodiment of the disclosure, the business performance and the SQL operation performance of the target database after data migration are analyzed, the output result of the same SQL is played back twice by comparison, the correctness of migration is verified by using an actual production business scene, the business risk caused by migration transformation is reduced, and the performance problem caused by migration can be found and solved in advance by comparing the response time of SQL playback twice and carrying out SQL early warning that the analysis response time is increased by more than a threshold value.

According to the embodiment of the disclosure, through the above operations S201 to S204, recording, converting, and playback comparing the application SQL to obtain the performance of the target database, and executing service migration again when it is determined that there is no problem in the performance of the target database, thereby avoiding a situation of service execution error after migration. The method realizes the playback of the full real data, covers various common production transactions in real service scenes, reduces the service risk caused by migration transformation through the recording and playing of the real production transactions, solves the problem that the manual case construction cannot cover the actual production scenes, reduces the production risk, is convenient to find potential function and performance problems of the migration in time, reduces the influence of heterogeneous migration and emergency backspace on a service system, simultaneously reduces the workload of the heterogeneous migration of the database through executing automatic recording and playing, and reduces the error probability caused by manual modification. The method has universality and universality, has no close coupling relation with a specific database, and is suitable for enterprises using various databases.

According to the embodiment of the disclosure, since different databases have their own dialects and have great differences in database object characteristics, DML syntax, function entry and exit formats, and the like, format conversion and installation of the database object are required before formal migration is performed. Specifically, before migrating the service data of the source database to the target database and the target backup database, the format of the database object of the source database is converted into an object format matched with the target database, and the database object after the format conversion is installed in the target database.

Specifically, the format of the database object of the source data may be converted into an object format matching the target database, and the DDL (database definition language) of the source database and the application SQL (database operation language) associated with the source database may be converted into a format matching the target database.

For example, the DDL object of the source database, including tables, indexes, stored procedures, internal service logic codes, views, etc., is extracted, the DDL (including the total SQL, PL/SQL statements, etc., of the stored procedures) is automatically converted into the target database format based on rules, and a small amount of manual modification is assisted to obtain the DDL conforming to the target database object. Meanwhile, the application SQL (used for executing preset business operation on business data) associated with the source database is converted into the SQL conforming to the target database format, so that the aim of reducing the development and modification workload can be achieved through automatic format conversion.

After the format of a database object of source data is converted into an object format matched with a target database, database object data and attributes of the source database and the target database are compared through a system view, omission is guaranteed, a DDL version of the target database object after format conversion is installed in the target database, and application SQL is installed in an application server of a business system.

According to the embodiment of the disclosure, the format of the database object of the source data is converted into the object format matched with the target database, and the database object after the format conversion is installed in the target database, so that the problem of database migration caused by difference in the database object formats and the like of different databases is solved, the stability of database migration is improved, and the migration workload can be reduced and the error probability caused by manual modification can be reduced through automatic conversion of the database object and SQL.

According to the embodiment of the disclosure, in the process of executing database migration, the service data of the source database needs to be migrated to the target database. The service data comprises forward migration full data and forward migration incremental data, the forward migration full data is transaction data generated by an application server connected with a source database and providing services to the outside before a time period for executing data migration, and the forward migration incremental data is transaction data generated by an application server connected with the source database and providing services to the outside within the time period for executing data migration.

According to the embodiment of the disclosure, because data migration needs a certain time, during the data migration, incremental service data generated by a source database providing a service externally also needs to be migrated to a target database together, so that the data of the source database and the target database are leveled, and data loss is avoided. Meanwhile, incremental data are migrated to a target database in advance, after performance comparison tests are completed, services are conveniently and quickly switched from a source database to the target database, and service traffic is switched from the source database to the target database through minute-level service traffic interception, so that the service time influenced by heterogeneous migration of the database is reduced to a minute level, the influence of database migration on online transaction services is reduced, and service continuity is improved.

According to the embodiment of the disclosure, the forward migration full amount data and the forward migration incremental data are migrated to the target database, and data replication can be realized by adopting a logic replication method. Incremental data can be copied by analyzing database logs to obtain SQL execution logic of related data, and the data logic is copied to a target database by executing corresponding SQL on the target database.

According to the embodiment of the present disclosure, the performance of the target database is obtained through the recording, converting, and playback comparison of the SQL, and when it is determined that the performance of the target database after data migration meets the preset performance judgment condition, that is, when the performance of the target database is stable, service migration between two databases is executed, specifically: and migrating the service data of the source database to the target database again, and switching the service of the application server which provides the service to the outside from the source database to the target database.

According to the embodiment of the present disclosure, during the period when the source database provides external services, after the SQL playback operation is performed, because the source database and the target database are different, partial SQL playback result data of the source database and the target database may be inconsistent, and after the SQL playback, partial data differences may exist between data in the target database and data in the source database.

According to an embodiment of the present disclosure, specifically, switching traffic of an application server that provides services to the outside from a source database to a target database includes:

firstly, switching part of services of an application server from a source database to a target database;

and then, under the condition that the execution result of part of the services of the application server related to the target database is correct, switching all the services of the application server from the source database to the target database.

According to the embodiment of the disclosure, during production, a small part of services are migrated to the target database in advance through the gray level stream-cutting strategy for testing, the stability and the safety of the target database are verified, and under the condition that the test result of the part of services is not problematic, all services are migrated, so that the risk of full stream-cutting can be reduced.

According to the embodiment of the disclosure, in the process of performing gray level cut-flow, services of different applications are distinguished, the services are divided into a first class of service and a second class of service according to service importance (for example, classified according to function + mechanism), and different gray level cut-flow strategies are executed on the first class of service and the second class of service.

Specifically, for the first type of service with lower importance, the switching from the source database to the target database includes: part of the read-only transaction services of the first type of services of the application server are switched to the target database from the source database, for example, part of the read-only transactions which can accept short-time data replication delay (such as 5 seconds) can be switched to the target database (such as asset query, account balance query and the like), gray level production is carried out, and after the test is passed, all the services are cut into streams.

Specifically, for the second type of service with higher importance, the switching from the source database to the target database includes: dividing a second class of service of the application server into a plurality of unit sub-services, wherein each unit sub-service is associated with one data subset in the service data (the service data can be divided into small logic data subsets according to cuttable units such as mechanisms); and switching the target unit sub-service in the plurality of unit sub-services from the target data subset in the source database to the target data subset in the target database. The target unit sub-transaction is associated with a target subset of data, e.g., the target unit sub-transaction is a deposit transaction for an account and the target subset of data is transaction data associated with the deposit transaction.

For the second type of service with higher importance, each data subset in the service data is respectively associated with different unit sub-services, for example, the data subset 1 is data related to a withdrawal service, the data subset 2 is data related to a deposit service, the data subsets are loosely coupled, when service migration is performed, migration is performed by taking the unit sub-services as a whole, and the data subsets in the source database of each unit sub-service are integrally switched to the data subsets in the target database. Therefore, the condition that the same data cannot be updated simultaneously in the source database and the target database can be avoided, and service execution errors are avoided. In the flow switching process, a small number of cutting units (unit sub-services) are tested and then gradually increased to 100%, and the influence of fault services during the test is reduced. During the period of cutting flow according to cutting units or switching full-volume transactions, transaction interception and caching are carried out for a short time, and a record is guaranteed not to be updated on two databases at the same time.

According to the embodiment of the disclosure, by the gray level stream switching method, flexible scheduling of business transactions between a migration source environment and a migration target environment can be realized, the transactions are intercepted and scheduled in the business stream switching process through trading the gray level stream switching part to the target database in advance, the transactions are switched between the transaction operation units corresponding to the source database and the target database, and short time and controllable risk of the business emergency rollback caused by production stream switching and problems are ensured.

According to the embodiment of the present disclosure, after switching the service of the application server providing the service to the outside from the source database to the target database, the method further includes performing reverse migration on the reverse migration incremental data of the target database to the source database, where the reverse migration incremental data is: after the service of the application server is switched from the source database to the target database, the application server connected with the target database provides the transaction data generated by the service to the outside. And under the condition that the performance of the target database does not meet the preset performance judgment condition, the service of the application server providing the service externally is switched back to the source database from the target database.

According to the embodiment of the disclosure, after the service of the application server is switched from the source database to the target database, the application server provides service to the outside through the target database, the application server connected with the target database provides service to the outside, incremental transaction data can be generated, and the incremental data is reversely copied to the source database, so that the data of the source database and the target database can be kept consistent.

According to the embodiment of the disclosure, in the process of performing gray level cut flow aiming at the second type of service with higher importance degree, a small number of unit sub-services are tested and migrated firstly, and bidirectional copying is performed according to the filter condition of a cutting unit, namely, the data subsets associated with the non-test unit sub-services are copied to the target database from the source database, and the data subsets associated with the test unit sub-services are copied to the source database from the target database, so that the service influence of faults during the test point period can be reduced through the gray level cut flow scheme.

Fig. 3 schematically illustrates a system diagram of a database performance determination method according to an embodiment of the present disclosure.

As shown in fig. 3, a system to which the database performance determining method of the embodiment of the present disclosure may be applied includes a source database, a target backup database, and a service system providing a service to the outside.

The business system is responsible for providing a transaction running environment, which can be an application container or an application server. The target database is a migrated target database, and is prepared to accept production transactions for a new environment. The target backup database is a source database backup database established by data replication, has the same database object as the source database, and can be used for data stock and incremental replication from the source database to the backup database.

Before the migration of the database, the source database is connected with the service system, SQL operations such as increasing, deleting, checking and modifying data and the like are carried out on the source database, and production transactions only run on the source database.

Before formal service migration, format conversion and installation of a database object are required. Specifically, the format of the database object of the source data may be converted into an object format matching the target database, and the DDL (database definition language) of the source database and the application SQL (database operation language) associated with the source database may be converted into a format matching the target database. After the format of a database object of source data is converted into an object format matched with a target database, database object data and attributes of the source database and the target database are compared through a system view, omission is guaranteed, a DDL version of the target database object after format conversion is installed in the target database, and application SQL is installed in an application server of a business system.

Before formally executing the service migration, in order to determine the performance of the target database, the embodiment of the disclosure obtains the performance of the target database through recording, converting, and playback comparison of the application program SQL, and executes the service migration again under the condition that no problem exists in the performance of the target database, so as to avoid the occurrence of a condition that the service execution is wrong after the migration.

Specifically, firstly, the full data and the incremental data of the source database are respectively copied to the target database and the target backup database in a data logic copying mode, so as to prepare data for subsequent SQL playback.

And then stopping the data logic copying, ensuring that the data of the target backup library and the target database are consistent, recording an SQL text running on a source database through network recording, performing format conversion, and playing back the recorded SQL in the target backup library and the target database through JAVA programs respectively, specifically, performing a first playback operation on the service data by the SQL before format conversion in the target backup library to obtain first playback data, and performing a second playback operation on the service data by the SQL after format conversion in the target database to obtain second playback data. And comparing and analyzing the first playback data and the second playback data to determine the performance of the target database after data migration. Problems can be found during the comparison of playback analysis functions, a function difference list is generated, application development and modification are submitted, the response time of the SQL playback twice is compared, the SQL with the response time increasing over a threshold value is analyzed, and performance problems caused by migration are found and solved in advance.

After playback is finished, under the condition that the performance of the target database is determined to have no problem, the source database data is copied and synchronized to the target database, incremental data copying is continued, and the final consistency of the two database data is guaranteed.

And after the data of the two databases are leveled, performing service switching. In the process of executing the service switching, a strategy of gray level switching is adopted, for example, firstly, a switching part of read-only transaction is sent to an application server connected with a target database for trial verification, after the transaction can normally run without error, transaction interception is carried out, forward incremental replication (incremental replication from a source database to the target database) is stopped, the full-scale transaction of the production service is switched to the application server connected with the target database, at the moment, the transaction takes the target database as an updating database, reverse incremental data replication from the target database to the source database is established, and the final consistency of the data of the two databases is ensured so as to be convenient for emergency switching at any time.

After the full service is migrated to the target database, if the transaction running on the target database has serious performance or function problems, the production transaction can be dispatched back to the application server connected with the source database by intercepting and switching the transaction and waiting for finishing the data replication from the 4 target database to the source database, so that the rapid emergency rollback is realized, and the influence of the general service can be controlled within 2 minutes. If the emergency rollback occurs, the previous steps are repeated again until the commissioning is successful. The method of the embodiment of the disclosure reduces the possibility of service switching to a greater extent after recording and playing performance analysis and gray level switching read-only transaction verification. And after the production transaction is safely operated in the target database for a preset time period, such as one month, the source database and the matched resources are recycled.

Based on the database performance determination method, the disclosure also provides a database performance determination device. The apparatus will be described in detail below with reference to fig. 4.

Fig. 4 schematically shows a block diagram of the structure of a database performance determination apparatus according to an embodiment of the present disclosure.

As shown in fig. 4, the database performance determination apparatus 400 of this embodiment includes a recording module 401, a first conversion module 402, a playback module 403, and a determination module 404.

The recording module 401 is configured to record a first format database operation language running in a source database under the condition that service data of the source database is migrated to a target database and a target standby database, where the first format database operation language is used to perform a predetermined data operation on the service data in the source database, the source database and the target database are different databases, and the source database and the target standby database are the same database;

a first conversion module 402, configured to convert the first format database operation language into a second format database operation language matched with the target database;

a playback module 403, configured to perform a first playback operation on the service data in the target backup library by using the first format database operation language to obtain first playback data, and perform a second playback operation on the service data in the target database by using the second format database operation language to obtain second playback data;

a determining module 404, configured to determine, according to the first playback data and the second playback data, performance of the target database after the data migration.

According to the embodiment of the present disclosure, the recording module 401, the first conversion module 402, the playback module 403, and the determination module 404 are used for performing recording, conversion, and playback comparison on the application SQL to obtain the performance of the target database, and then performing service migration when it is determined that there is no problem in the performance of the target database, thereby avoiding a situation of service execution error after migration. The device realizes playback of the full real data, covers various common production transactions in real service scenes, reduces service risks caused by migration transformation through recording and playing of the real production transactions, solves the problem that manual case construction cannot cover actual production scenes, reduces production risks, is convenient for finding potential function and performance problems of migration in time, reduces the influence of heterogeneous migration and emergency backspacing on a service system, and simultaneously reduces the workload of heterogeneous migration of the database through executing automatic recording and playing and reduces the error probability caused by manual modification. The method has universality and universality, has no close coupling relation with a specific database, and is suitable for enterprises using various databases.

The migration module is used for migrating the service data of the source database to the target database again under the condition that the performance of the target database after data migration meets the preset performance judgment condition; the first switching module is used for switching the service of the application server which provides the service to the outside from the source database to the target database.

The anti-migration module is configured to, after switching a service of an application server that provides a service to the outside from a source database to a target database, anti-migrate reverse migration incremental data of the target database to the source database, where the reverse migration incremental data is: after the service of the application server is switched from the source database to the target database, the application server connected with the target database provides the transaction data generated by the service to the outside; and the second switching module is used for switching back the service of the application server which provides the service to the outside from the target database to the source database under the condition that the performance of the target database does not meet the preset performance judgment condition.

The first switching unit is used for switching part of services of the application server from a source database to a target database; and the second switching unit is used for switching all the services of the application server from the source database to the target database under the condition that the execution result of part of the services of the application server related to the target database is correct.

According to the embodiment of the present disclosure, the first switching unit includes a first switching subunit, configured to switch a part of the read-only transaction traffic of the first class of traffic of the application server from the source database to the target database.

The dividing subunit is used for dividing the second type of service of the application server into a plurality of unit sub-services, wherein each unit sub-service is associated with one data subset in the service data; and the second switching subunit is used for switching the target unit sub-service in the plurality of unit sub-services from the target data subset in the source database to the target data subset in the target database.

The second conversion module is used for converting the format of the database object of the source data into an object format matched with the target database before the service data of the source database is migrated to the target database and the target standby database; and the installation module is used for installing the database object after the format conversion in the target database.

According to the embodiment of the present disclosure, any plurality of the recording module 401, the first conversion module 402, the playback module 403, and the determination module 404 may be combined into one module to be implemented, or any one of them may be split into a plurality of modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of the other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the recording module 401, the first converting module 402, the playback module 403, and the determining module 404 may be implemented at least partially as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented by hardware or firmware in any other reasonable manner of integrating or packaging a circuit, or implemented in any one of three implementations of software, hardware, and firmware, or in a suitable combination of any of them. Alternatively, at least one of the recording module 401, the first conversion module 402, the playback module 403 and the determination module 404 may be at least partially implemented as a computer program module, which when executed may perform a corresponding function.

As shown in fig. 5, an electronic device 500 according to an embodiment of the present disclosure includes a processor 501 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. The processor 501 may comprise, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or associated chipset, and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), among others. The processor 501 may also include on-board memory for caching purposes. Processor 501 may include a single processing unit or multiple processing units for performing different actions of a method flow according to embodiments of the disclosure.

In the RAM 503, various programs and data necessary for the operation of the electronic apparatus 500 are stored. The processor 501, the ROM 502, and the RAM 503 are connected to each other by a bus 504. The processor 501 performs various operations of the method flows according to the embodiments of the present disclosure by executing programs in the ROM 502 and/or the RAM 503. Note that the programs may also be stored in one or more memories other than the ROM 502 and the RAM 503. The processor 501 may also perform various operations of method flows according to embodiments of the present disclosure by executing programs stored in the one or more memories.

According to an embodiment of the present disclosure, electronic device 500 may also include an input/output (I/O) interface 505, input/output (I/O) interface 505 also being connected to bus 504. The electronic device 500 may also include one or more of the following components connected to the I/O interface 505: an input portion 506 including a keyboard, a mouse, and the like; an output portion 507 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 508 including a hard disk and the like; and a communication section 505 including a network interface card such as a LAN card, a modem, or the like. The communication section 505 performs communication processing via a network such as the internet. A drive 510 is also connected to the I/O interface 505 as needed. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as necessary, so that a computer program read out therefrom is mounted on the storage section 508 as necessary.

The present disclosure also provides a computer-readable storage medium, which may be contained in the apparatus/device/system described in the above embodiments; or may exist separately and not be assembled into the device/apparatus/system. The computer-readable storage medium carries one or more programs which, when executed, implement the method according to an embodiment of the disclosure.

According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, a computer-readable storage medium may include ROM 502 and/or RAM 503 and/or one or more memories other than ROM 502 and RAM 503 described above.

Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the method illustrated in the flow chart. When the computer program product runs in a computer system, the program code is used for causing the computer system to realize the database performance determination method provided by the embodiment of the disclosure.

The computer program performs the above-described functions defined in the system/apparatus of the embodiments of the present disclosure when executed by the processor 501. The systems, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.

In one embodiment, the computer program may be hosted on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted, distributed in the form of a signal on a network medium, downloaded and installed through the communication section 505, and/or installed from the removable medium 511. The computer program containing program code may be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 505, and/or installed from the removable medium 511. The computer program, when executed by the processor 501, performs the above-described functions defined in the system of the embodiments of the present disclosure. The systems, devices, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.

In accordance with embodiments of the present disclosure, program code for executing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, these computer programs may be implemented using high level procedural and/or object oriented programming languages, and/or assembly/machine languages. The programming language includes, but is not limited to, programming languages such as Java, C + +, python, the "C" language, or the like. The program code may execute entirely on the user computing device, partly on the user device, partly on a remote computing device, or entirely on the remote computing device or server. In situations involving remote computing devices, the remote computing devices may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to external computing devices (e.g., through the internet using an internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Those skilled in the art will appreciate that various combinations and/or combinations of features recited in the various embodiments and/or claims of the present disclosure can be made, even if such combinations or combinations are not expressly recited in the present disclosure. In particular, various combinations and/or combinations of the features recited in the various embodiments and/or claims of the present disclosure may be made without departing from the spirit or teaching of the present disclosure. All such combinations and/or associations are within the scope of the present disclosure.

The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in the embodiments cannot be used advantageously in combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be devised by those skilled in the art without departing from the scope of the present disclosure, and such alternatives and modifications are intended to be within the scope of the present disclosure.

Claims

1. A database performance determination method, comprising:

under the condition that service data of a source database is migrated to a target database and a target standby database, recording a first format database operation language running in the source database, wherein the first format database operation language is used for executing a predetermined data operation on the service data in the source database, the source database and the target database are different databases, and the source database and the target standby database are the same database;

executing a first playback operation on the service data in the target standby database by using the first format database operation language to obtain first playback data, and executing a second playback operation on the service data in the target database by using the second format database operation language to obtain second playback data;

and determining the performance of the target database after data migration according to the first playback data and the second playback data.

2. The method of claim 1, wherein:

the service data includes forward migration full volume data and forward migration incremental data, the forward migration full volume data is transaction data generated by an application server connected with the source database and providing services to the outside before a time period for performing data migration, and the forward migration incremental data is transaction data generated by an application server connected with the source database and providing services to the outside within the time period for performing data migration.

3. The method of claim 1, further comprising:

under the condition that the performance of the target database meets the preset performance judgment condition after the data migration is determined, migrating the service data of the source database to the target database again;

4. The method of claim 3, further comprising, after the traffic of the application server to be served externally is switched from the source database to the target database:

reversely migrating the reverse migration incremental data of the target database to the source database, wherein the reverse migration incremental data is as follows: after the service of the application server is switched from the source database to the target database, the application server connected with the target database provides transaction data generated by the service to the outside; and

and under the condition that the performance of the target database does not meet a preset performance judgment condition, switching back the service of the application server providing the service to the outside from the target database to the source database.

5. The method of claim 3, wherein the switching traffic to the application server providing the service from the source database to the target database comprises:

switching part of services of the application server from the source database to the target database;

and under the condition that the execution result of the part of the service of the application server related to the target database is correct, switching all the services of the application server from the source database to the target database.

6. The method of claim 5, wherein the switching the portion of the traffic of the application server from the source database to the target database comprises:

and switching part of the read-only transaction service of the first type service of the application server from the source database to the target database.

7. The method of claim 5, wherein the switching the portion of the traffic of the application server from the source database to the target database comprises:

8. The method of claim 1, further comprising, prior to migrating the business data of the source database to the target database and the target backup database:

converting the format of the database object of the source data into an object format matched with a target database;

9. A database performance determination apparatus, comprising:

10. An electronic device, comprising:

one or more processors;

a storage device for storing one or more programs,

wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-8.

11. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 8.

12. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 8.