CN116662191A - Data testing method, device, computer equipment and storage medium - Google Patents
Data testing method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN116662191A CN116662191A CN202310664610.0A CN202310664610A CN116662191A CN 116662191 A CN116662191 A CN 116662191A CN 202310664610 A CN202310664610 A CN 202310664610A CN 116662191 A CN116662191 A CN 116662191A
- Authority
- CN
- China
- Prior art keywords
- target
- output table
- test task
- database
- table name
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 99
- 230000010076 replication Effects 0.000 claims abstract description 120
- 238000013101 initial test Methods 0.000 claims abstract description 115
- 238000000034 method Methods 0.000 claims abstract description 33
- 238000004590 computer program Methods 0.000 claims abstract description 29
- 238000004458 analytical method Methods 0.000 claims description 6
- 238000010276 construction Methods 0.000 claims description 2
- 238000010998 test method Methods 0.000 claims description 2
- 238000012545 processing Methods 0.000 abstract description 5
- 238000005516 engineering process Methods 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 8
- 229910021532 Calcite Inorganic materials 0.000 description 7
- 230000008569 process Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000005192 partition Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000011109 contamination Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 229910021389 graphene Inorganic materials 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Debugging And Monitoring (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present application relates to a data testing method, apparatus, computer device, storage medium and computer program product. Relates to the field of data processing, and can be used in the field of financial science and technology or other related fields. The method comprises the following steps: acquiring an initial test task and analyzing each database statement in the initial test task to obtain an output table name corresponding to each database statement; inquiring a target replication table according to a first identifier corresponding to the initial test task and a second identifier corresponding to the output table name aiming at each output table name in each database statement; under the condition that a target replication table exists, replacing the corresponding output table name in the database statement according to the replication table name of the target replication table meeting the preset data structure condition to obtain a target database statement; and constructing a target test task based on the multiple target database sentences. By adopting the method, the efficiency of the test task can be improved.
Description
Technical Field
The present application relates to the field of data processing technology, and in particular, to a data testing method, apparatus, computer device, storage medium, and computer program product.
Background
With the development of data analysis technology, data analysts typically analyze data in a database through correct SQL (Structured Query Language, a database language) logic to obtain index data. When encountering complex SOL logic, the data analyst can verify through multiple tests, thereby ensuring the correctness of the analysis task.
In the current data testing method, the output table contains the data required by the testing task, and the output table is the original data in the output table database. To prevent contamination of the data in the database by test tasks, the data in the replication table is used for analysis. Therefore, when the analyst executes the test task each time, the analyst manually establishes a copy table which is the same as the output table in the test task, replaces the output table with the copy table in the test task, obtains a replaced test task, and further executes the replaced test task.
However, in the current data testing method, operations such as table creation and replacement are manually performed each time a testing task is executed, which results in lower efficiency of the testing task.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a data testing method, apparatus, computer device, computer readable storage medium, and computer program product.
In a first aspect, the present application provides a data testing method. The method comprises the following steps:
acquiring an initial test task and analyzing each database statement in the initial test task to obtain an output table name corresponding to each database statement;
querying a target replication table according to a first identifier corresponding to the initial test task and a second identifier corresponding to the output table name aiming at each output table name in each database statement;
under the condition that the target copying table exists, replacing the corresponding output table name in the database statement according to the copying table name of the target copying table meeting the preset data structure condition to obtain a target database statement;
and constructing a target test task based on a plurality of target database sentences.
In one embodiment, the obtaining an initial test task and analyzing each database statement in the initial test task to obtain an output table name corresponding to each database statement includes:
responding to the submitting operation of an initial test task, and acquiring the initial test task;
analyzing each database statement contained in the initial test task to obtain an output table name contained in each database statement;
And determining a first identifier corresponding to the initial test task.
In one embodiment, each database statement includes at least one system variable therein, and the method further includes:
assigning values to the system variables in the database sentences in the initial test task according to a preset replacement rule to obtain a first test task;
the querying, for each output table name in each database statement, a target replication table according to a first identifier corresponding to the initial test task and a second identifier corresponding to the output table name, including:
and inquiring a target replication table according to the first identifier corresponding to the first test task and the second identifier corresponding to the output table name in each database statement.
In one embodiment, the querying the target replication table for each of the output table names in each of the database statements according to the first identifier corresponding to the initial test task and the second identifier corresponding to the output table name includes:
determining a second identifier corresponding to the output table name in a preset output table database aiming at each output table name in each database statement;
And determining whether a target replication table exists in a preset replication table database according to the first identifier corresponding to the initial test task and the second identifier corresponding to the output table name.
In one embodiment, after the querying the target replication table according to the first identifier corresponding to the initial test task and the second identifier corresponding to the output table name for each output table name in each database statement, the method further includes:
searching a target output table matched with the output table name in a preset output table database based on the output table name under the condition that the target copy table does not exist;
creating a target replication table consistent with the target output table structure; the copy table name of the target copy table is different from the output table name of the target output table;
and replacing the output table name with the copy table name of the target copy table in the database statement based on a preset replacement frame to obtain a target database statement.
In one embodiment, when the target replication table exists, replacing the output table name corresponding to the database statement according to the replication table name of the target replication table meeting a preset data structure condition, to obtain a target database statement, including:
Under the condition that the target copying table exists, inquiring a target output table corresponding to the output table name in a preset output table database;
judging whether the structure of the target output table is the same as the structure of the target replication table;
under the condition that the structure of the target output table is the same as that of the target replication table, confirming that the target replication table meets the preset data structure condition;
and replacing the output table name in the database statement with the copy table name of the target copy table to obtain the target database statement.
In one embodiment, after the determining whether the structure of the target output table and the structure of the target replication table are the same, the method further includes:
in the case where the structure of the target output table and the structure of the target copy table are not identical, the step of creating a target copy table in conformity with the structure of the target output table is performed.
In a second aspect, the application further provides a data testing device. The device comprises:
the analysis module is used for acquiring an initial test task and analyzing each database statement in the initial test task to obtain an output table name corresponding to each database statement;
The query module is used for querying a target replication table according to a first identifier corresponding to the initial test task and a second identifier corresponding to the output table name aiming at each output table name in each database statement;
the replacing module is used for replacing the corresponding output table name in the database statement according to the copy table name of the target copy table meeting the preset data structure condition under the condition that the target copy table exists, so as to obtain a target database statement;
and the construction module is used for constructing a target test task based on a plurality of target database sentences.
In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor which when executing the computer program performs the steps of:
acquiring an initial test task and analyzing each database statement in the initial test task to obtain an output table name corresponding to each database statement;
querying a target replication table according to a first identifier corresponding to the initial test task and a second identifier corresponding to the output table name aiming at each output table name in each database statement;
Under the condition that the target copying table exists, replacing the corresponding output table name in the database statement according to the copying table name of the target copying table meeting the preset data structure condition to obtain a target database statement;
and constructing a target test task based on a plurality of target database sentences.
In a fourth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:
acquiring an initial test task and analyzing each database statement in the initial test task to obtain an output table name corresponding to each database statement;
querying a target replication table according to a first identifier corresponding to the initial test task and a second identifier corresponding to the output table name aiming at each output table name in each database statement;
under the condition that the target copying table exists, replacing the corresponding output table name in the database statement according to the copying table name of the target copying table meeting the preset data structure condition to obtain a target database statement;
And constructing a target test task based on a plurality of target database sentences.
In a fifth aspect, the present application also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the steps of:
acquiring an initial test task and analyzing each database statement in the initial test task to obtain an output table name corresponding to each database statement;
querying a target replication table according to a first identifier corresponding to the initial test task and a second identifier corresponding to the output table name aiming at each output table name in each database statement;
under the condition that the target copying table exists, replacing the corresponding output table name in the database statement according to the copying table name of the target copying table meeting the preset data structure condition to obtain a target database statement;
and constructing a target test task based on a plurality of target database sentences.
The data testing method, the data testing device, the computer equipment, the storage medium and the computer program product are used for acquiring an initial testing task and analyzing each database statement in the initial testing task to obtain an output table name corresponding to each database statement; querying a target replication table according to a first identifier corresponding to the initial test task and a second identifier corresponding to the output table name aiming at each output table name in each database statement; under the condition that the target copying table exists, replacing the corresponding output table name in the database statement according to the copying table name of the target copying table meeting the preset data structure condition to obtain a target database statement; and constructing a target test task based on a plurality of target database sentences. By adopting the method, each output table name is obtained by analyzing each database statement in the initial test task, and the target replication table is inquired aiming at each output table name. Under the condition that the target replication table exists, the replication table name of the target replication table is replaced, so that a target database statement is obtained, automatic replacement of the output table and the replication table is realized, pollution to data of the output table database is avoided, and efficiency of a test task is improved.
Drawings
FIG. 1 is a flow chart of a data testing method in one embodiment;
FIG. 2 is a flow chart illustrating a step of parsing a database statement in one embodiment;
FIG. 3 is a diagram of the structure of a task table in one embodiment;
FIG. 4 is a flow diagram of the system variable assignment step in one embodiment;
FIG. 5 is a flow diagram of a process for querying a target copy table in one embodiment;
FIG. 6 is a schematic diagram of a replication table in one embodiment;
FIG. 7 is a flow diagram of an alternative output table name step in one embodiment;
FIG. 8 is a flow chart illustrating the steps for creating a target copy table in one embodiment;
FIG. 9 is a flow chart of a data testing method in another embodiment;
FIG. 10 is a flow chart of a data testing method according to another embodiment;
FIG. 11 is a flow chart of a data testing method in one embodiment;
FIG. 12 is a flow chart illustrating a method for controlling a service module to perform data testing in one embodiment;
FIG. 13 is a block diagram of a data testing device in one embodiment;
fig. 14 is an internal structural diagram of a computer device in one embodiment.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
In one embodiment, as shown in fig. 1, a data testing method is provided, and the method is applied to a computer device for illustration, and includes the following steps:
step 102, obtaining an initial test task and analyzing each database statement in the initial test task to obtain an output table name corresponding to each database statement.
Wherein the initial test task comprises at least one database statement (Structured Query Language, SQL). The database statement is a database language having various functions such as data manipulation and data definition.
In practice, the target user submits the initial test task through a display screen in the computer device before the computer device acquires the initial test task. And the computer equipment responds to the submitting operation of the initial test task and acquires the initial test task. Then, the computer device analyzes each database statement contained in the initial test task to obtain an output table name and a system variable corresponding to each database statement in each database statement. After obtaining the output table name and the system variable corresponding to the database statement, the computer equipment determines a first identifier corresponding to the initial test task.
Step 104, for each output table name in each database statement, inquiring the target replication table according to the first identifier corresponding to the initial test task and the second identifier corresponding to the output table name.
In implementation, before querying the target replication table corresponding to each output table name, the computer device needs to determine the second identifier corresponding to the output table name in a preset output table database. Then, the computer device queries the target replication table for each output table name in each database statement based on the first identification corresponding to the initial test task and the second identification corresponding to the output table name.
And step 106, under the condition that the target replication table exists, replacing the corresponding output table name in the database statement according to the replication table name of the target replication table meeting the preset data structure condition, and obtaining the target database statement.
In practice, the replacement framework and data structure conditions are preset in the computer device. And the computer equipment judges whether the target copying table meets the preset data structure condition under the condition that the target copying table exists. And under the condition that the target replication table meets the data structure condition, the computer equipment replaces the output table name in the database statement with the replication table name of the target replication table based on the replacement frame to obtain the target database statement.
And step 108, constructing a target test task based on the multiple target database sentences.
In implementations, the computer device constructs a target test task based on the plurality of target database statements. Then, the computer device executes the target test task to obtain a test result.
In an alternative embodiment, after completion of execution of the target test task, the computer device clears the target replication table in accordance with a preset clearing period.
Alternatively, the cleaning period may be 7 days or 10 days, which is not limited in the embodiment of the present application.
In the data testing method, each output table name is obtained by analyzing each database statement in the initial testing task, and the target replication table is queried for each output table name. Under the condition that the target replication table exists, the replication table name of the target replication table is replaced, so that a target database statement is obtained, automatic replacement of the output table and the replication table is realized, pollution to data of the output table database is avoided, and efficiency of a test task is improved.
In one embodiment, the specific process in step 102 includes:
step 202, responding to the submitting operation of the initial testing task, and acquiring the initial testing task.
In practice, the target user submits an initial test task to the computer device through a display screen in the computer device prior to performing the data testing method. And the computer equipment responds to the submitting operation of the initial test task and acquires the initial test task.
Alternatively, the target user may be a data analyst or a database manager, and the present application is not limited to the target user.
And 204, analyzing each database statement contained in the initial test task to obtain the output table name contained in each database statement.
The output table is a table related to INSERT in the SQL sentence, and the input table is a table related to SELECT in the SQL sentence.
In implementation, the computer device parses each database statement included in the initial test task to obtain an output table name and a system variable included in each database statement.
In an alternative embodiment, a database statement is included in the initial test task, the database statement being as follows:
INSERT OVERWRITE TABLE db.product_insight PARTITION(dt＝'${date}')
SELECT name,SUM(price)FROM db.product_detail
where dt＝'${date}'GROUP BY name；
the computer device parses the database statement to obtain an output table name db. Product_weight, an input table name db. Product_detail, and a system variable $ { date }, which are contained in the database statement.
Step 206, determining a first identifier corresponding to the initial test task.
Wherein the first identifier corresponding to the initial test task is an ID (Identity document, unique code) of the initial test task.
In implementation, when the initial test task is created, the computer device stores data in the initial test task into a task table corresponding to the initial test task. The computer equipment determines a first identifier corresponding to the initial test task in the task table.
In an alternative embodiment, the structure of the task table corresponding to the initial test task preset in the computer device is shown in fig. 3. The computer equipment determines a first identification of the initial test task in a task table corresponding to the initial test task, and obtains a task_id of the initial test task.
In this embodiment, an initial test task is obtained and each database statement in the initial test task is parsed to obtain an output table name in each database statement, so that the output table name is convenient to replace subsequently, and a first identifier of the initial test task is determined, so that a target replication table is convenient to search subsequently.
In one embodiment, each database statement contains at least one system variable, which is assigned prior to querying the target replication table. As shown in fig. 4, the data testing method further includes:
Step 402, assigning values to all system variables in the database sentences in the initial test task according to preset replacement rules to obtain a first test task.
In an implementation, the computer device is preset with replacement rules. The computer equipment assigns values to the system variables according to preset replacement rules aiming at each system variable in each database statement to obtain assigned database statements. Then, the computer device constructs a first test task according to the plurality of assigned database statements.
In an alternative embodiment, the preset replacement rule in the computer device is to replace the current time with a date variable minus date data in a day format yyyy-MM-dd. For example, a database statement in a computer device is as follows:
INSERT OVERWRITE TABLE db.product_insight PARTITION(dt＝'${date}')
SELECT name,SUM(price)FROM db.product_detail
where dt＝'${date}'GROUP BY name；
wherein, the system variable is $ { date }, in SQL.
If the submission time of the initial test task is 2023-04-15:12:05:23, the computer equipment assigns 20230414 to the system variable, and the assigned database statement is obtained as follows:
INSERT OVERWRITE TABLE db.product_insight PARTITION(dt＝'20230414')
SELECT name,SUM(price)FROM db.product_detail
where dt＝'20230414'GROUP BY name；
the computer device overwrites the data into the partition of the db.product_weight table with dt of 20230414.
After completion of step 402, a first test task is obtained. Thus, the specific implementation of step 104 includes:
And step 404, inquiring the target replication table according to the first identifier corresponding to the first test task and the second identifier corresponding to the output table name in each database statement.
In implementation, for each database statement in the first test task, the computer device queries the target replication table in a preset replication table database according to the first identifier corresponding to the first test task and the second identifier corresponding to the output table name in the database statement.
In the embodiment, the system variable in each database statement is copied based on the preset replacement rule, so that the manual replacement of the system variable is avoided, the automatic replacement of the system variable is realized, and the efficiency of the test task is improved.
In one embodiment, as shown in FIG. 5, the specific process of step 104 includes:
step 502, determining a second identifier corresponding to the output table name in a preset output table database according to each output table name in each database statement.
The target output table comprises an output table name and a second identifier corresponding to the output table name. The second identifier corresponding to the output table name is the ID of the target output table.
In an implementation, the computer device queries, for each output table name in each database, a target output table corresponding to the output table name in a preset output table database. The computer device then determines a second identification corresponding to the output table name in the target output table.
Step 504, determining whether a target replication table exists in a preset replication table database according to the first identifier corresponding to the initial test task and the second identifier corresponding to the output table name.
Wherein, the computer equipment is preset with a copy table database which is used for storing the copy table which is already established. Each replication table stores a first identifier corresponding to the test task and a second identifier corresponding to the output table name, as shown in fig. 6. In fig. 6, the test table ID is a second identifier corresponding to the output table name, and the task ID is a first identifier corresponding to the test task.
In implementation, the computer device queries an initial target replication table corresponding to the first identifier in a preset replication table database according to the first identifier corresponding to the initial test task. If the initial target replication table exists in the replication database, the computer equipment confirms whether the target replication table exists in the initial target replication table according to the second identification corresponding to the output table name. If the original target copy table does not exist in the copy table database, the computer equipment confirms that the target copy table does not exist in the copy table database.
In this embodiment, the second identifier corresponding to the output table name is determined, and the target copy table is queried according to the first identifier and the second identifier, so that the output table name is conveniently replaced according to the target copy table, and the storage space for storing the copy table is reduced.
In one embodiment, in the absence of a target copy table, it is necessary to determine a target output table from the output table names and create the target copy table. Then, the output table name in the database statement is replaced with the copy table name of the target copy table. As shown in fig. 7, after the step 104 is performed, the specific processing procedure of the data testing method further includes:
step 702, in the case that the target copy table does not exist, searching a target output table matched with the output table name in a preset output table database based on the output table name.
In practice, the output table database is preset in the computer device. The computer equipment searches a target output table matched with the output table name in a preset output table database based on the output table name.
At step 704, a target replication table is created that is consistent with the target output table structure.
Wherein the copy table name of the target copy table is different from the output table name of the target output table.
In an implementation, a computer device creates a target replication table from a target output table that is consistent with the target output table structure. Then, the computer copies the data in the target output table to the target copy table, and gives the target user a right to read and write the target copy table.
And step 706, replacing the output table name with the copy table name of the target copy table in the database statement based on a preset replacement frame to obtain the target database statement.
Wherein, a replacement frame is preset in the computer equipment, and the replacement frame is Apache Calcite (a database management frame).
In implementation, the computer device replaces the output table name with the copy table name of the target copy table in the database statement based on the preset Apache Calcite to obtain the target database statement.
In the embodiment, under the condition that the target replication table does not exist, the target replication table is created, the replication table name of the target replication table is replaced, so that a target database statement is obtained, automatic replacement of the output table and the replication table is realized, pollution to data of the output table database is avoided, and efficiency of a test task is improved.
In one embodiment, as shown in FIG. 8, the specific process of step 106 includes:
step 802, in the case that the target copy table exists, querying a preset output table database for a target output table corresponding to the output table name.
In practice, the output table database is preset in the computer device. The computer equipment searches a target output table matched with the output table name in a preset output table database based on the output table name.
Step 804, determining whether the structure of the target output table and the structure of the target copy table are the same.
In an implementation, a computer device determines whether a structure of a target output table and a structure of a target replication table are the same.
Step 806, in the case that the structure of the target output table is the same as the structure of the target copy table, confirming that the target copy table satisfies the preset data structure condition.
In implementation, a data structure condition is preset in the computer device, where the data structure condition is that the structure of the target output table is the same as that of the target copy table. The computer device confirms that the target copy table satisfies the data structure condition in the case that the structure of the target output table and the structure of the target copy table are the same.
And 808, replacing the output table name in the database statement with the copy table name of the target copy table to obtain the target database statement.
In implementation, the computer device replaces the output table name with the copy table name of the target copy table in the database statement based on a preset replacement frame to obtain the target database statement.
In the embodiment, under the condition that the target replication table exists, the replication table name of the target replication table is replaced to obtain the target database statement, so that the automatic replacement of the output table and the replication table is realized, the pollution to the data of the output table database is avoided, and the efficiency of the test task is improved.
In one embodiment, in the case where the structure of the target output table and the structure of the target copy table are not identical, it is necessary to create the target copy table identical to the structure of the target output table. After the execution of the step 804 is performed,
in the case where the structure of the target output table and the structure of the target copy table are not identical, the step of creating the target copy table in conformity with the structure of the target output table is performed.
In an implementation, the computer device creates a target replication table consistent with the structure of the target output table from the target output table if the structure of the target output table and the structure of the target replication table are not the same. Then, the computer device copies the data in the target output table to the target copy table, and gives the target user a right to read and write the target copy table.
In the embodiment, under the condition that the structures of the target replication table and the target output table are inconsistent, the target replication table is created, the replication table names of the target replication table are replaced, and the target database statement is obtained, so that the automatic replacement of the output table in the output table database and the replication table in the replication table database is realized, the pollution to the data of the output table database is avoided, and the efficiency of the test task is improved.
In an alternative embodiment, as shown in fig. 9, the data testing method includes: the method comprises the steps that the computer equipment obtains an initial test task and analyzes each database statement in the initial test task to obtain an output table name corresponding to each database statement. And the computer equipment queries the target replication table according to the first identifier corresponding to the initial test task and the second identifier corresponding to the output table name aiming at each output table name in each database statement. The computer device determines whether a target copy table corresponding to the output table name exists. In the case where the target copy table exists, the computer apparatus determines whether the structure of the target copy table coincides with the structure of the target output table corresponding to the output table name. And under the condition that the structure of the target replication table is consistent with that of the target output table, the computer equipment replaces the corresponding output table name in the database statement according to the replication table name of the target replication table to obtain the target database statement.
In the case where there is no target copy table or in the case where the structure of the target copy table is inconsistent with the structure of the target output table, the computer apparatus creates a target copy table consistent with the structure of the target output table. And then, the computer equipment replaces the corresponding output table name in the database statement according to the copy table name of the target copy table to obtain the target database statement. The computer device constructs a target test task based on the plurality of target database statements. The computer device then performs the target test task.
In an alternative embodiment, as shown in fig. 10, the implementation of the data testing method specifically includes:
step 1001, submit test run task.
In practice, a data analyst submits test execution tasks to a computer device. And responding to the submitting operation of the test running task, and acquiring the test running task by the computer equipment to obtain an initial test task. Then, the computer device analyzes each database statement contained in the initial test task to obtain an output table name and a system variable contained in each database statement.
Step 1002, the system variables in the task are replaced with specific values.
In implementation, the computer device assigns values to each system variable in the database statement in the initial test task according to a preset replacement rule for each database statement in the initial test task, so as to obtain a first test task.
Step 1003, analyzing an output table in the task, creating tables with different table names in the same structure, and giving authority to an analyst.
In an implementation, a computer device creates a target replication table that is consistent with the target output table structure from a target output table corresponding to the output table name. Then, the computer copies the data in the target output table to the target copy table, and gives the target user a right to read and write the target copy table.
And step 1004, replacing the output table in the task with a newly built table through calcite.
Wherein, the Calcite is Apache Calcite.
In implementation, the computer device replaces the output table name with the copy table name of the target copy table in the database statement based on the preset Apache Calcite to obtain the target database statement.
In step 1005, an instance is generated and submitted to the execution service run.
In implementation, the computer device builds an instance (target test task) based on a plurality of target database statements. The computer device then submits the instance to the executive service for execution.
As shown in fig. 11, a control service module for performing the data testing method is integrated in the computer device. And submitting the test running task to the control service by the target user, wherein the control service in the computer equipment obtains an initial test task. Then, the control service in the computer device performs a data test method based on the initial test task, resulting in an instance (target test task). The computer device executes the instance.
The specific process of controlling the service module to perform the data testing method in the computer device is shown in fig. 12. Specifically, the computer device analyzes each database statement contained in the test running task to obtain an output table name contained in each database statement. Then, the computer device creates a target copy table in accordance with the target output table structure from the target output table corresponding to the output table name. Then, the computer copies the data in the target output table to the target copy table, and gives the target user a right to read and write the target copy table. And the computer equipment replaces the output table name with the copy table name of the target copy table in the database statement based on the preset Apache Calcite to obtain the target database statement. The computer device builds an instance (target test task) based on the plurality of target database statements.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides a data testing device for realizing the above related data testing method. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation of one or more embodiments of the data testing device provided below may refer to the limitation of the data testing method hereinabove, and will not be repeated herein.
In one embodiment, as shown in FIG. 13, there is provided a data testing apparatus 1300 comprising: parsing module 1301, querying module 1302, replacing module 1303 and constructing module 1304, wherein:
the parsing module 1301 is configured to obtain an initial test task and parse each database statement in the initial test task to obtain an output table name corresponding to each database statement.
The query module 1302 is configured to query, for each output table name in each database statement, the target replication table according to the first identifier corresponding to the initial test task and the second identifier corresponding to the output table name.
And the replacing module 1303 is used for replacing the corresponding output table name in the database statement according to the copy table name of the target copy table meeting the preset data structure condition under the condition that the target copy table exists, so as to obtain the target database statement.
A building module 1304 for building a target test task based on the plurality of target database statements.
In an exemplary embodiment, parsing module 1301 includes:
and the acquisition sub-module is used for responding to the submitting operation of the initial test task and acquiring the initial test task.
And the analysis sub-module is used for analyzing each database statement contained in the initial test task to obtain the output table name contained in each database statement.
And the first determination submodule is used for determining a first identifier corresponding to the initial test task.
In an exemplary embodiment, each database statement contains at least one system variable, and the data testing apparatus 1300 further comprises:
and the assignment sub-module is used for assigning values to all system variables in the database sentences in the initial test task according to a preset replacement rule aiming at each database sentence to obtain a first test task.
Wherein, the query module 1302 includes:
and the first query sub-module is used for querying the target replication table according to the first identifier corresponding to the first test task and the second identifier corresponding to the output table name in each database statement.
In an exemplary embodiment, the query module 1302 includes:
the second determining sub-module is used for determining a second identifier corresponding to the output table name in a preset output table database according to each output table name in each database statement.
And the third determination submodule is used for determining whether a target replication table exists in a preset replication table database according to the first identifier corresponding to the initial test task and the second identifier corresponding to the output table name.
In an exemplary embodiment, the data testing apparatus 1300 further comprises, prior to execution of the query module 1302:
And the second query sub-module is used for searching a target output table matched with the output table name in a preset output table database based on the output table name under the condition that the target copy table does not exist.
The first creation sub-module is used for creating a target replication table consistent with the structure of the target output table; the copy table name of the target copy table is different from the output table name of the target output table.
And the first replacing sub-module is used for replacing the output table name with the copy table name of the target copy table in the database statement based on a preset replacing frame to obtain the target database statement.
In an exemplary embodiment, the query module 1303 includes:
and the third query sub-module is used for querying a target output table corresponding to the output table name in a preset output table database under the condition that the target copy table exists.
And the judging sub-module is used for judging whether the structure of the target output table is the same as the structure of the target replication table.
And the fourth determination submodule is used for determining that the target replication table meets the preset data structure condition under the condition that the structure of the target output table is the same as that of the target replication table.
And the second replacing sub-module is used for replacing the output table name in the database statement with the copy table name of the target copy table to obtain the target database statement.
In an exemplary embodiment, after the determination sub-module executes, the data testing apparatus 1300 further includes:
and the second creating sub-module is used for executing the step of creating the target replication table consistent with the structure of the target output table under the condition that the structure of the target output table is different from that of the target replication table.
The respective modules in the above-described data testing apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a terminal, and the internal structure thereof may be as shown in fig. 14. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a data testing method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those skilled in the art that the structure shown in fig. 14 is merely a block diagram of a portion of the structure associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements are applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided comprising a memory and a processor, the memory having stored therein a computer program, the processor when executing the computer program performing the steps of:
acquiring an initial test task and analyzing each database statement in the initial test task to obtain an output table name corresponding to each database statement;
inquiring a target replication table according to a first identifier corresponding to the initial test task and a second identifier corresponding to the output table name aiming at each output table name in each database statement;
under the condition that a target replication table exists, replacing the corresponding output table name in the database statement according to the replication table name of the target replication table meeting the preset data structure condition to obtain a target database statement;
and constructing a target test task based on the multiple target database sentences.
In one embodiment, the processor when executing the computer program further performs the steps of:
responding to the submitting operation of the initial test task, and acquiring the initial test task;
analyzing each database statement contained in the initial test task to obtain an output table name contained in each database statement;
and determining a first identifier corresponding to the initial test task.
In one embodiment, the processor when executing the computer program further performs the steps of:
assigning values to all system variables in the database sentences in the initial test task according to a preset replacement rule aiming at each database sentence to obtain a first test task;
and inquiring the target replication table according to the first identifier corresponding to the first test task and the second identifier corresponding to the output table name in each database statement.
In one embodiment, the processor when executing the computer program further performs the steps of:
determining a second identifier corresponding to the output table name in a preset output table database aiming at each output table name in each database statement;
and determining whether a target replication table exists in a preset replication table database according to the first identifier corresponding to the initial test task and the second identifier corresponding to the output table name.
In one embodiment, the processor when executing the computer program further performs the steps of:
searching a target output table matched with the output table name in a preset output table database based on the output table name under the condition that the target copying table does not exist;
creating a target replication table consistent with the target output table structure; the copy table name of the target copy table is different from the output table name of the target output table;
and replacing the output table name with the copy table name of the target copy table in the database statement based on a preset replacement frame to obtain the target database statement.
In one embodiment, the processor when executing the computer program further performs the steps of:
under the condition that a target copying table exists, inquiring a target output table corresponding to an output table name in a preset output table database;
judging whether the structure of the target output table is the same as that of the target replication table;
under the condition that the structure of the target output table is the same as that of the target copying table, confirming that the target copying table meets the preset data structure condition;
and replacing the output table name in the database statement with the copy table name of the target copy table to obtain the target database statement.
In one embodiment, the processor when executing the computer program further performs the steps of:
In the case where the structure of the target output table and the structure of the target copy table are not identical, the step of creating the target copy table in conformity with the structure of the target output table is performed.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, carries out the steps of the method embodiments described above.
In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.
The user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or sufficiently authorized by each party.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, or the like, but is not limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.
Claims (10)
1. A method of testing data, the method comprising:
acquiring an initial test task and analyzing each database statement in the initial test task to obtain an output table name corresponding to each database statement;
querying a target replication table according to a first identifier corresponding to the initial test task and a second identifier corresponding to the output table name aiming at each output table name in each database statement;
Under the condition that the target copying table exists, replacing the corresponding output table name in the database statement according to the copying table name of the target copying table meeting the preset data structure condition to obtain a target database statement;
and constructing a target test task based on a plurality of target database sentences.
2. The method according to claim 1, wherein the obtaining an initial test task and analyzing each database statement in the initial test task to obtain an output table name corresponding to each database statement includes:
responding to the submitting operation of an initial test task, and acquiring the initial test task;
analyzing each database statement contained in the initial test task to obtain an output table name contained in each database statement;
and determining a first identifier corresponding to the initial test task.
3. The method of claim 1, wherein each of the database statements contains at least one system variable, the method further comprising:
assigning values to the system variables in the database sentences in the initial test task according to a preset replacement rule to obtain a first test task;
The querying, for each output table name in each database statement, a target replication table according to a first identifier corresponding to the initial test task and a second identifier corresponding to the output table name, including:
and inquiring a target replication table according to the first identifier corresponding to the first test task and the second identifier corresponding to the output table name in each database statement.
4. The method of claim 1, wherein for each of the output table names in each of the database statements, querying a target replication table based on a first identifier corresponding to the initial test task and a second identifier corresponding to the output table name, comprises:
determining a second identifier corresponding to the output table name in a preset output table database aiming at each output table name in each database statement;
and determining whether a target replication table exists in a preset replication table database according to the first identifier corresponding to the initial test task and the second identifier corresponding to the output table name.
5. The method of claim 1, wherein for each of the output table names in each of the database statements, after querying a target replication table according to a first identifier corresponding to the initial test task and a second identifier corresponding to the output table name, the method further comprises:
Searching a target output table matched with the output table name in a preset output table database based on the output table name under the condition that the target copy table does not exist;
creating a target replication table consistent with the target output table structure; the copy table name of the target copy table is different from the output table name of the target output table;
and replacing the output table name with the copy table name of the target copy table in the database statement based on a preset replacement frame to obtain a target database statement.
6. The method according to claim 1, wherein, in the case where the target replication table exists, replacing the output table name corresponding to the database statement according to the replication table name of the target replication table that satisfies a preset data structure condition, to obtain a target database statement, includes:
under the condition that the target copying table exists, inquiring a target output table corresponding to the output table name in a preset output table database;
judging whether the structure of the target output table is the same as the structure of the target replication table;
under the condition that the structure of the target output table is the same as that of the target replication table, confirming that the target replication table meets the preset data structure condition;
And replacing the output table name in the database statement with the copy table name of the target copy table to obtain the target database statement.
7. The method of claim 6, wherein after said determining whether the structure of the target output table and the structure of the target replication table are the same, the method further comprises:
in the case where the structure of the target output table and the structure of the target copy table are not identical, the step of creating a target copy table in conformity with the structure of the target output table is performed.
8. A data testing apparatus, the apparatus comprising:
the analysis module is used for acquiring an initial test task and analyzing each database statement in the initial test task to obtain an output table name corresponding to each database statement;
the query module is used for querying a target replication table according to a first identifier corresponding to the initial test task and a second identifier corresponding to the output table name aiming at each output table name in each database statement;
the replacing module is used for replacing the corresponding output table name in the database statement according to the copy table name of the target copy table meeting the preset data structure condition under the condition that the target copy table exists, so as to obtain a target database statement;
And the construction module is used for constructing a target test task based on a plurality of target database sentences.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 7 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310664610.0A CN116662191A (en) | 2023-06-06 | 2023-06-06 | Data testing method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310664610.0A CN116662191A (en) | 2023-06-06 | 2023-06-06 | Data testing method, device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116662191A true CN116662191A (en) | 2023-08-29 |
Family
ID=87727543
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310664610.0A Pending CN116662191A (en) | 2023-06-06 | 2023-06-06 | Data testing method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116662191A (en) |
-
2023
- 2023-06-06 CN CN202310664610.0A patent/CN116662191A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210374563A1 (en) | Solution Automation | |
| CN109977110B (en) | Data cleaning method, device and equipment | |
| US8108367B2 (en) | Constraints with hidden rows in a database | |
| US10824758B2 (en) | System and method for managing enterprise data | |
| CN112650766B (en) | Database data operation method, system and server | |
| US11080305B2 (en) | Relational log entry instituting system | |
| EP2369506B1 (en) | System and method of optimizing performance of schema matching | |
| CN111061475B (en) | Software code generating method, device, computer equipment and storage medium | |
| AU2017265144B2 (en) | Information retrieval | |
| CN113792077B (en) | Data processing method, program product, readable medium and electronic device | |
| CN104541297A (en) | Extensibility for sales predictor (SPE) | |
| CN113535766A (en) | Workflow configuration method, workflow configuration device, electronic device, and storage medium | |
| US20130124484A1 (en) | Persistent flow apparatus to transform metrics packages received from wireless devices into a data store suitable for mobile communication network analysis by visualization | |
| CN113139372A (en) | Information processing apparatus, storage medium, and information processing method | |
| US8468116B2 (en) | Rule creation method and rule creating apparatus | |
| CN116662191A (en) | Data testing method, device, computer equipment and storage medium | |
| US11688027B2 (en) | Generating actionable information from documents | |
| CN113506099A (en) | Configuration system, method, computer device and storage medium for reporting service | |
| US12099531B2 (en) | Information retrieval | |
| CN113468280B (en) | Data cognition method, device, equipment and storage medium | |
| CN117171193A (en) | Data generation method, device, computer equipment and storage medium | |
| CN116756022A (en) | Data preparation method, device, computer equipment and storage medium | |
| CN117033242A (en) | Test data generation method, device, computer equipment and storage medium | |
| CN117762937A (en) | SQL sentence optimization method, SQL sentence optimization device, SQL sentence optimization storage medium and SQL sentence optimization computer device | |
| CN116401035A (en) | Execution sequence determining method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |