CN115543836A

CN115543836A - Script quality detection method and related equipment

Info

Publication number: CN115543836A
Application number: CN202211305496.4A
Authority: CN
Inventors: 曾利强; 肖成英
Original assignee: Ping An Bank Co Ltd
Current assignee: Ping An Bank Co Ltd
Priority date: 2022-10-24
Filing date: 2022-10-24
Publication date: 2022-12-30

Abstract

The application discloses a script quality detection method and related equipment, which can acquire a script to be detected, analyze the script to be detected and obtain at least one temporary table corresponding to the script to be detected; analyzing the temporary tables aiming at each temporary table to obtain an analysis result corresponding to the temporary table, wherein the analysis result comprises fields in the temporary tables and a dependent source table; acquiring a script quality rule set; and according to the quality rule, performing quality detection on the fields in each temporary table and the dependent source table in the script to be detected, and generating a quality detection result of the script to be detected. This application can be through treating the detection script and analyzing to utilize the quality rule to come to carry out quality testing to the analytic result, can improve the quality testing efficiency and the accuracy of script like this, reduce manpower and materials cost, avoid script quality problems to flow into the condition that arouses the production accident in the production environment.

Description

Script quality detection method and related equipment

Technical Field

The application relates to the technical field of computers, in particular to a script quality detection method and related equipment.

Background

The habits, styles and levels of writing code for different developers vary from developer to developer. The quality of the code affects the quality of the software system, so that the quality of the code needs to be detected, and the problem of the quality of the code is prevented from flowing into a production environment to cause production accidents.

In the prior art, the quality problem of the modern code is usually solved through a traditional manual review mode, a big data script is complex in logic and can be found only by penetrating deep analysis layer by layer, and the quality problems of part scripts are various and difficult to be recognized in a standardized manner, so that an engineer with abundant data development experience consumes a long time to manually check and find the problems, the quality detection efficiency is low, a large amount of manpower is consumed, meanwhile, the quality detection mode is easy to omit the problem finding, and finally the problems flow into a production environment to cause production accidents.

Disclosure of Invention

The embodiment of the application provides a script quality detection method and related equipment, wherein the related equipment comprises a script quality detection device, electronic equipment, a computer readable storage medium and a computer program product, the quality detection efficiency and accuracy of the script can be improved, the cost of manpower and material resources is reduced, and the condition that the script quality problem flows into a production environment to cause production accidents is avoided.

The embodiment of the application provides a script quality detection method, which comprises the following steps:

acquiring a script to be detected, and analyzing the script to be detected to obtain at least one temporary table corresponding to the script to be detected;

analyzing the temporary tables aiming at each temporary table to obtain an analysis result corresponding to the temporary table, wherein the analysis result comprises fields in the temporary tables and a dependent source table;

acquiring a script quality rule set, wherein the script quality rule set comprises at least one quality rule;

and according to the quality rule, performing quality detection on the fields in each temporary table and the dependent source table in the script to be detected, and generating a quality detection result of the script to be detected.

Correspondingly, an embodiment of the present application provides a script quality detection apparatus, including:

the script acquisition unit is used for acquiring a script to be detected and analyzing the script to be detected to obtain at least one temporary table corresponding to the script to be detected;

the analysis unit is used for analyzing the temporary tables aiming at each temporary table to obtain analysis results corresponding to the temporary tables, wherein the analysis results comprise fields in the temporary tables and dependent source tables;

the rule obtaining unit is used for obtaining a script quality rule set, and the script quality rule set comprises at least one quality rule;

and the detection unit is used for performing quality detection on the fields in each temporary table and the dependent source table in the script to be detected according to the quality rules to generate a quality detection result of the script to be detected.

The electronic device provided by the embodiment of the application comprises a processor and a memory, wherein the memory stores a plurality of instructions, and the processor loads the instructions to execute the steps in the script quality detection method provided by the embodiment of the application.

The embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps in the script quality detection method provided in the embodiment of the present application.

In addition, a computer program product is provided in the embodiments of the present application, and includes a computer program or instructions, and the computer program or instructions, when executed by a processor, implement the steps in the script quality detection method provided in the embodiments of the present application.

The embodiment of the application provides a script quality detection method and related equipment, which can acquire a script to be detected, analyze the script to be detected and obtain at least one temporary table corresponding to the script to be detected; analyzing the temporary tables aiming at each temporary table to obtain an analysis result corresponding to the temporary table, wherein the analysis result comprises fields in the temporary tables and a dependent source table; acquiring a script quality rule set, wherein the script quality rule set comprises at least one quality rule; and according to the quality rule, performing quality detection on the fields in each temporary table and the dependent source table in the script to be detected, and generating a quality detection result of the script to be detected. This application can be through treating the detection script and analyzing to utilize the quality rule to come to carry out quality testing to the analytic result, can improve the quality testing efficiency and the accuracy of script like this, reduce manpower and materials cost, avoid script quality problems to flow into the condition that arouses the production accident in the production environment.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1a is a scene schematic diagram of a script quality detection method provided in an embodiment of the present application;

FIG. 1b is a flowchart of a script quality detection method provided in an embodiment of the present application;

FIG. 1c is another flowchart of a script quality detection method provided by an embodiment of the present application;

FIG. 1d is another flowchart of a script quality detection method provided by an embodiment of the present application;

FIG. 1e is a diagram illustrating a script quality detection method according to an embodiment of the present application;

FIG. 2 is another flowchart of a script quality detection method provided by an embodiment of the present application;

FIG. 3 is a schematic structural diagram of a script quality detection apparatus provided in an embodiment of the present application;

fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The embodiment of the application provides a script quality detection method and related equipment, and the related equipment can comprise a script quality detection device, electronic equipment, a computer readable storage medium and a computer program product. The script quality detection apparatus may be specifically integrated in an electronic device, and the electronic device may be a terminal or a server.

It is understood that the script quality detection method of the present embodiment may be executed on the terminal, may also be executed on the server, and may also be executed by both the terminal and the server. The above examples should not be construed as limiting the present application.

As shown in fig. 1a, the terminal and the server execute the script quality detection method together as an example. The script quality detection system provided by the embodiment of the application comprises a terminal 10, a server 11 and the like; the terminal 10 and the server 11 are connected via a network, for example, a wired or wireless network connection, and the like, wherein the script quality detecting device can be integrated in the server.

The server 11 may be configured to: acquiring a script to be detected, and analyzing the script to be detected to obtain at least one temporary table corresponding to the script to be detected; analyzing the temporary tables aiming at each temporary table to obtain an analysis result corresponding to the temporary table, wherein the analysis result comprises fields in the temporary tables and a dependent source table; acquiring a script quality rule set, wherein the script quality rule set comprises at least one quality rule; and according to the quality rule, performing quality detection on the fields in each temporary table and the dependent source table in the script to be detected, and generating a quality detection result of the script to be detected. The server 11 may be a single server, or may be a server cluster or a cloud server composed of a plurality of servers.

Wherein, the terminal 10 may be configured to: sending the script to be detected to the server 11, so that the server 11 performs quality detection on the script to be detected, and receiving a quality detection result of the script to be detected sent by the server 11. The terminal 10 may include a mobile phone, a smart television, a tablet Computer, a notebook Computer, a Personal Computer (PC), or the like. A client, which may be an application client or a browser client or the like, may also be provided on the terminal 10.

The following are detailed below. It should be noted that the following description of the embodiments is not intended to limit the preferred order of the embodiments.

The embodiment will be described from the perspective of a script quality detection apparatus, which may be specifically integrated in an electronic device, which may be a server or a terminal or the like.

As shown in fig. 1b, the specific flow of the script quality detection method may be as follows:

101. and acquiring a script to be detected, and analyzing the script to be detected to obtain at least one temporary table corresponding to the script to be detected.

The temporary table is a table established in a system temporary folder, and the database does not generally serialize the temporary table onto a disk, but exists in a memory of a database server and is destroyed after being used. Specifically, when the Data amount of the operated table is huge and other tables need to be correlated for query, or when the operated Data is temporary Data and there are many DML (Data management Language) operations in a short period, or when many tables need to be connected during query, if it may be inefficient to directly operate the service table of the database, then efficiency may be improved by using the temporary table.

Specifically, in this embodiment, the script to be detected may be a script file in the hive/spark sql.

hive is a data warehouse processing tool with Hadoop packaged at the bottom layer, is used for data extraction, transformation and loading, and is a mechanism capable of storing, inquiring and analyzing large-scale data stored in Hadoop. Hadoop is an open-source framework that allows large data to be stored and processed in a distributed environment across computer clusters using a simple programming model.

Spark SQL is a module that Spark uses to process structured data. Spark is a fast, general-purpose computing engine designed for large-scale data processing.

102. And analyzing the temporary tables aiming at each temporary table to obtain an analysis result corresponding to the temporary table, wherein the analysis result comprises fields in the temporary tables and a dependent source table.

Optionally, in this embodiment, the step of "analyzing, for each temporary table, the temporary table to obtain an analysis result corresponding to the temporary table" may include:

analyzing the temporary tables aiming at each temporary table to obtain an initial analysis result corresponding to the temporary table, wherein the initial analysis result comprises fields in the temporary tables and a dependent initial source table;

performing recursive analysis on the dependency relationship of the fields aiming at each field in the initial source table to obtain a source table on which the fields depend;

and acquiring the analysis result corresponding to the temporary table according to the initial analysis result and the source table on which each field in the initial source table depends.

The initial parsing result corresponding to the temporary table obtained through parsing may include a field in the temporary table and a dependent initial source table, where the initial source table may specifically be a dependent table of a target type statement in the temporary table, such as a where conditional dependent table, a join conditional dependent table, a group by and order by dependent table, and the like. Specifically, each segment of the temporary table is analyzed, and the obtained initial analysis result may include a temporary table name, a field name, field processing logic, a where condition dependency table and field, a join condition dependency table and field, a group by and order by logic and dependency table and field, and the like.

The fields and the dependent initial source table of the temporary table in the initial analysis result and the source table on which each field in the initial source table depends may be determined as the analysis result corresponding to the temporary table.

The method includes the steps of performing recursion analysis on the dependency relationship of fields in an initial source table, specifically, obtaining a final processing logic, a final logic dependency table and fields, a final where condition dependency table and fields, a final join condition dependency table and fields, a final group by and final order by logic and dependency tables and fields of each field in the initial source table according to a temporary dependency corresponding to each field in the initial source table and continuously performing recursion.

Where a where statement may be used to specify the criteria for selection. join statements are used to query data from two or more tables based on the relationships between columns in the tables. The group by statement is used to group the result set (selected data) according to the specified fields, and if the specified fields of some records have the same value, they will be merged into one piece of data. The order by statement is used to order the result set according to the specified columns.

103. A script quality rule set is obtained, the script quality rule set including at least one quality rule.

The quality rules in the script quality rule set may be preset by engineers in data development experience, and the types of the quality rules may be various, and may be specifically set according to actual situations, which is not limited in this embodiment.

104. And according to the quality rule, performing quality detection on the fields in each temporary table and the dependent source table in the script to be detected, and generating a quality detection result of the script to be detected.

Optionally, in this embodiment, the step of performing quality detection on the fields in each temporary table and the dependent source table in the script to be detected according to the quality rule to generate a quality detection result of the script to be detected may include:

detecting whether the fields in the temporary table contain target fields or not according to the quality rules aiming at each temporary table in the script to be detected;

when the field in the temporary table contains the target field, judging whether the type of the source table on which the target field depends is a temporary table type;

and when the type of the source table on which the target field depends is not the type of the temporary table, generating a quality problem corresponding to the temporary table to obtain a quality detection result of the script to be detected.

The target field may be specifically an x field. * The field is specifically a field indicating that an index has been established. The source table type on which the target field depends is not a temporary table type, i.e., the source table type on which the target field depends is a non-temporary table.

Specifically, the write method of the field and the dependent non-temporary table reads the whole field of the table, which increases the read consumption and affects the time efficiency and stability. The present embodiment can detect the quality problem of this writing method by detecting the quality rules of the target field and the source table type on which the target field depends; specifically, whether the field corresponding to each analyzed temporary table contains a field or not can be judged according to the quality rule; if the field is contained, judging whether the source table of the field is a non-temporary table, and if the source table of the field is the non-temporary table, generating the quality problem that the field in the temporary table depends on the non-temporary table.

for each temporary table in the script to be detected, determining a target source table which does not belong to the type of the temporary table from source tables which the temporary table depends on according to the quality rule;

judging whether a field corresponding to a target type statement depending on the target source table contains a partition field of the target source table;

and determining the quality problem corresponding to the temporary table based on the judgment result to obtain the quality detection result of the script to be detected.

The target type statement may be a where statement, and the like, which is not limited in this embodiment. The target source table is specifically a non-temporary table.

In particular, if the referenced non-temporary table does not use the partition field, the partition data pulled by the backward partition is more and more, and the data runs more and more slowly, which affects the timeliness and stability. According to the embodiment, according to the quality rule that the partition table is not provided with partition field filtering, the analysis result of each section of the temporary table is traversed, the source table of the non-temporary table is obtained, and whether the source table and the field of the where condition contain the partition field of the source table is judged aiming at the source table of the non-temporary table.

If the judgment result is that the field corresponding to the target type statement depending on the target source table does not contain the partition field of the target source table, the quality problem that the partition table in the temporary table does not have partition field filtering is generated.

Optionally, in this embodiment, the step "determining, based on the determination result, the quality problem corresponding to the temporary table to obtain the quality detection result of the script to be detected" may include:

when a field corresponding to a target type statement depending on the target source table contains a partition field of the target source table, judging whether statement logic corresponding to the target type statement contains a first logic symbol;

when the statement logic corresponding to the target type statement contains a first logic symbol, judging whether the statement logic corresponding to the target type statement contains a second logic symbol;

and when the statement logic corresponding to the target type statement does not contain a second logic symbol, generating a quality problem corresponding to the temporary table to obtain a quality detection result of the script to be detected.

The first logic symbol may be smaller, and the second logic symbol may be larger, which is not limited in this embodiment.

Specifically, if the referenced non-temporary table uses the partition field for filtering, but the filtering condition is only less than but not more than the write condition, the partition data pulled by the future partition is more and more, and the data is slower and slower, which affects the aging and stability. In this embodiment, the analysis result of each section of the temporary table may be traversed according to a quality rule for detecting that a partition watchband has a partition field for filtering, but only a smaller value but not a larger value, so as to obtain a source table of a non-temporary table; aiming at a source table of a non-temporary table, judging whether the source table and the field of the where condition contain a partition field of the source table, and if the source table contains the partition field of the source table, judging whether the logic of the where condition contains the partition field smaller than the partition field; if the logic of the where condition contains the partition field smaller than the threshold value, judging whether the logic of the where condition contains the partition field larger than the threshold value; if the logic of the where condition does not contain a partition field greater than, then the partition band that generated the segment of the temporary table is left with the partition field to filter, but only has a quality problem of less than without write greater than.

according to the quality rule, performing quality detection on the fields in the temporary table and the dependent source table to obtain the quality problem corresponding to the temporary table for each temporary table in the script to be detected;

combining fields in the script to be detected according to the quality rule to obtain a target character string; matching the target character string with a preset keyword;

and generating a quality detection result of the script to be detected according to the matching result and the quality problem corresponding to each temporary table.

The fields in the script to be detected are combined, specifically, the fields in the script to be detected are spliced, so that the target character string is obtained. The preset keywords may be units, select distict, count (distict, in (select, etc.), which is not limited in this embodiment.

Specifically, this embodiment may further detect a field of the full script, and detect whether there is a writing method using a unit instead of a unit all in the full script, or detect whether there are select distintint and count (select, instances, etc. of the use of the select), which may cause performance degradation of the script due to automatic sorting of the unit instead of the unit all, and thus affect the timeliness and stability of data.

The method can search a full script according to a quality rule for detecting the unions instead of the unions all, and combines fields in the script into a string of character strings to accurately match union keywords; the left brackets and spaces are then removed, matching the unionselect keyword. And if the matching result is that the use of the union exists in the script, generating the quality problem that the union, not the union all, exists in the script to be detected.

Specifically, both the union and union all keys function to merge two result sets into one, but both differ in terms of usage and efficiency. On one hand, processing of the repeated results: the unit will screen out duplicate records after table linking, and the unit all will not remove duplicate records. On the other hand, the processing of the ranking: the unions will be sorted in the order of the fields; union all simply merges the two results and returns them. A union all is significantly faster than a union in efficiency, so a union all is used if it can be confirmed that the two result sets of the merge do not contain duplicate data and no ordering is required.

The method comprises the steps of detecting a select distinting and a count (distinting and in (quality rules of select usage), retrieving a full script, merging the full script into a row of character strings, deleting a blank, deleting a join (select character string), matching whether the select distinting or the count (distinting or in (select keywords), and generating the select distinting and the count (distinting and in (select and exists) in the script to be detected if the matching result is that the keywords are included.

As shown in fig. 1c, in the development process of a software product, multiple stages such as requirement analysis, data sourcing, model design, code development, testing, commissioning and the like may be included, and after the code development is completed, a script quality problem generally needs to be detected, and then a software test is performed. The script quality detection method can provide an automatic script quality problem detection tool, script codes can be automatically analyzed and the script quality problem can be detected by using the automatic script quality problem detection tool, development is guided to be solved in advance, code quality problem identification and processing efficiency is improved, script quality is guaranteed, and rework cost in a script test stage is reduced.

Specifically, the script quality problem automatic detection tool based on script parsing provided by the application mainly comprises two modules: script parsing and script quality problem detection, refer to fig. 1d.

The script analysis steps are as follows:

1. reading a data warehouse script: reading a processing script corresponding to each operation table in the data warehouse system;

2. analyzing the script to obtain a temporary table: analyzing each script to obtain all temporary tables of each script;

3. analyzing the temporary table: analyzing each section of temporary table to obtain a temporary table name, a field name, field processing logic, a where condition dependent table and field, a join condition dependent table and field, group by and order by logic and dependent table and field;

4. identify all fields of the table: identifying a final write table and all fields of the write table, where the write table may specifically be the initial source table in the above embodiment;

5. obtaining the processing logic of each field of the table: according to the temporary dependence corresponding to each field of the write-in table, continuously recursing to obtain the final processing logic of each field of the write-in table, the final logic dependence table and field, the final where condition dependence table and field, the final join condition dependence table and field, and the final group by and final order by logic and dependence table and field;

6. and writing the analysis result into a database.

The script quality problem detection mainly comprises the following 6 types of problems, specifically:

1. detecting quality problems of select

The writing method can read all fields of the table, increase reading consumption and influence the time efficiency and stability. The quality problem detection step may include: firstly, judging whether the field corresponding to each analyzed temporary table contains a field; if yes, then judging whether the source table of the field is a non-temporary table.

2. Detecting quality problems without partition field filtering for partition tables

The writing method can cause that the data of the partitions pulled by the partitions in the future is more and more, the running is slower, and the timeliness and the stability are influenced. The quality problem detection step may include: and traversing the analysis result of each section of temporary table, acquiring the non-temporary table of the source table, and judging whether the source table and the field of the where condition contain the partition field of the source table.

3. Detecting quality problems of partitions with partition field filtering conditions but with less than and no more than writes

The writing method can cause that the data of the partitions pulled by the partitions in the future is more and more, the running is slower, and the timeliness and the stability are influenced. The quality problem detection procedure may be as follows:

step1, traversing each section of temporary table analysis result to obtain the non-temporary table of the source table

Step2, judging whether the source table and the field of the where condition contain the partition field of the source table or not

Step3, judging whether the logic of the where condition contains a partition field smaller than

Step4, judging whether the logic of the where condition contains the partition field larger than

4. Detecting quality problems with table associations using name class fields

The writing method has a high probability that data association is not achieved due to inconsistency of name data, and data quality is affected. The quality problem detection step may include:

step1, traversing each section of temporary table analysis result, and acquiring the corresponding field of the associated key

Step2, recursively acquiring fields of the final non-temporary table of fields corresponding to the associated keys

Step3, marking each field with a label whether the data contains Chinese or not on the metadata

Step4, judging whether the field of the final non-temporary table contains a Chinese data label or not

5. Detecting quality problems with units other than units all

The writing method can cause performance reduction of the script due to automatic sequencing of the unions, and further influences data timeliness and stability. The quality problem detection step may include: retrieving a full script, combining the full script into a string of character strings, and accurately matching a union keyword; the left brackets and spaces are then removed, matching the unionselect keyword.

6. Detecting select distinting, count (distinting, in (select, exists (quality problem of select usage)

The writing parallelism is reduced, and the data tilt is caused, thereby affecting the time efficiency and stability. The quality problem detection step may include: the full script is retrieved, merged into a string, the spaces are deleted, the join (select string) is deleted, then, whether the match contains a select distintint or count (distintint or in (select keywords) is matched.

After the detection is finished, the embodiment may also generate a problem list report, and support query and export, as shown in fig. 1 e. The problem list report can comprise a report display SQL quality problem list, a problem list search and problem detail export process support according to conditions such as library names and table names, problems are clear and transparent, and processing progress can be controlled.

Compare the artifical code review discernment script quality problems of current use, the quality problems in the automatic identification script of this application based on script parsing's automatic identification script quality problems instrument can be fast and relatively comprehensive automatic identification script, guide development personnel independently discover, solve the problem, promote discernment rate of accuracy and efficiency greatly, the problem is clear transparent, but the management and control of processing progress, a large amount of manpowers and time have been saved, promote data system's stability, ageing and data quality simultaneously.

As can be seen from the above, the present embodiment may obtain a script to be detected, and analyze the script to be detected to obtain at least one temporary table corresponding to the script to be detected; analyzing the temporary tables aiming at each temporary table to obtain an analysis result corresponding to the temporary table, wherein the analysis result comprises fields in the temporary tables and a dependent source table; acquiring a script quality rule set, wherein the script quality rule set comprises at least one quality rule; and according to the quality rule, performing quality detection on the fields in each temporary table and the dependent source table in the script to be detected, and generating a quality detection result of the script to be detected. This application can be through treating the detection script and analyzing to utilize the quality rule to come to carry out quality testing to the analytic result, can improve the quality testing efficiency and the accuracy of script like this, reduce manpower and materials cost, avoid script quality problems to flow into the condition that arouses the production accident in the production environment.

The following will describe the script quality detecting device in further detail by way of example as being specifically integrated in a server according to the method described in the previous embodiment.

An embodiment of the present application provides a script quality detection method, and as shown in fig. 2, a specific flow of the script quality detection method may be as follows:

201. the server obtains the script to be detected and analyzes the script to be detected to obtain at least one temporary table corresponding to the script to be detected.

Specifically, in this embodiment, the script to be detected may be a script file in hive/spark sql.

The temporary table is a table established in a system temporary folder, and the database does not generally serialize the temporary table onto a disk, but exists in a memory of a database server and is destroyed after being used.

202. And the server analyzes the temporary tables aiming at each temporary table to obtain an initial analysis result corresponding to the temporary table, wherein the initial analysis result comprises fields in the temporary tables and a dependent initial source table.

203. And the server carries out recursive analysis on the dependency relationship of the fields aiming at each field in the initial source table to obtain the source table on which the fields depend.

204. And the server acquires the analysis result corresponding to the temporary table according to the initial analysis result and the source table on which each field in the initial source table depends, wherein the analysis result comprises the field in the temporary table and the source table on which each field depends.

205. The server obtains a script quality rule set, which includes at least one quality rule.

206. And the server performs quality detection on the fields in each temporary table and the dependent source table in the script to be detected according to the quality rule to generate a quality detection result of the script to be detected.

Specifically, the write method of the field and the dependent non-temporary table reads the whole field of the table, which increases the read consumption and affects the time efficiency and stability. The present embodiment can detect the quality problem of this writing method by detecting the quality rules of the target field and the source table type on which the target field depends; specifically, whether the field corresponding to each segment of the analyzed temporary table contains a field may be judged according to the quality rule; if the field is contained, judging whether the source table of the field is a non-temporary table, and if the source table of the field is the non-temporary table, generating the quality problem that the field in the temporary table depends on the non-temporary table.

If the judgment result is that the field corresponding to the target type statement depending on the target source table does not contain the partition field of the target source table, generating the quality problem that the partition table in the temporary table does not carry the partition field filtering.

Specifically, if the referenced non-temporary table uses partition fields for filtering, but the filtering condition is only less than and not more than, the partition data pulled by the future partition is more and more, and the data runs more and more slowly, which affects the timeliness and stability. In this embodiment, the analysis result of each section of the temporary table may be traversed according to a quality rule for detecting that a partition watchband has a partition field for filtering, but only a smaller value but not a larger value, so as to obtain a source table of a non-temporary table; aiming at a source table of a non-temporary table, judging whether the source table and the field of the where condition contain a partition field of the source table, and if the source table contains the partition field of the source table, judging whether the logic of the where condition contains the partition field smaller than the partition field; if the logic of the where condition contains the partition field smaller than the partition field, judging whether the logic of the where condition contains the partition field larger than the partition field; if the logic of the where condition does not contain a partition field greater than, then the partition band that generated the segment of the temporary table is left with the partition field to filter, but only has a quality problem of less than without write greater than.

As can be seen from the above, in this embodiment, the script to be detected may be obtained through the server, and the script to be detected is analyzed, so as to obtain at least one temporary table corresponding to the script to be detected; analyzing the temporary tables aiming at each temporary table to obtain an initial analysis result corresponding to the temporary table, wherein the initial analysis result comprises fields in the temporary tables and a dependent initial source table; performing recursive analysis on the dependency relationship of the fields aiming at each field in the initial source table to obtain a source table on which the fields depend; acquiring an analysis result corresponding to the temporary table according to the initial analysis result and a source table on which each field in the initial source table depends, wherein the analysis result comprises the fields in the temporary table and the source table on which the field depends; acquiring a script quality rule set, wherein the script quality rule set comprises at least one quality rule; and according to the quality rule, performing quality detection on the fields in each temporary table and the dependent source table in the script to be detected, and generating a quality detection result of the script to be detected. This application can be through treating the detection script and analyzing to utilize the quality rule to come to carry out quality testing to the analytic result, can improve the quality testing efficiency and the accuracy of script like this, reduce manpower and materials cost, avoid script quality problems to flow into the condition that arouses the production accident in the production environment.

In order to better implement the above method, an embodiment of the present application further provides a script quality detection apparatus, which may include a script obtaining unit 301, a parsing unit 302, a rule obtaining unit 303, and a detection unit 304, as shown in fig. 3, as follows:

(1) A script acquisition unit 301;

and the script acquisition unit is used for acquiring the script to be detected and analyzing the script to be detected to obtain at least one temporary table corresponding to the script to be detected.

(2) An analysis unit 302;

and the analysis unit is used for analyzing the temporary tables to obtain analysis results corresponding to the temporary tables, wherein the analysis results comprise fields in the temporary tables and dependent source tables.

Optionally, in some embodiments of the present application, the parsing unit may include a parsing subunit, a recursive parsing subunit, and an obtaining subunit, as follows:

the analysis subunit is configured to analyze each temporary table to obtain an initial analysis result corresponding to the temporary table, where the initial analysis result includes a field in the temporary table and a dependent initial source table;

a recursive analysis subunit, configured to perform recursive analysis on the dependency relationship of the field for each field in the initial source table, so as to obtain a source table on which the field depends;

and the obtaining subunit is configured to obtain, according to the initial analysis result and the source table on which each field in the initial source table depends, an analysis result corresponding to the temporary table.

(3) A rule acquisition unit 303;

the rule obtaining unit is used for obtaining a script quality rule set, and the script quality rule set comprises at least one quality rule.

(4) A detection unit 304;

and the detection unit is used for carrying out quality detection on the fields in each temporary table and the dependent source table in the script to be detected according to the quality rule and generating a quality detection result of the script to be detected.

Optionally, in some embodiments of the present application, the detecting unit may include a first detecting subunit, a first judging subunit, and a first generating subunit, as follows:

the first detecting subunit is configured to detect, for each temporary table in the script to be detected, whether a field in the temporary table includes a target field according to the quality rule;

a first determining subunit, configured to determine, when a field in the temporary table includes the target field, whether a type of a source table on which the target field depends is a temporary table type;

and the first generating subunit is configured to generate a quality problem corresponding to the temporary table when the type of the source table on which the target field depends is not the type of the temporary table, and obtain a quality detection result of the script to be detected.

Optionally, in some embodiments of the present application, the detecting unit may include a first determining subunit, a second determining subunit, and a second determining subunit, as follows:

the first determining subunit is configured to determine, for each temporary table in the script to be detected, a target source table that does not belong to the type of the temporary table from the source tables that the temporary table depends on according to the quality rule;

a second judging subunit, configured to judge whether a field corresponding to the target type statement that depends on the target source table includes a partition field of the target source table;

and the second determining subunit is configured to determine, based on the determination result, the quality problem corresponding to the temporary table, and obtain a quality detection result of the script to be detected.

Optionally, in some embodiments of the application, the second determining subunit may be specifically configured to, when a field corresponding to a target type statement that depends on the target source table includes a partition field of the target source table, determine whether statement logic corresponding to the target type statement includes a first logic symbol; when the statement logic corresponding to the target type statement contains a first logic symbol, judging whether the statement logic corresponding to the target type statement contains a second logic symbol; and when the statement logic corresponding to the target type statement does not contain a second logic symbol, generating a quality problem corresponding to the temporary table to obtain a quality detection result of the script to be detected.

Optionally, in some embodiments of the present application, the detecting unit may include a second detecting subunit, a combining subunit, and a second generating subunit, as follows:

the second detection subunit is configured to, for each temporary table in the script to be detected, perform quality detection on a field in the temporary table and a dependent source table according to the quality rule, to obtain a quality problem corresponding to the temporary table;

the merging subunit is used for merging the fields in the script to be detected according to the quality rule to obtain a target character string; matching the target character string with preset keywords;

and the second generation subunit is used for generating a quality detection result of the script to be detected according to the matching result and the quality problems corresponding to the temporary tables.

As can be seen from the above, in this embodiment, the script obtaining unit 301 may obtain a script to be detected, and analyze the script to be detected to obtain at least one temporary table corresponding to the script to be detected; for each temporary table, analyzing the temporary table through an analyzing unit 302 to obtain an analysis result corresponding to the temporary table, where the analysis result includes fields in the temporary table and a dependent source table; acquiring, by a rule acquisition unit 303, a script quality rule set, the script quality rule set including at least one quality rule; and performing quality detection on the fields in each temporary table and the dependent source table in the script to be detected through the detection unit 304 according to the quality rule, so as to generate a quality detection result of the script to be detected. This application can be through treating the detection script and analyzing to utilize the quality rule to come to carry out quality testing to the analytic result, can improve the quality testing efficiency and the accuracy of script like this, reduce manpower and materials cost, avoid script quality problems to flow into the condition that arouses the production accident in the production environment.

An electronic device according to an embodiment of the present application is further provided, as shown in fig. 4, which shows a schematic structural diagram of the electronic device according to the embodiment of the present application, where the electronic device may be a terminal or a server, and specifically:

the electronic device may include components such as a processor 401 of one or more processing cores, memory 402 of one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that the electronic device configuration shown in fig. 4 does not constitute a limitation of the electronic device and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components. Wherein:

the processor 401 is a control center of the electronic device, connects various parts of the entire electronic device using various interfaces and lines, and performs various functions of the electronic device and processes data by operating or executing software programs and/or modules stored in the memory 402 and calling data stored in the memory 402. Optionally, processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 401.

The memory 402 may be used to store software programs and modules, and the processor 401 may execute various functional applications and script quality detection by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to use of the electronic device, and the like. Further, the memory 402 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 access to the memory 402.

The electronic device further comprises a power supply 403 for supplying power to the various components, and preferably, the power supply 403 is logically connected to the processor 401 through a power management system, so that functions of managing charging, discharging, and power consumption are realized through the power management system. The power supply 403 may also include any component of one or more dc or ac power sources, recharging systems, power failure detection circuitry, power converters or inverters, power status indicators, and the like.

The electronic device may further include an input unit 404, and the input unit 404 may be used to receive input numeric or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

Although not shown, the electronic device may further include a display unit and the like, which are not described in detail herein. Specifically, in this embodiment, the processor 401 in the electronic device loads the executable file corresponding to the process of one or more application programs into the memory 402 according to the following instructions, and the processor 401 runs the application programs stored in the memory 402, thereby implementing various functions as follows:

acquiring a script to be detected, and analyzing the script to be detected to obtain at least one temporary table corresponding to the script to be detected; analyzing the temporary tables aiming at each temporary table to obtain an analysis result corresponding to the temporary table, wherein the analysis result comprises fields in the temporary tables and a dependent source table; acquiring a script quality rule set, wherein the script quality rule set comprises at least one quality rule; and according to the quality rule, performing quality detection on the fields in each temporary table and the dependent source table in the script to be detected, and generating a quality detection result of the script to be detected.

The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.

As can be seen from the above, the present embodiment may obtain a script to be detected, and analyze the script to be detected to obtain at least one temporary table corresponding to the script to be detected; analyzing the temporary tables aiming at each temporary table to obtain an analysis result corresponding to the temporary table, wherein the analysis result comprises fields in the temporary tables and a dependent source table; acquiring a script quality rule set, wherein the script quality rule set comprises at least one quality rule; and according to the quality rule, performing quality detection on the fields in each temporary table and the dependent source table in the script to be detected, and generating a quality detection result of the script to be detected. According to the script quality detection method and device, the script to be detected can be analyzed, quality detection is carried out on the analysis result through the quality rules, so that the quality detection efficiency and accuracy of the script can be improved, the cost of manpower and material resources is reduced, and the condition that the quality problem of the script flows into a production environment to cause production accidents is avoided.

It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor.

To this end, embodiments of the present application provide a computer-readable storage medium, in which a plurality of instructions are stored, where the instructions can be loaded by a processor to execute the steps in any one of the script quality detection methods provided in the embodiments of the present application. For example, the instructions may perform the steps of:

Wherein the computer-readable storage medium may include: read Only Memory (ROM), random Access Memory (RAM), magnetic or optical disks, and the like.

Since the instructions stored in the computer-readable storage medium can execute the steps in any script quality detection method provided in the embodiments of the present application, beneficial effects that can be achieved by any script quality detection method provided in the embodiments of the present application can be achieved, which are detailed in the foregoing embodiments and will not be described herein again.

According to an aspect of the application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the methods provided in the various alternative implementations of the script quality detection aspect described above.

The script quality detection method and the related devices provided by the embodiments of the present application are introduced in detail above, and a specific example is applied in the present application to explain the principle and the implementation of the present application, and the description of the above embodiments is only used to help understand the method and the core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. A script quality detection method is characterized by comprising the following steps:

2. The method according to claim 1, wherein the parsing the temporary table for each temporary table to obtain the parsing result corresponding to the temporary table comprises:

3. The method according to claim 1, wherein the performing quality inspection on the fields in each temporary table and the dependent source table in the script to be inspected according to the quality rules to generate the quality inspection result of the script to be inspected comprises:

when the fields in the temporary table contain the target fields, judging whether the types of the source tables depended on by the target fields are temporary table types or not;

4. The method according to claim 1, wherein the performing quality inspection on the fields in each temporary table and the dependent source table in the script to be inspected according to the quality rules to generate the quality inspection result of the script to be inspected comprises:

5. The method according to claim 4, wherein the determining the quality problem corresponding to the temporary table based on the determination result to obtain the quality detection result of the script to be detected comprises:

6. The method according to claim 1, wherein the performing quality inspection on the fields in each temporary table and the dependent source table in the script to be inspected according to the quality rules to generate the quality inspection result of the script to be inspected comprises:

combining fields in the script to be detected according to the quality rule to obtain a target character string; matching the target character string with preset keywords;

7. A script quality detecting apparatus, comprising:

8. An electronic device comprising a memory and a processor; the memory stores an application program, and the processor is configured to execute the application program in the memory to perform the operations of the script quality detecting method of any one of claims 1 to 6.

9. A computer readable storage medium storing instructions adapted to be loaded by a processor to perform the steps of the script quality detection method of any one of claims 1 to 6.

10. A computer program product comprising a computer program or instructions, characterized in that the computer program or instructions, when executed by a processor, implement the steps in the script quality detection method of any one of claims 1 to 6.