CN114385656A

CN114385656A - Script detection method and device, computer equipment and storage medium

Info

Publication number: CN114385656A
Application number: CN202210038271.0A
Authority: CN
Inventors: 张羽
Original assignee: Ping An Puhui Enterprise Management Co Ltd
Current assignee: Ping An Puhui Enterprise Management Co Ltd
Priority date: 2022-01-13
Filing date: 2022-01-13
Publication date: 2022-04-22

Abstract

The invention relates to the field of data processing and artificial intelligence, and discloses a script detection method and device, computer equipment and a storage medium. The script detection control is configured according to script data to be executed, the script detection control is executed to judge whether a table corresponding to the script data is a partition table or not, whether a field in the partition table is updated or not is analyzed, if yes, whether a partition connection field exists or not is detected, and if not, a detection log is generated, and abnormity is prompted. By the method, the abnormity of the partition table is detected, and the problem that historical partitions cannot be synchronized by executing scripts after fields are newly added in the partition table is solved.

Description

Script detection method and device, computer equipment and storage medium

Technical Field

The present invention relates to the field of data processing, and in particular, to a script detection method and apparatus, a computer device, and a storage medium.

Background

In order to improve the efficiency of database operations such as data query, a host data table in a database is usually provided with a plurality of partitions at present, and then batch operations are realized through a batch processing script.

Disclosure of Invention

The present application mainly aims to provide a script detection method, device, computer device and storage medium, so as to solve the problem that the newly added fields of the partition table in the prior art cannot be updated to the historical partitions.

The invention provides a script detection method based on hive sql in a first aspect, which comprises the following steps:

acquiring ddl script data to be executed in a hive database;

analyzing the ddl script data to obtain table information corresponding to a script, and configuring a script detection control based on the table information, wherein the script detection control is a function tool for detecting keyword abnormality of the ddl script data;

executing the script detection control, and extracting script sections of newly added fields in ddl script data;

judging whether a partition connection field exists in the script segment or not;

if not, extracting the address information of the script segment, generating an abnormal log based on the address information, and prompting abnormality.

Optionally, in a first implementation manner of the first aspect of the present invention, the acquiring ddl script data to be executed in the hive database includes:

determining a test library to be detected, wherein the test library comprises a plurality of hive databases;

constructing a scanning script based on the test library;

executing the scanning script, and traversing the execution states of the scripts of all the hive databases in the test library;

and selecting ddl script data of which the execution state is a to-be-executed state.

Optionally, in a second implementation manner of the first aspect of the present invention, the parsing the ddl script data to obtain table information corresponding to a script, and configuring a script detection control based on the table information includes:

calling a preset script analysis tool, and extracting a table creation command and a script directory address in ddl script data;

and configuring test contents in the blank script detection control based on the table creation command and the script directory address to obtain the script detection control corresponding to the ddl script data.

Optionally, in a third implementation manner of the first aspect of the present invention, the configuring test contents in a blank script detection control based on the table creation command and the script directory address to obtain a script detection control corresponding to the ddl script data includes:

identifying an address field and a start control field in a blank script detection control;

and adding the script directory address into the address field by using a mian method, and adding the table creation command into the starting control field to obtain a script detection control corresponding to the ddl script data.

Optionally, in a fourth implementation manner of the first aspect of the present invention, the extracting, by the script detection control, a script segment of a newly added field in the ddl script data includes:

after receiving a script detection instruction, executing the script detection control, and traversing each script code in ddl script data;

detecting whether each script code has a newly added field;

and if so, intercepting the script segment where the newly added field is located.

Optionally, in a fifth implementation manner of the first aspect of the present invention, the detecting whether there is a newly added field in each script code includes:

identifying a structured query statement in the script code to obtain an SQL set;

splitting each structured query statement in the SQL set by a getCode method, and removing annotations to obtain an SQL list;

determining whether a script of an alter added field exists in the script code or not in a regular analysis mode according to the SQL list;

if yes, determining that a newly added field exists in the script code;

and if not, determining that the newly added field does not exist in the script code.

Optionally, in a sixth implementation manner of the first aspect of the present invention, the table information further includes a table name; the extracting the address information of the script segment and generating an abnormal log based on the address information includes:

according to the table name, querying a historical table building statement corresponding to the table name from a script directory address of the ddl script data;

judging whether a table corresponding to ddl script data is a partition table or not based on the historical table building statement;

if so, extracting address information of the script segment, and generating an abnormal log according to a preset abnormal log format, wherein the abnormal log format is address information + an abnormal field.

The second aspect of the present invention provides a script detection apparatus based on hive sql, wherein the script detection apparatus comprises:

the acquisition module is used for acquiring ddl script data to be executed in the hive database;

the configuration module is used for analyzing the ddl script data to obtain table information corresponding to a script, and configuring a script detection control based on the table information, wherein the script detection control is a function tool used for detecting keyword abnormality of the ddl script data;

the extracting module is used for executing the script detection control and extracting script sections of newly added fields in ddl script data;

the judging module is used for judging whether the script section has a partition connection field or not;

and the log generation module is used for extracting the address information of the script section when judging that the partition connection field does not exist, generating an abnormal log based on the address information and prompting abnormality.

Optionally, in a first implementation manner of the second aspect of the present invention, the obtaining module includes:

the device comprises a determining unit, a judging unit and a judging unit, wherein the determining unit is used for determining a test library to be detected, and the test library comprises a plurality of hive databases;

the construction unit is used for constructing a scanning script based on the test library;

the traversal unit is used for executing the scanning script and traversing the execution states of all scripts of the hive database in the test library;

and the selection unit is used for selecting ddl script data of which the execution state is a to-be-executed state.

Optionally, in a second implementation manner of the second aspect of the present invention, the configuration module includes:

the extracting unit is used for calling a preset script analyzing tool and extracting a table creating command and a script directory address in ddl script data;

and the configuration unit is used for configuring the test content in the blank script detection control based on the table creation command and the script directory address to obtain the script detection control corresponding to the ddl script data.

Optionally, in a third implementation manner of the second aspect of the present invention, the configuration unit is specifically configured to:

Optionally, in a fourth implementation manner of the second aspect of the present invention, the extraction module includes:

the execution unit is used for executing the script detection control and traversing each script code in the ddl script data after receiving a script detection instruction;

the detection unit is used for detecting whether newly added fields exist in the script codes or not;

and the intercepting unit is used for intercepting the script section where the newly added field is located when the newly added field is detected.

Optionally, in a fifth implementation manner of the second aspect of the present invention, the detecting unit is specifically configured to:

if yes, determining that a newly added field exists in the script code;

Optionally, in a sixth implementation manner of the second aspect of the present invention, the table information further includes a table name; the log generation module comprises:

the query unit is used for querying a historical table building statement corresponding to the table name from the script directory address of the ddl script data according to the table name;

the judging unit is used for judging whether the table corresponding to the ddl script data is a partition table or not based on the historical table building statement;

and the generating unit is used for extracting the address information of the script section when the partition table is judged, and generating an abnormal log according to a preset abnormal log format, wherein the abnormal log format is address information plus an abnormal field.

A third aspect of the present invention provides a computer apparatus comprising: a memory and at least one processor, the memory having instructions stored therein; the at least one processor invokes the instructions in the memory to cause the computer device to perform the steps of the hive sql-based script detection method described above.

A fourth aspect of the present invention provides a computer-readable storage medium having stored therein instructions, which, when run on a computer, cause the computer to perform the steps of the above-described hive sql-based script detection method.

In the technical scheme of the invention, the method specifically comprises the steps of detecting various tables in a database through a configuration detection control, judging whether the various tables have partition tables or not, analyzing whether fields in the partition tables are updated or not, detecting whether partition connection fields exist or not if the partition tables have the partition tables, generating a detection log if the partition connection fields do not exist, and prompting abnormity. By the method, the abnormity of the partition table is detected, and the problem that historical partitions cannot be synchronized by executing scripts after fields are newly added in the partition table is solved.

Drawings

FIG. 1 is a schematic diagram of a first embodiment of a script detection method based on hive sql according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a second embodiment of the script detection method based on hive sql according to the embodiment of the present invention;

FIG. 3 is a schematic diagram of a third embodiment of a script detection method based on hive sql according to the embodiment of the present invention;

FIG. 4 is a schematic diagram of an embodiment of a script detection apparatus based on hive sql according to the embodiment of the present invention;

FIG. 5 is a schematic diagram of another embodiment of a script detection apparatus based on hive sql according to the embodiment of the present invention;

FIG. 6 is a diagram of an embodiment of a computer device in an embodiment of the invention.

Detailed Description

The script detection method based on hive sql is characterized in that all scripts are traversed through ddl script directories filled by users, and if the scripts are scripts of newly-added fields, whether keywords cascade are added or not and whether the scripts lack the keywords or not are checked to determine whether a list is changed into a partition list or not. The table name is extracted regularly, the ddl directory is traversed by the table name, the table building script of the latest version of the table is found out, whether the table is partitioned or not is judged, and the result is fed back, so that synchronous abnormity detection between the building script code of the partition table and the historical partition is realized, the execution success rate of the script code is improved, the abnormity appearance is reduced, and the problem that the abnormity cannot be detected due to the fact that the new partition cannot be detected particularly when the new partition is executed is solved.

The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," or "having," and any variations thereof, are intended to cover non-exclusive inclusions, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

For convenience of understanding, a specific flow of the embodiment of the present invention is described below, and referring to fig. 1, a first embodiment of the script detection method based on hive sql in the embodiment of the present invention includes the following implementation steps:

101. acquiring ddl script data to be executed in a hive database;

in this step, the acquisition of ddl script data can be realized by scanning a script, specifically:

constructing a scanning script based on the test library;

In practical application, the scan script is an automatic scan tool with an execution state field as a scan reference, and can be embodied in a program mode or a control mode, preferably embodied in a control mode, a scan function is added according to a script storage address and a script frame of a test library as a design basis, the script storage address is traversed through the scan function, configuration information in the script, specifically an execution state field in the configuration information, is traversed, whether the execution state field is marking information of a state to be executed is judged, if yes, the execution state field is marked selectively, and after all traversal is finished, the script is extracted based on the selected mark, so that ddl script data is obtained.

102. Analyzing ddl script data to obtain table information corresponding to the script, and configuring a script detection control based on the table information;

in this step, the script detection control is a function tool for detecting the abnormality of the ddl script data keyword. The parsing of the ddl script data here specifically extracts information such as type information of a table corresponding to the ddl script data, a table creation command corresponding to the script, and a storage location of the script, and even further includes creation history data of the table corresponding to the ddl script data.

In this embodiment, when configuring the script detection control according to the table information, specifically, the scanning script is configured to form the script detection control, specifically, the test content in the scanning script is modified to identify the table creation command and the field information, and after recompiling the modified scanning script, the visualization processing algorithm is called to render the modified scanning script into the control form, so as to obtain the script detection control.

103. Executing the script detection control, and extracting script sections of newly added fields in ddl script data;

in the step, whether the field in the ddl script data is updated or increased is detected through the field information in the script detection control, and if yes, the whole script section in the newly added field is intercepted by utilizing the intercepting field in the script detection control.

In practical application, the script detection control comprises a script analysis model, a partition function identification model and a cutting model, each script code in ddl script data is analyzed through the script analysis model, the analysis value is based on a function, a function body in each script code is identified, the code polarity is separated based on the function body to obtain a plurality of code segments, then the function body in each code segment is identified through the partition function identification model, the code segments belonging to the partition function are extracted, highlighting is carried out in the script codes, the starting part and the ending part of the highlighting are identified through the cutting model based on the highlighting, a cutting tool is executed, the cutting tool is extracted from the script codes, and the corresponding script segment is obtained.

104. Judging whether a partition connection field exists in the script segment or not;

in the step, field definition bytes in the script segments are identified, the definition of the partition connection fields is screened out based on the identification result, if the definition bytes of the fields exist in the identification result, the partition connection fields exist in the script segments, otherwise, the partition connection fields do not exist.

105. If the address information does not exist, the address information where the script section is located is extracted, an abnormal log is generated based on the address information, and abnormality is prompted.

Extracting a name (namely a table name) of a table corresponding to the ddl script data, and inquiring a historical table building statement corresponding to the table name from a script directory address of the ddl script data based on the table name;

specifically, a query function is called to traverse data by using a script directory address, wherein during the traversal, after a corresponding table is queried through a table name, a history table statement of the table is acquired based on actual information of the table, such as execution logic and a log recording position, and based on the actual information.

and identifying a field used for determining the type of the language table in the history table sentence, and determining whether the table corresponding to the ddl script data is a partition table or not based on the actual copying of the field, namely, the field can also be understood as a creating command for identifying whether the history table creating command belongs to the partition table or not, if so, determining that the table corresponding to the ddl script data is the partition table, otherwise, determining that the table is the system table.

In this embodiment, if the system table is determined, the method further includes detecting whether there is abnormal data in the system table; the abnormal data at least comprises part or all of the partition tables in the system table with the partition table number larger than or equal to a preset value, a system table carrying a _ test key field, a system table carrying a _ init key field or a system table occupying storage resources and ranked before a preset rank; if the abnormal data exist in the system table, generating a cleaning script; and running the cleaning script to clean the abnormal data.

By implementing the method, the script detection control is configured according to the script data to be executed, the script detection control is executed to judge whether the table corresponding to the script data is a partition table or not, whether a field in the partition table is updated or not is analyzed, if yes, whether a partition connection field exists or not is detected, and if not, a detection log is generated, and abnormity is prompted. By the method, the abnormity of the partition table is detected, and the problem that historical partitions cannot be synchronized by executing scripts after fields are newly added in the partition table is solved.

Referring to fig. 2, a second embodiment of the script detection method based on hive sql according to the embodiment of the present invention includes:

201. acquiring ddl script data to be executed in a hive database;

202. calling a preset script analysis tool, and extracting a table creation command and a script directory address in ddl script data;

the script parsing tool is also a script parsing file in practice, source structures in script data are parsed by executing the script parsing file to obtain a mark creation command, and then a script directory address is determined based on execution logic for executing the script data.

203. Configuring test contents in a blank script detection control based on the table creation command and the script directory address to obtain a script detection control corresponding to ddl script data;

the specific implementation of the step is as follows: identifying an address field and a start control field in a blank script detection control;

In practical application, the script detection control can be configured by utilizing an open source or a Java framework, specifically, a function body of the detection control is arranged in the open source or the Java framework, and in an application process, a test content part and a traversal address in the function body are configured based on a table creation command and are set as a script directory address, so that the script detection control is generated.

Specifically, the table creation command includes a set of execution logic and specific instructions, and when test content is configured, the execution logic and the specific instructions need to be separated and extracted, then the execution logic is converted into a logic language in a function body in a detection control to obtain a logic relationship, the specific instructions are added into the logic relationship to form the test content, the test content is filled into the detection control, and meanwhile, a script directory address is also configured to control the traversal position of the control.

204. Executing the script detection control, and extracting script sections of newly added fields in ddl script data;

205. judging whether a partition connection field exists in the script segment or not;

206. if the address information does not exist, the address information where the script section is located is extracted, an abnormal log is generated based on the address information, and abnormality is prompted.

In this embodiment, the table information further includes a table name; the extracting the address information of the script segment and generating an abnormal log based on the address information includes:

In practical application, the partition connection field is a cascade keyword, after the existence of the alter field is detected, whether the cascade keyword exists is further detected, if not, a detected log is output, and a user is prompted to process the log.

In summary, all scripts are traversed through the ddl script directory filled by the user, and if the scripts are scripts of newly added fields, whether a keyword cascade is added or not is detected, and whether the scripts lack keywords or not is detected, whether the list is changed into a partition list or not is required to be checked. The table name is extracted by regular mode, the ddl directory is traversed by the table name, the table building script of the latest version of the table is found out, whether the table is partitioned or not is judged, the result is fed back, and the problem that the newly added field of the partition table cannot be updated to the historical partition in the prior art is solved.

Referring to fig. 3, a third embodiment of the script detection method based on hive sql according to the embodiment of the present invention includes:

301. acquiring ddl script data to be executed in a hive database;

302. analyzing ddl script data to obtain table information corresponding to the script, and configuring a script detection control based on the table information;

specifically, a preset script analysis tool is called, and a table creation command and a script directory address in ddl script data are extracted; and configuring test contents in the blank script detection control based on the table creation command and the script directory address to obtain the script detection control corresponding to the ddl script data.

303. After receiving a script detection instruction, executing a script detection control, and traversing each script code in ddl script data;

in the step, all script codes in a test library are traversed by executing a script scanning model and a partition table identification model in a script detection control to extract the script codes corresponding to the partition table, specifically, table names corresponding to all script codes in ddl script data are firstly extracted, a traversal queue is created based on the table names, a function body and a command field in the script codes are scanned by using the script scanning model based on the queue, the function body and the command field are converted into an execution flow logic diagram, and then table creation contents in the execution flow logic diagram are identified by using the partition table model to obtain an identification result.

304. Detecting whether each script code has a newly added field;

in the step, identifying a structured query statement in the script code to obtain an SQL set;

specifically, when identifying a structured query statement, a code structure for checking whether a partition table exists is inserted for query, a creating function of the table is used as an identification object for the code structure, if the creating function exists in the script code, a code statement in the creating function is extracted, and the code statement is converted into the structured query statement to form an SQL set.

specifically, a getCode interface is called, an SQL set is read based on the interface, structured query statements in the SQL set are interpolated based on a communication protocol corresponding to a getCode method to obtain structured statements and Chinese annotations, then the Chinese annotations are deleted, the structured statements are converted into an uppercase form, and an SQL list is generated according to an execution sequence.

the regular analysis mode refers to a mode of repeatedly loading the structured statement to determine whether a new partition is generated or not so as to determine whether the alter field exists in the script code or not.

If yes, determining that a newly added field exists in the script code;

In practical application, extracting the script code of the partition table, analyzing the alter field in the script code, if the alter field exists, determining that the script code exists, determining that a newly added field exists, otherwise, determining that the newly added field does not exist. Of course, if the alter field exists, determining whether the alter field is a new field content is further included, and specifically, calculating a check value in a hash operation manner to determine the new field content.

305. If yes, intercepting the script section where the newly-added field is located;

specifically, the script segment may be intercepted by executing a cut model in the script detection control, where the cut model determines a complete code segment of the newly added field, such as a beginning byte and an ending byte of the recognition function.

306. Judging whether a partition connection field exists in the script segment or not;

307. if the address information does not exist, the address information where the script section is located is extracted, an abnormal log is generated based on the address information, and abnormality is prompted.

By the embodiment of the scheme, synchronous abnormity detection between the created script codes of the partition table and the historical partitions is realized, the execution success rate of the script codes is improved, the appearance of abnormity is reduced, and especially the problem that the abnormity cannot be detected due to the fact that the new partitions cannot be detected during execution is solved. Specifically, all scripts are traversed through a ddl script directory filled by a user, and if the scripts are scripts of newly added fields, whether keywords are added or not is detected, and whether the scripts lack the keywords or not is detected, whether the list is changed into a partition list or not is required. And extracting the table name by regular mode, traversing the ddl directory by using the table name, finding out a table building script of the latest version of the table, judging whether the table is partitioned or not, and feeding back a result.

In the above description of the script detection method based on hive sql in the embodiment of the present invention, a script detection apparatus based on hive sql in the embodiment of the present invention is described below, and referring to fig. 4, an embodiment of the script detection apparatus based on hive sql in the embodiment of the present invention includes:

the obtaining module 401 is configured to obtain ddl script data to be executed in the hive database;

a configuration module 402, configured to analyze the ddl script data to obtain table information corresponding to a script, and configure a script detection control based on the table information, where the script detection control is a function tool for detecting keyword abnormality of the ddl script data;

an extracting module 403, configured to execute the script detection control, and extract a script segment of a newly added field in the ddl script data;

a judging module 404, configured to judge whether a partition connection field exists in the script segment;

and a log generating module 405, configured to, when it is determined that the partition connection field does not exist, extract address information where the script segment is located, generate an abnormal log based on the address information, and prompt an abnormality.

Referring to fig. 5, another embodiment of the script detection apparatus based on hive sql according to the embodiment of the present invention includes:

Wherein the obtaining module 401 includes:

the determining unit 4011 is configured to determine a test library to be detected, where the test library includes a plurality of hive databases;

the construction unit 4012 is configured to construct a scan script based on the test library;

the traversing unit 4013 is configured to execute the scanning script and traverse the execution states of all scripts of the hive database in the test library;

and the selecting unit 4014 is configured to select ddl script data whose execution state is a to-be-executed state.

Wherein the configuration module 402 comprises:

the extracting unit 4021 is configured to invoke a preset script parsing tool, and extract a table creation command and a script directory address in the ddl script data;

a configuration unit 4022, configured to configure test content in the blank script detection control based on the table creation command and the script directory address, to obtain a script detection control corresponding to the ddl script data.

In this embodiment, the configuration unit 4022 is specifically configured to:

Wherein the extracting module 403 comprises:

the execution unit 4031 is configured to execute the script detection control after receiving the script detection instruction, and traverse each script code in the ddl script data;

a detecting unit 4032, configured to detect whether a newly added field exists in each script code;

an intercepting unit 4033, configured to intercept, when a newly added field is detected, a script segment where the newly added field is located.

In this embodiment, the detecting unit 4032 is specifically configured to:

if yes, determining that a newly added field exists in the script code;

Wherein the table information further comprises a table name; the log generation module 405 includes:

a query unit 4051, configured to query, according to the table name, a historical table building statement corresponding to the table name from a script directory address of the ddl script data;

a judging unit 4052, configured to judge, based on the historical table building statement, whether a table corresponding to the ddl script data is a partition table;

the generating unit 4053 is configured to, when it is determined that the partition table is the partition table, extract address information where the script segment is located, and generate an exception log according to a preset exception log format, where the exception log format is address information + an exception field.

In summary, the detection control is configured on the GG link to detect various tables in the database, determine whether a partition table exists in the various tables, and analyze whether a field in the partition table is updated, if so, detect whether a partition connection field exists, and if not, generate a detection log and prompt an exception. By the method, the abnormity of the partition table is detected, and the problem that historical partitions cannot be synchronized by executing scripts after fields are newly added in the partition table is solved.

Referring to fig. 6, an embodiment of a computer device according to an embodiment of the present invention will be described in detail from the perspective of hardware processing.

Fig. 6 is a schematic structural diagram of a computer device 600 according to an embodiment of the present invention, which may have a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 610 (e.g., one or more processors) and a memory 620, and one or more storage media 630 (e.g., one or more mass storage devices) for storing applications 633 or data 632. Memory 620 and storage medium 630 may be, among other things, transient or persistent storage. The program stored in the storage medium 630 may include one or more modules (not shown), each of which may include a sequence of instructions for operating on the computer device 600. Further, the processor 610 may be configured to communicate with the storage medium 630 to execute a series of instruction operations in the storage medium 630 on the computer device 600.

The computer device 600 may also include one or more power supplies 640, one or more wired or wireless network interfaces 650, one or more input-output interfaces 660, and/or one or more operating systems 631, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, and so forth. Those skilled in the art will appreciate that the computer device configuration illustrated in FIG. 6 is not intended to be limiting of the computer devices provided herein and may include more or fewer components than those illustrated, or some components may be combined, or a different arrangement of components.

The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium, and may also be a volatile computer-readable storage medium, where instructions are stored, and when the instructions are executed on a computer, the instructions cause the computer to execute the steps of the above-mentioned hive sql-based script detection method.

In practical applications, the above-provided method can be implemented based on Artificial Intelligence (AI) which is a theory, method, technique and application system that simulates, extends and expands human Intelligence, senses environment, acquires knowledge and uses knowledge to obtain the best result by using a digital computer or a machine controlled by a digital computer. The cloud server may be implemented based on a server, and the server may be an independent server, or may be a cloud server that provides basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a web service, cloud communication, middleware service, a domain name service, a security service, a Content Delivery Network (CDN), a big data and artificial intelligence platform, and the like.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A script detection method based on hive sql is characterized by comprising the following steps:

acquiring ddl script data to be executed in a hive database;

2. The hive sql-based script detection method according to claim 1, wherein the obtaining ddl script data to be executed in the hive database comprises:

constructing a scanning script based on the test library;

3. The hive sql-based script detection method according to claim 1, wherein the parsing the ddl script data to obtain table information corresponding to a script, and configuring a script detection control based on the table information includes:

4. The hive sql-based script detection method according to claim 3, wherein the configuring test contents in a blank script detection control based on the table creation command and the script directory address to obtain a script detection control corresponding to the ddl script data comprises:

5. The hive sql-based script detection method according to any one of claims 1 to 4, wherein the line of the script detection control extracts script segments of newly added fields in the ddl script data, and the method comprises the following steps:

detecting whether each script code has a newly added field;

6. The hive sql-based script detection method according to claim 5, wherein the detecting whether there is a new field in each script code comprises:

if yes, determining that a newly added field exists in the script code;

7. The hive sql-based script detection method of claim 6, wherein the table information further comprises a table name; the extracting the address information of the script segment and generating an abnormal log based on the address information includes:

8. A hive sql-based script detection device is characterized by comprising:

9. A computer device, characterized in that the computer device comprises: a memory having instructions stored therein and at least one processor, the memory and the at least one processor interconnected by a line;

the at least one processor invoking the instructions in the memory to cause the computer device to perform the steps of the hive sql-based script detection method of any of claims 1-7.

10. A computer-readable storage medium, having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the steps of the hive sql-based script detection method according to any of claims 1-7.