CN116244759A

CN116244759A - Request verification method, device and system based on calculation engine

Info

Publication number: CN116244759A
Application number: CN202310253116.5A
Authority: CN
Inventors: 郭健; 张乐
Original assignee: New H3C Big Data Technologies Co Ltd
Current assignee: New H3C Big Data Technologies Co Ltd
Priority date: 2023-03-15
Filing date: 2023-03-15
Publication date: 2023-06-09

Abstract

The invention discloses a request verification method, a request verification device and a request verification system based on a computing engine, wherein the method comprises the following steps: receiving an SQL request from a client, analyzing the SQL request to obtain an original data table, wherein the original data table comprises at least one table name used by at least one SQL statement, and setting a temporary table name corresponding to each table name; transmitting a creation instruction, wherein the creation instruction is used for creating a temporary data table with the same structure and different table names as the original data table; writing a small amount of data into the created temporary data table; the table names in the temporary data table are used for replacing table names in the original data table to generate a target SQL statement, the target SQL statement is sent to output an operation result when the temporary data table written with a small amount of data is executed, and the operation result is used for verifying the SQL request, so that a large amount of data in the original data table is prevented from being operated.

Description

Request verification method, device and system based on calculation engine

Technical Field

The present invention relates to the field of data processing technologies, and in particular, to a method, an apparatus, and a system for verifying a request based on a computing engine.

Background

Apache Spark (Spark for short) is an open source parallel processing framework, is a fast and general-purpose computing engine designed for large-scale data processing, and supports the use of in-memory processing to improve the performance of large data analysis applications. The method has the greatest characteristics that: both the calculated data and the intermediate results are stored in memory, thereby greatly reducing input/output I/O overhead. Spark SQL under Spark is based on Spark Core, which is a module that Spark uses to process structured data. Spark SQL contains more information on structured data and operations thereon. Spark SQL uses this information for additional optimization, making the manipulation of structured data more efficient and convenient.

Currently, related technologies mainly pre-execute SQL logic statements in various manners, such as triggering SQL and judging its state through the execution result and checking whether the execution result meets expectations, or increase monitoring of intermediate results by parsing SQL and presetting breakpoints. For example, as shown in fig. 1, one verification method is pre-execution, i.e., the most common Spark task general verification method. The main flow is as follows: firstly writing SQL sentences into Spark programs, compiling and packaging the completed programs, and uploading the completed programs to a cluster; after the cluster receives, submitting the task to run through Spark; checking a log of the task on a UI (resource manager) interface of the cluster (namely Apache Hadoop YARN), and verifying an execution result; if the task fails to execute or the verification result does not meet the expectations, repeating the steps until the verification is successful.

The above method has the disadvantages that if the table data amount used in the existing SQL is large, the SQL is very time-consuming to execute, and whether the result meets the expectations is difficult to judge. In addition, if the executed SQL statement is of a type of modifying data, deleting data, modifying a table structure and the like, once the task fails to execute, the original data is irreversibly affected.

Disclosure of Invention

The invention provides a request verification method, a request verification device and a request verification system based on a computing engine, and particularly discloses the following technical scheme:

in a first aspect, an embodiment of the present invention discloses a request verification method based on a computing engine, where the method is applicable to a server, and the method includes:

receiving an SQL request from a client, wherein the SQL request comprises at least one SQL statement;

analyzing the SQL request to obtain an original data table, wherein the original data table comprises at least one table name used by at least one SQL statement, and setting a temporary table name corresponding to each table name;

transmitting a creation instruction, wherein the creation instruction is used for creating a temporary data table with the same structure and different table names as the original data table;

writing a small amount of data into the created temporary data table;

replacing the table name in the original data table with the table name of the temporary data table, generating a target SQL statement, sending the target SQL statement, and outputting an operation result when the operation temporary data table written with a small amount of data is executed by the target SQL statement, wherein the operation result is used for verifying the SQL request.

With reference to the first aspect, in a possible implementation manner of the first aspect, the writing a small amount of data into the temporary data table includes: and sending a first writing instruction, wherein the first writing instruction is used for indicating that a small amount of data is sampled from the original data table and writing the small amount of data into the temporary data table.

With reference to the first aspect, in another possible implementation manner of the first aspect, the method further includes: and sending a second writing instruction, wherein the second writing instruction is used for writing data into the temporary data table again after writing a small amount of data once and when the data for running SQL logic is absent.

With reference to the first aspect, in a further possible implementation manner of the first aspect, the method further includes: receiving an operation result output after the target SQL statement is executed; and forwarding an operation result to the client, wherein the operation result is used for judging whether the logic of the target SQL statement is correct or not.

In a second aspect, the embodiment of the present invention further discloses a request verification method based on a computing engine, where the method may be applied to a network device, such as a machine where a Spark server is located, and the method includes:

receiving a creation instruction sent by a server;

creating a temporary data table according to the creation instruction, wherein the temporary data table has the same structure and different table names with the original data table of the server;

writing a small amount of data into a temporary data table according to the data interaction of the server;

receiving a target SQL statement sent by a server, wherein the table name of an original data table in the target SQL statement is replaced by the table name of a temporary data table;

and executing the target SQL statement by using the temporary data table written with a small amount of data, and outputting an operation result.

With reference to the second aspect, in a possible implementation manner of the second aspect, writing a small amount of data into the temporary data table according to data interaction of the server includes: receiving a first writing instruction sent by a server; and according to the first writing instruction, sampling a small amount of data from the original data table, and writing the small amount of data into the temporary data table.

With reference to the second aspect, in another possible implementation manner of the second aspect, the method further includes: receiving a second writing instruction sent by a server; and according to the second writing instruction, acquiring the data to be written, and writing the data to be written into the temporary data table again.

With reference to the second aspect, in a further possible implementation manner of the second aspect, the method further includes: and sending an operation result to the server, wherein the operation result is used for judging whether the logic of the target SQL statement is correct or not and verifying the SQL request of the client.

In a third aspect, the embodiment of the present invention further discloses a request verification device based on a computing engine, where the device includes:

the first receiving unit is used for receiving an SQL request from the client, wherein the SQL request comprises at least one SQL statement;

the first processing unit is used for analyzing the SQL request to obtain an original data table, wherein the original data table comprises at least one table name used by at least one SQL statement, and a temporary table name corresponding to each table name is set;

a first transmitting unit for transmitting a creation instruction for creating a temporary data table having the same structure as the original data table and a different table name, and writing a small amount of data into the created temporary data table;

the first processing unit is further configured to replace a table name in the original data table with a table name of the temporary data table, generate a target SQL statement, and send the target SQL statement through the first sending unit, where the target SQL statement outputs an operation result when the operation temporary data table written with a small amount of data is executed, and the operation result is used to verify the SQL request.

In a fourth aspect, the embodiment of the present invention further discloses a request verification device of a computing engine, where the device includes:

the second receiving unit is used for receiving the creation instruction sent by the server;

the second processing unit is used for creating a temporary data table according to the creation instruction, wherein the temporary data table has the same structure as the original data table of the server and different table names; writing a small amount of data into a temporary data table according to the data interaction of the server;

the second receiving unit is also used for receiving a target SQL statement sent by the server, wherein the table name of the original data table in the target SQL statement is replaced by the table name of the temporary data table;

the second processing unit is further used for executing the target SQL statement by using the temporary data table written with the small amount of data and outputting an operation result, wherein the operation result is used for verifying the SQL request of the client.

In a fifth aspect, an embodiment of the present invention further discloses a table processing system, where the system includes: the system comprises a client, a server and network equipment, wherein the client is used for sending an SQL request to the server, and the SQL request comprises at least one SQL statement; the server is a request verification device of the computing engine of the third aspect; the network device is the request verification device of the computing engine of the fourth aspect.

In a sixth aspect, the embodiment of the invention further discloses an electronic device, which comprises a processor and a memory, wherein the memory is coupled with the processor; the memory has stored thereon computer readable program instructions which, when executed by the processor, implement the first aspect or any implementation of the first aspect, and the computing engine based request verification method of the second aspect or any implementation of the second aspect.

Optionally, the electronic device is a terminal device or a client; the electronic device may also be a network device such as a server or a cluster of servers.

In addition, the embodiment of the invention also discloses a computer readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method for verifying the request based on the computing engine according to the first aspect or any implementation manner of the first aspect and the second aspect or any implementation manner of the second aspect is realized.

The Spark SQL-based table processing method, device and system provided by the embodiment of the invention have the following beneficial effects:

first, the verification speed is improved: by analyzing table names used in SQL, establishing temporary data tables with the same structure and different table names, and writing a small amount of data, when the method is operated, only the temporary data table written with a small amount of data is operated, and the original data table is not operated, so that the problems that the verification time is overlong and the verification speed is influenced due to the fact that the data quantity of the original data table is too large, such as 100W data, are avoided.

Second, validating available SQL statements is more comprehensive: the method can be applied to other types of SQL sentences (such as table building, deleting, data inserting and the like) besides the more common query sentences.

Thirdly, improving data security: because the method uses the temporary data table newly created aiming at the original data table and does not directly act on the original data table, the problems of original data pollution, false deletion and the like caused by executing the wrong SQL statement are avoided, and the safety of the original data is ensured.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a SQL verification method provided by the invention;

FIG. 2 is a schematic diagram of a Spark SQL scenario provided in an embodiment of the present invention;

FIG. 3 is a flowchart of a request verification method based on a computing engine according to an embodiment of the present invention;

FIG. 4 is a flowchart of another request verification method based on a computing engine according to an embodiment of the present invention;

FIG. 5 is a block diagram of a request verification device of a computing engine according to an embodiment of the present invention;

FIG. 6 is a block diagram of another computing engine request verification apparatus according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made apparent and fully in view of the accompanying drawings, in which some, but not all embodiments of the invention are shown. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

The embodiment of the application provides a more comprehensive, safe and rapid SQL verification method, which can be applied to a Spark SQL verification scene, as shown in fig. 2, in a Spark SQL verification system, the method comprises a client 100, a server 200 and a cluster 300, and the client 100, the server 200 and the cluster 300 can be connected and communicated through a network protocol.

Wherein the client 100 may be a browser, installed on a PC machine with a windows system. In addition, the client 100 may also be a terminal device, which may also be referred to as a User Equipment (UE), including but not limited to various personal computers PCs, notebooks, tablets, and tablets. In fig. 2, the client 100 is a desktop or PC.

The server 200 may be a stand-alone server or a server cluster formed by a plurality of servers, or may be a server providing cloud services/cloud computing, a data center, a Web server, a network service, cloud communication, a content delivery network (Content Delivery Network, CDN), and basic cloud computing services such as big data and an artificial intelligence platform.

Cluster 300 is a network device such as a server, or may be a server cluster, and both server 200 and cluster 300 are physical or virtual machines that install a Linux operating system.

Optionally, the cluster 300 is specifically a machine where the Spark server is located, such as a physical machine, a virtual machine, and so on.

The technical scheme provided by the embodiment of the application mainly relates to the SQL statement result verification process, and is particularly applied to Spark SQL execution environment, wherein Spark SQL is a module used for processing structured data in Spark, provides a programming abstraction (DataFrame), and can be used as a query engine of distributed SQL.

The Spark SQL can convert the calculation tasks of the data into RDD through SQL, and then submit the RDD to the cluster for calculation, and is similar to the way that Hive converts the calculation tasks of the data into MapReduce through SQL, so that the complexity of writing a Spark data calculation operation program is greatly simplified, and the execution efficiency is higher than that of a MapReduce calculation model. Where Hive is a data warehouse tool based on Hadoop.

Spark SQL consists of 4 parts in total, core, catalyst, hive and Hive-Thriftserver. Spark Core: is responsible for processing the input/output of Data, acquiring Data (such as RDD, HDFS, parquet file, JSON file and the like) from different Data sources, and then outputting the query result into a Data Frame. In general, spark Core contains the basic functions of Spark, such as memory computation, task scheduling, deployment mode, failure recovery, storage management, etc. In this embodiment, the server 200 and/or the cluster 300 mainly perform the corresponding steps of the method through Spark Core, so as to improve the verification speed and data security.

It should be noted that, the server 200 and the cluster 300 may be disposed in the same device, or may be disposed in different devices, which is not limited in this embodiment.

The method provided in the embodiments of the present application is described in detail below.

The embodiment of the invention discloses a table processing method based on Spark SQL, which comprises the following steps of:

in step 101, the client sends an SQL request to the server, and correspondingly, the server receives the SQL request from the client.

Specifically, a user writes an SQL statement through a client and generates an SQL request, wherein the SQL request comprises at least one SQL statement.

Step 102, the server analyzes the SQL request to obtain an original data table, wherein the original data table comprises at least one table name used by at least one SQL statement, and a temporary table name corresponding to each table name is set.

After the server analyzes the original data table, N table names are obtained, N is more than or equal to 1 and is a positive integer, one or more of the N table names can be used for each SQL statement, and a temporary table name is set for each of the N table names at the server.

For example, the SQL statement input by the client is: insert intra default.tab 1 select a.id, a.name, b.name from default.tab 2 a left join default.tab 3 b on a.id=b.id; the server analyzes the request SQL to obtain at least one table name comprising: defaults.tab 1, defaults.tab 2, defaults.tab 3. Then, the temporary table name automatically generated by the server is as follows: default. Table1_tmp_timestamp, default. Table2_tmp_timestamp, default. Table3_tmp_timestamp.

The server may generate temporary table names by customization.

Step 103, the server sends a creation instruction, where the creation instruction is used to instruct to create a temporary data table with the same structure and different table names as the original data table.

Specifically, the server sends the creation instruction to a network device, such as a device where the Spark server is located, and after receiving the creation instruction, the device generates a temporary data table. The temporary data table has the same structure and different table names as the original data table in the step 101. The table names are different, specifically, N tables are included in the temporary data table, but the table names of the N tables are different from those of the N tables in the original data table. For example, one of the original data tables has a table name of default.tab 1, and the temporary table name corresponding to the table name in the temporary data table has a temporary table name of default.tab 1_tmp_timestamp.

Step 104, the server writes a small amount of data into the created temporary data table.

One embodiment is realized by a sampling mode. Specifically, the server sends a first write instruction to the network device, where the first write instruction is used to instruct the network device to sample a small amount of data from an original data table, and write the small amount of data into the temporary data table. The writing means that a small amount of data is copied from the original data to the temporary data table, and the data in the original data table is not changed.

By sampling, the network device samples a small amount of data from the original data table and writes it into temporary data tables, each temporary data table in this way. For example, a request to write a small amount of data (e.g., a first write instruction) is sent by a server to a network device, where the write instruction is executed. Specific instructions may be in a manner using SQL, such as: insert intot temporary table as select from raw data table limit 100. The explanation is as follows: data not exceeding 100 is selected from the original data table to be inserted into the temporary table.

Optionally, the method further comprises: after the instruction of writing data is finished, the client can check the data in the temporary data table, check whether the written small amount of data accords with the service requirement of executing SQL, and if not, the written data needs to be modified and perfected.

In a possible implementation manner, the server sends a second writing instruction to the network device, where the second writing instruction is used to write data again to the temporary data table after writing a small amount of data once and when the data for running the SQL logic is absent.

Specifically, the client can check all temporary data tables corresponding to the original data tables in the SQL, upload the data file after selecting a certain temporary data table, send the data of the file to the server after analyzing the data of the file, splice the server into an instruction for writing the data of the temporary table, namely a second writing instruction, send the second writing instruction to the network equipment, and write the data into the temporary data table after receiving the second writing instruction by the network equipment. After the network equipment finishes writing the data, the client can check whether the written data meets the requirements, if not, writing a small amount of data again until the existing small amount of data in all temporary data tables can support SQL verification. If the client checks for compliance, step 105 is performed.

And 105, replacing the table names in the original data table with the table names in the temporary data table to generate a target SQL statement.

Specifically, the substitution table name includes: and replacing all table names in the original data table with table names in the temporary data table. The target SQL statement outputs an operation result when being executed by the temporary data table written with a small amount of data, and the operation result is used for verifying the SQL request of the client.

In an example, the target SQL statement includes a temporary table name that is replaced, e.g., after replacing the temporary table name, the resulting target SQL statement is: insert intra default. Table1_tmp_timestamp select a.id, a.name, b.name from default. Table2_tmp_timestamp a left join default. Table3_tmp_timestamp b on a.id=b.id.

The method does not change the data of the original table, only writes a small amount of data of the original table into the temporary data table, thereby improving the speed of requesting verification.

And step 106, the server sends the target SQL statement. Correspondingly, the network device receives the target SQL statement.

And step 107, the network device executes the target SQL statement by using the temporary data table written with a small amount of data, and outputs the operation result.

The specific process comprises the following steps: the network device generates an operation result according to the logic processing written in the SQL sentence and the temporary data table containing a small amount of data written in the conversion step 104.

For example, each table of the original data table contains 100W (ten thousand) data, a sampling method is used to query 100 data from each original data table and then write the data into the temporary data table, the writing mode can be triggered by a server, the server issues a command similar to a first writing command and/or a second writing command to the network device to execute, and the network device outputs an operation result after operating the target SQL statement.

And step 108, the network equipment sends the operation result to the server.

And step 109, after receiving the operation result, the server sends the operation result to the client.

In addition, the method further comprises:

after the client checks the operation result, if the operation result is correct, the original SQL statement can be issued for execution, if the operation result is incorrect, whether the operation result is a data problem or a logic problem of the SQL statement needs to be further judged, if the operation result is the logic problem of the SQL statement, the logic of the target SQL statement needs to be modified, so that the influence of wrong business logic on an original data table is avoided, if the operation result is a problem of writing data in a temporary data table, a file can be uploaded to the temporary data table again, and after the data of the temporary data table is updated, verification is performed.

The Spark SQL-based table processing method provided by the embodiment comprises the following beneficial effects:

Optionally, the method further comprises: the server and the network equipment keep the operation result and record the verification result of the client, thereby facilitating the subsequent check and improving the usability of the operation result. And after the verification of the operation result is finished, the system, such as a server or network equipment, also deletes the corresponding temporary data table at regular time, so that the data redundancy is avoided.

In this embodiment, the network device and the server side include Spark SQL calculation engines for executing the target SQL statement, where the Spark SQL calculation engines may be other calculation engines capable of analyzing the execution plan, and the other calculation engines include Hive SQL calculation engines, or clickHouse calculation engines, which is not limited in this embodiment.

In another embodiment, the present application further provides a request verification method based on a computing engine, where the method may apply a network device, such as a machine where a Spark server is located, as shown in fig. 4, and the method includes:

step 201, receiving a creation instruction sent by a server.

Step 202, creating a temporary data table according to the creation instruction, wherein the temporary data table has the same structure and different table names with the original data table of the server.

And 203, writing a small amount of data into the temporary data table according to the data interaction of the server.

And step 204, receiving a target SQL statement sent by the server, wherein the table name of the original data table in the target SQL statement is replaced by the table name of the temporary data table.

And 205, executing a target SQL statement by using the temporary data table written with a small amount of data, and outputting an operation result.

In addition, the method further comprises: the network device sends the operation result to the server, wherein the operation result is used for judging whether the logic of the target SQL statement is correct or not and verifying the SQL request of the client.

For a specific process, refer to the foregoing method steps in the embodiment shown in fig. 3, and this embodiment is not described herein again.

According to the request verification method for the Spark SQL task, the temporary data tables with the same structure and different table names are newly created on the network equipment side, a small amount of data is written in the temporary data tables, the small amount of data written in the temporary data tables is executed by SQL sentences replacing the temporary table names, and a Spark SQL result is obtained, so that a large amount of data in the original data tables are prevented from being operated, and verification efficiency and safety are improved.

In addition, after the verification is completed, the server periodically sends an instruction to the network equipment to delete the temporary data table generated in the verification process, so that the data redundancy and the resource occupation of the network equipment are avoided.

In another embodiment, the present application further provides a request verification device based on a computing engine, as shown in fig. 5, where the table device is configured to implement the foregoing request verification method shown in fig. 3 and fig. 4, and specifically the device includes: the first receiving unit 501, the first processing unit 502, and the first transmitting unit 503, and in addition, the apparatus may further include other functional modules such as a storage unit, which is not limited in this embodiment.

The first receiving unit 501 is configured to receive an SQL request from a client, where the SQL request includes at least one SQL statement.

The first processing unit 502 is configured to parse the SQL request to obtain an original data table, where the original data table includes at least one table name used by at least one SQL statement, and set a temporary table name corresponding to each table name.

A first sending unit 503, configured to send a creation instruction, where the creation instruction is used to instruct to create a temporary data table with the same table name and different from the original data table structure, and write a small amount of data into the created temporary data table.

The first processing unit 502 is further configured to replace a table name in the temporary data table with a table name in the original data table, generate a target SQL statement, and send the target SQL statement through the first sending unit 503, where the target SQL statement outputs an operation result when executed by the temporary data table written with a small amount of data, and the operation result is used to verify the SQL request.

Optionally, in a specific implementation manner of this embodiment, the first sending unit 503 is further configured to send a first write instruction, where the first write instruction is used to instruct sampling a small amount of data from the original data table, and write the small amount of data into the temporary data table.

Optionally, in another specific implementation manner of this embodiment, the first sending unit 503 is further configured to send a second write instruction, where the second write instruction is used to write the data again to the temporary data table after writing a small amount of data at a time, and when the data for running SQL logic is absent.

Optionally, in another specific implementation manner of this embodiment, the first receiving unit 501 is further configured to receive the running result output after the target SQL statement is executed; the first sending unit 503 is further configured to forward the operation result to a client, where the operation result is used to determine whether the logic of the target SQL statement is correct.

In addition, another request verification device based on a computing engine is provided in the embodiment of the present application, as shown in fig. 6, where the table processing device is configured to perform the foregoing request verification methods shown in fig. 3 and fig. 4, and specifically the device includes: the second receiving unit 601, the second processing unit 602, the second transmitting unit 603, and the like, and in addition, the apparatus may further include other functional modules such as a storage unit, which is not limited in this embodiment.

The second receiving unit 601 is configured to receive a creation instruction sent by the server.

A second processing unit 602, configured to create a temporary data table according to the creation instruction, where the temporary data table has the same structure and a different table name as the original data table of the server; and writing a small amount of data into the temporary data table according to the data interaction of the server.

The second receiving unit 601 is further configured to receive a target SQL statement sent by the server, where a table name of an original data table in the target SQL statement is replaced by a table name of a temporary data table.

The second processing unit 602 is further configured to execute a target SQL statement using the temporary data table written with a small amount of data, and output an operation result, where the operation result is used to verify an SQL request of the client.

Optionally, in a specific implementation manner of this embodiment, the second receiving unit 601 is further configured to receive a first write instruction sent by the server; the second processing unit 602 is further configured to sample a small amount of data from the original data table according to the first write instruction, and write the small amount of data into the temporary data table.

Optionally, in another specific implementation manner of this embodiment, the second receiving unit 601 is further configured to receive a second write instruction sent by the server; the second processing unit 602 is further configured to obtain data to be written according to the second writing instruction, and write the data to be written into the temporary data table again.

Optionally, in another specific implementation manner of this embodiment, the second sending unit 603 is configured to send an operation result to the server, where the operation result is used to determine whether the logic of the target SQL statement is correct.

In addition, the embodiment of the application also provides a request verification system of the computing engine, which comprises the following steps: client, server, network device, etc. The client is configured to send an SQL request to the server, where the SQL request includes at least one SQL statement, the server is a request verification device shown in fig. 5, and the network device is a request verification device shown in fig. 6. The system combines with the client, can realize the request verification method of the computing engine shown in the figure 3 or the figure 4, and effectively improves the security and the efficiency of task verification.

In addition, an electronic device is provided in an embodiment of the present invention, as shown in fig. 7, where the electronic device may include a processor 110 and a memory 120, where the processor 110 and the memory 120 may be connected by a bus or other manner, and in fig. 7, the connection is exemplified by a bus. In addition, the electronic device further includes at least one interface 130, where the at least one interface 130 may be a communication interface or other interfaces, and the embodiment is not limited thereto.

The processor 110 may be a central processing unit (Central Processing Unit, CPU). The processor 110 may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSPs), application specific integrated circuits (Application Specific Integrated Circuit, ASICs), field programmable gate arrays (Field-Programmable Gate Array, FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above.

The memory 120 is used as a non-transitory computer readable storage medium for storing non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to a table processing method in an embodiment of the present invention. The processor 110 executes various functional applications of the processor and data processing, i.e., implements the request verification method of the compute engine in the above-described method embodiments, by running non-transitory software programs, instructions, and modules stored in the memory 120.

Memory 120 may include a storage program area that may store an operating system, at least one application program required for functionality, and a storage data area; the storage data area may store data created by the processor 110, etc. In addition, memory 120 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 120 may optionally include memory located remotely from processor 110, which may be connected to processor 110 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

In this embodiment, the memory 120 may be used to store data such as an original data table, a temporary data table, a target SQL statement, and a running result.

In addition, at least one interface 130 is used for communication of the electronic device with external devices, such as with devices such as clients, network devices, etc. Optionally, at least one interface 130 may also be used to connect peripheral input, output devices, such as a keyboard, display screen, etc.

Alternatively, the electronic device may be a server or network device of the foregoing embodiment, or may also be a client.

The one or more modules are stored in the memory 120 and when executed by the processor 110 perform a request validation method of a compute engine as in the embodiments of fig. 3-4.

It will be appreciated by those skilled in the art that implementing all or part of the above-described embodiment method may be implemented by a computer program to instruct related hardware, where the program may be stored in a computer readable storage medium, and the program may include the above-described embodiment method when executed. The storage medium may be a magnetic Disk, an optical disc, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a Flash Memory (Flash Memory), a Hard Disk (HDD), a Solid State Drive (SSD), or the like; the storage medium may also comprise a combination of memories of the kind described above.

Although embodiments of the present invention have been described in connection with the accompanying drawings, various modifications and variations may be made by those skilled in the art without departing from the spirit and scope of the invention, and such modifications and variations are within the scope of the invention as defined by the appended claims.

Claims

1. A computing engine-based request verification method, the method comprising:

writing a small amount of data into the created temporary data table;

replacing the table names in the original data table with the table names in the temporary data table, generating a target SQL statement, and sending the target SQL statement, wherein the target SQL statement outputs an operation result when the temporary data table written with a small amount of data is executed, and the operation result is used for verifying the SQL request.

2. The method of claim 1, wherein writing a small amount of data into the created temporary data table comprises:

and sending a first writing instruction, wherein the first writing instruction is used for indicating that a small amount of data is sampled from an original data table and writing the small amount of data into the temporary data table.

3. The method according to claim 2, wherein the method further comprises:

and sending a second writing instruction, wherein the second writing instruction is used for writing data into the temporary data table again after writing a small amount of data once and when the data for running SQL logic is absent.

4. A method according to any one of claims 1-3, wherein the method further comprises:

receiving the operation result output after the target SQL statement is executed;

and forwarding the operation result to the client, wherein the operation result is used for judging whether the logic of the target SQL statement is correct or not.

5. A computing engine-based request verification method, the method comprising:

receiving a creation instruction sent by a server;

writing a small amount of data into the temporary data table according to the data interaction of the server;

receiving a target SQL statement sent by the server, wherein the table name of an original data table in the target SQL statement is replaced by the table name of the temporary data table;

6. The method of claim 5, wherein writing a small amount of data to the temporary data table based on the server's data interactions, comprises:

receiving a first writing instruction sent by the server;

and sampling a small amount of data from an original data table according to the first writing instruction, and writing the small amount of data into the temporary data table.

7. The method of claim 6, wherein the method further comprises:

receiving a second writing instruction sent by the server;

and acquiring data to be written according to the second writing instruction, and writing the data to be written into the temporary data table again.

8. The method according to any one of claims 5-7, further comprising:

and sending the operation result to the server, wherein the operation result is used for judging whether the logic of the target SQL statement is correct or not and verifying the SQL request of the client.

9. A computing engine-based request verification apparatus, the apparatus comprising:

the first processing unit is used for analyzing the SQL request to obtain an original data table, wherein the original data table comprises at least one table name used by at least one SQL statement, and temporary table names corresponding to each table name are set;

a first sending unit, configured to send a creation instruction, where the creation instruction is used to create a temporary data table with the same structure and different table names as the original data table, and write a small amount of data into the created temporary data table;

the first processing unit is further configured to replace a table name in the temporary data table with a table name in the original data table, generate a target SQL statement, and send the target SQL statement through the first sending unit, where the target SQL statement outputs an operation result when the temporary data table written with a small amount of data is executed, and the operation result is used to verify the SQL request.

10. A computing engine-based request verification apparatus, the apparatus comprising:

the second processing unit is used for creating a temporary data table according to the creation instruction, wherein the temporary data table has the same structure and different table names with the original data table of the server; writing a small amount of data into the temporary data table according to the data interaction of the server;

the second receiving unit is further configured to receive a target SQL statement sent by the server, where a table name of an original data table in the target SQL statement is replaced by a table name of the temporary data table;

the second processing unit is further configured to execute the target SQL statement by using the temporary data table written with a small amount of data, and output an operation result, where the operation result is used to verify an SQL request of the client.

11. A request verification system for a computing engine, the system comprising: client, server, and network device, wherein,

the client is used for sending an SQL request to the server, wherein the SQL request comprises at least one SQL statement;

the server is a compute engine based request verification device of claim 9;

the network device is a compute engine based request verification apparatus as claimed in claim 10.

12. An electronic device comprising a processor and a memory, the memory coupled to the processor;

the memory has stored thereon computer readable program instructions which, when executed by the processor, implement the compute engine based request verification method of any of claims 1 to 8.

13. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements the compute engine based request verification method of any one of claims 1 to 8.