CN112965711A

CN112965711A - Job test method and apparatus, electronic device, and storage medium

Info

Publication number: CN112965711A
Application number: CN202110240733.2A
Authority: CN
Inventors: 曹振
Original assignee: Beijing Kingsoft Cloud Network Technology Co Ltd
Current assignee: Beijing Kingsoft Cloud Network Technology Co Ltd
Priority date: 2021-03-04
Filing date: 2021-03-04
Publication date: 2021-06-15

Abstract

The application provides a job testing method and device, electronic equipment and a storage medium, wherein the method comprises the following steps: detecting a trigger operation executed on a target visual interface, wherein the trigger operation is used for triggering the test of a target job corresponding to a target job flow in the target visual interface, and the target job is a job for processing a target data source by using a target processing operator; responding to the trigger operation, and selecting target test data corresponding to the target data source, wherein the target test data is used for replacing the target data source; and sending the target operation flow and the target identification information of the target test data to the target server so that the target server processes the target test data by using a target processing operator to obtain a target test result of the target operation. By the method and the device, the problems of low testing efficiency and poor user experience caused by complex testing process in the operation testing mode in the related technology are solved.

Description

Job test method and apparatus, electronic device, and storage medium

Technical Field

The present application relates to the field of data processing, and in particular, to a job testing method and apparatus, an electronic device, and a storage medium.

Background

In a real-time computing platform, a user can drag operator development operation, and the three types of operators are respectively: source (input, indicating from which data is read), processing operator (indicating how data is to be processed after being read), sink (output, indicating to which processed data is to be written).

When a user develops a job, for some logics which are particularly complex in processing and are likely to be wrongly written, the processing operators are generally required to be adjusted for realization for many times. Therefore, the logic which is particularly complex in processing generally needs to be tested and run for many times, namely, real data is read from the source for many times, the data is written into the sink through a processing operator, and then whether the data is in accordance with the expectation is inquired through a data source inquiry mode.

In the process of each test operation, the real-time computing platform needs to perform data management to acquire the physical address information of the source and the sink, and then connects the corresponding source and sink, and if the data source is unavailable, the test cannot be performed. Moreover, the user needs to operate the source and sink data sources to verify whether the own processing logic meets expectations, and the method is not friendly to the user.

Therefore, the operation test mode in the related technology has the problems of low test efficiency and poor user experience caused by complex test process.

Disclosure of Invention

The application provides a job testing method and device, electronic equipment and a storage medium, which are used for at least solving the problems of low testing efficiency and poor user experience caused by complex testing process in a job testing mode in the related technology.

According to an aspect of an embodiment of the present application, there is provided a job testing method including: detecting a trigger operation executed on a target visual interface, wherein the trigger operation is used for triggering a target job corresponding to a target job flow in the target visual interface to be tested, and the target job is a job processed on a target data source by using a target processing operator; responding to the trigger operation, and selecting target test data corresponding to the target data source, wherein the target test data is used for replacing the target data source; and sending the target operation flow and the target identification information of the target test data to a target server so that the target server processes the target test data by using the target processing operator to obtain a target test result of the target operation.

Optionally, selecting the target test data corresponding to the target data source includes: and selecting the target test data for replacing the target data source through a target configuration page, wherein the target configuration page is a page which is popped up after the trigger operation is detected and is used for configuring the test parameters of the target operation.

Optionally, after detecting the triggering operation performed on the target visualization interface, the method further includes: configuring target test parameters of the target operation through the target configuration page, wherein the target test parameters comprise at least one of the following: the reading rate of the target test data, and the computing resources for running the target operation.

Optionally, after sending the target workflow and the target identification information of the target test data to a target server, the method further includes: receiving target display information sent by the target server, wherein the target display information is used for describing the target test result; and displaying the target display information through a target display page so as to display the target test result on the target display page.

According to another aspect of the embodiments of the present application, there is also provided a job testing method, including: acquiring a target operation flow of target operation and target identification information of target test data, wherein the target operation is the operation of processing a first data source by using a target processing operator, and the target test data is used for replacing the first data source; constructing the target job by using the target job flow and the target identification information, wherein the first data source in the target job is replaced by the target test data; and testing the target operation through a target computing engine to obtain a target test result of the target operation, wherein the target test result is obtained after the target test data is processed by using the target processing operator.

Optionally, after the target job is tested by the target computing engine to obtain the target test result of the target job, the method further includes: receiving the target test result input by the target computing engine through a target link through a target input interface; and storing the target test result.

Optionally, before the target job is built by using the target job flow and the target identification information, the method further includes: replacing the data source information of the first data source in the target operation flow with the target identification information; under the condition that a second data source of the target operation is not associated with a registered data source, replacing data source information of the second data source in the target operation flow with target output information to obtain an updated target operation flow; the second data source is a data source into which the test result of the target job is written, and the target output information is used for indicating that the test result of the target job is output to the target input interface through the target link.

Optionally, after the target job is tested by the target computing engine to obtain the target test result of the target job, the method further includes: and sending target display information for describing the target test result to target terminal equipment so as to display the target test result on the target terminal equipment.

According to still another aspect of an embodiment of the present application, there is also provided a job testing apparatus including: the system comprises a detection unit, a data processing unit and a data processing unit, wherein the detection unit is used for detecting a trigger operation executed on a target visual interface, the trigger operation is used for triggering the test of a target job corresponding to a target job flow in the target visual interface, and the target job is a job processed on a target data source by using a target processing operator; the selecting unit is used for responding to the trigger operation and selecting target test data corresponding to the target data source, wherein the target test data is used for replacing the target data source; and the first sending unit is used for sending the target operation process and the target identification information of the target test data to a target server so that the target server processes the target test data by using the target processing operator to obtain a target test result of the target operation.

Optionally, the selecting unit includes: and the selecting module is used for selecting the target test data for replacing the target data source through a target configuration page, wherein the target configuration page is a page which is popped up after the trigger operation is detected and is used for configuring the test parameters of the target operation.

Optionally, the apparatus further comprises: a configuration unit, configured to configure, through the target configuration page, target test parameters of the target job after the trigger operation performed on the target visualization interface is detected, where the target test parameters include at least one of: the reading rate of the target test data, and the computing resources for running the target operation.

Optionally, the apparatus further comprises: a first receiving unit, configured to receive target display information sent by a target server after the target identification information of the target test data and the target workflow is sent to the target server, where the target display information is used to describe the target test result; and the display unit is used for displaying the target display information through a target display page so as to display the target test result on the target display page.

According to still another aspect of an embodiment of the present application, there is also provided a job testing apparatus including: the system comprises an acquisition unit, a storage unit and a processing unit, wherein the acquisition unit is used for acquiring a target operation flow of target operation and target identification information of target test data, the target operation is operation for processing a first data source by using a target processing operator, and the target test data is used for replacing the first data source; a construction unit, configured to construct the target job by using the target job flow and the target identification information, where the first data source in the target job is replaced with the target test data; and the test unit is used for testing the target operation through a target calculation engine to obtain a target test result of the target operation, wherein the target test result is obtained after the target test data is processed by using the target processing operator.

Optionally, the apparatus further comprises: a second receiving unit, configured to receive, through a target input interface, the target test result input by the target computing engine through a target link after the target computing engine tests the target job to obtain the target test result of the target job; and the storage unit is used for storing the target test result.

Optionally, the apparatus further comprises: a first replacing unit, configured to replace data source information of the first data source in the target workflow with the target identification information before the target job is constructed using the target workflow and the target identification information; a second replacing unit, configured to replace, when a second data source of the target job is not associated with a registered data source, data source information of the second data source in the target job flow with target output information, so as to obtain an updated target job flow; the second data source is a data source into which the test result of the target job is written, and the target output information is used for indicating that the test result of the target job is output to the target input interface through the target link.

Optionally, the apparatus further comprises: and the second sending unit is used for sending target display information for describing the target test result to target terminal equipment after the target operation is tested by the target computing engine to obtain the target test result of the target operation, so that the target test result is displayed on the target terminal equipment.

According to another aspect of the embodiments of the present application, there is also provided an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory communicate with each other through the communication bus; wherein the memory is used for storing the computer program; a processor for performing the method steps in any of the above embodiments by running the computer program stored on the memory.

According to a further aspect of the embodiments of the present application, there is also provided a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to perform the method steps of any of the above embodiments when the computer program is executed.

In the embodiment of the application, a data file (e.g., a text file) is used to replace a source data source, and a trigger operation performed on a target visualization interface is detected, where the trigger operation is used to trigger a test on a target job corresponding to a target workflow in the target visualization interface, and the target job is a job processed on the target data source by using a target processing operator; responding to the trigger operation, and selecting target test data corresponding to the target data source, wherein the target test data is used for replacing the target data source; the target operation process and the target identification information of the target test data are sent to the target server, so that the target server processes the target test data by using a target processing operator to obtain a target test result of the target operation, and the source data source connection process can be shielded by using a data file to replace a source data source, so that the purpose of simplifying the test process of the operation can be realized, the technical effects of improving the operation rapidity, improving the operation test efficiency and improving the user experience are achieved, and the problems of low test efficiency and poor user experience caused by the complex test process in the operation test mode in the related technology are solved.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.

FIG. 1 is a schematic diagram of a hardware environment for an alternative job testing method according to an embodiment of the present application;

FIG. 2 is a schematic flow chart diagram illustrating an alternative job testing method according to an embodiment of the present application;

FIG. 3 is a schematic diagram of an alternative job testing method according to an embodiment of the present application;

FIG. 4 is a schematic flow chart diagram illustrating an alternative job testing method according to an embodiment of the present application;

FIG. 5 is a schematic flow chart diagram illustrating yet another alternative job testing method according to an embodiment of the present application;

FIG. 6 is a schematic diagram of an alternative job testing method according to an embodiment of the present application;

FIG. 7 is a block diagram of an alternative job testing apparatus according to an embodiment of the present application;

FIG. 8 is a block diagram of an alternative job testing apparatus according to an embodiment of the present application;

fig. 9 is a block diagram of an alternative electronic device according to an embodiment of the present application.

Detailed Description

In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

According to one aspect of an embodiment of the present application, a job testing method is provided. Alternatively, in the present embodiment, the job testing method described above may be applied to a hardware environment formed by the terminal 102 and the server 104 as shown in fig. 1. As shown in fig. 1, the server 104 is connected to the terminal 102 through a network, and may be used to provide services (e.g., game services, application services, etc.) for the terminal or a client installed on the terminal. When applied, a database may be provided on the server or separate from the server for providing data storage services for the server 104.

The network may include, but is not limited to, at least one of: wired networks, wireless networks. The wired network may include, but is not limited to, at least one of: wide area networks, metropolitan area networks, local area networks, which may include, but are not limited to, at least one of the following: WIFI (Wireless Fidelity), bluetooth. The terminal 102 may not be limited to a PC, a mobile phone, a tablet computer, etc.

The job testing method according to the embodiment of the present application may be executed by the server 104, may be executed by the terminal 102, or may be executed by both the server 104 and the terminal 102. The terminal 102 may execute the job testing method according to the embodiment of the present application by a client installed thereon.

Taking the terminal 102 to execute the job testing method in this embodiment as an example, fig. 2 is a schematic flow chart of an optional job testing method according to an embodiment of the present application, and as shown in fig. 2, the flow chart of the method may include the following steps:

step S202, detecting a trigger operation executed on the target visualization interface, where the trigger operation is used to trigger a test on a target job corresponding to a target job flow in the target visualization interface, and the target job is a job processed on the target data source by using a target processing operator.

The job testing method in this embodiment can be applied to a network architecture having a job testing function, such as a real-time computing network architecture, in which a unit test can be performed on processing logic of a job developed by a user. In developing a job, it is simply assumed that the processing logic of the job developed by the user produces a corresponding result under the user-defined output. To verify the correctness of the processing logic of a job, the user developed job may be tested using unit testing.

Taking a real-time computing network architecture as an example, the real-time computing network architecture may include a real-time computing platform, a real-time computing service, and a real-time computing engine, where the real-time computing platform may operate on a terminal device side, the real-time computing service may operate on a server side, and the real-time computing engine may operate on the terminal device side or the server side.

The real-time computing platform may be a user operating platform that provides users with the ability to build real-time computing jobs. The real-time computing platform may take many forms, and may be, for example, a big data platform. For users, they are only aware of the real-time computing platform. A real-time computing service is a background service, imperceptible to a user, which may refer to a device that provides the relevant operations to maintain the lifecycle of a real-time computing job. The real-time computing engine is an engine providing a real-time computing service, for example, a flink (a distributed stream data streaming engine), which may test the created job to obtain a test result.

It should be noted that the real-time computing service is an example of a target server, and the operations performed by the real-time computing service in the embodiments of the present application may all be performed by the target server. The target server may have a variety of functions, such as performing maintenance operations for the lifecycle of a job (e.g., a real-time computing job), and also, for example, converting a workflow into an implementation graph of the job, which is job processing logic (job processing flow) recognizable by the compute engine. The target server is not limited in this embodiment.

The job testing method in this embodiment may be executed by a terminal device on the real-time computing platform side, where the terminal device may operate a Linux system (an operating system). A visual interface, which may be a user development page, may be displayed on the terminal device. A user can drag an operator development job on the visual interface, and the operator development job has various types, which can include but are not limited to: source data source, processing operator and sink data source.

For a target user (corresponding to a target object), a target visualization interface may be displayed on a target terminal device of the target user. According to the processing logic of the target operation, the target user can drag the operators of the source data source, the processing operator and the sink data source to a target area of the visual interface and connect the operators with one another, the connecting line among the operators represents the flow direction of the data, and a data flow chart, namely a target operation flow, is formed after the connecting line.

The job processing flow indicated by the target job flow is as follows: data is read from a target data source (i.e., source data source, input data source), the read data is processed by using a target processing operator, and the result of the target processing operator is stored in an output data source (sink data source). Correspondingly, the target job is a job that processes the target data source using the target processing operator. Different jobs may use any data source, depending on the processing logic. For the target job, the number of target data sources may be one or more, and the number of output data sources may also be one or more.

After the target workflow of the target job is constructed, the target user may perform a trigger operation on the target visualization interface, where the trigger operation may be a single operation or a combination of multiple operations, such as a click operation, a double click operation, a drag operation, and the like. For example, click a target button on the target visualization interface, e.g., a "unit test" button, to trigger testing of the target job.

The target terminal device may detect the above-mentioned trigger operation executed by the target object (corresponding to the target user, and may be identified by the target account, the terminal device number, and the like) on the target visual interface. In response to the trigger operation, the target terminal device may trigger a process of testing the target job.

And step S204, responding to the trigger operation, and selecting target test data corresponding to the target data source, wherein the target test data is used for replacing the target data source.

In response to the trigger operation, the target terminal device may acquire job flow information of the target job flow. The acquired workflow information may include a plurality of pieces of attribute information of the target data source (e.g., identification information of the target data source), attribute information of the target processing operator, and attribute information of the output data source, for example.

In the related art, the source data source or the sink data source may be associated with a registered data source, and the registered data source may be provided by a user. Firstly, a user registers a data source in a platform, and relevant information of the data source, such as physical address information, a table structure and the like, can be filled in during registration; then, in one job, the user can drag the source and sink operators and select which data source this source or sink uses among them. When the data source is selected, the data source can be selected according to the name of the data source, so that the real-time computing platform can inquire the physical address, the table structure and the like of the corresponding data source according to the registered data source information.

The data source may include, but is not limited to, at least one of: kafka (a high throughput distributed publish-subscribe messaging system) message queue, MySQL (a relational database management system), hbase (a distributed, column-oriented open source database), hive (a data warehouse tool).

For example, as shown in fig. 3, for a workflow, it may read data from some physical data source (e.g., kafka message queue), and perform complex sql (Structured Query Language) processing to obtain a test result of the workflow, and the test result may be written into some physical data source (e.g., MySQL, hbase, live, etc.).

Further, in a general user scenario, the user's processing logic may be: and splitting one data into a plurality of data according to different logics, and then storing the data in different data sources for different service scenes. Then, a processing flow as shown in fig. 3 may occur, i.e., the data is first split into two parts through one sql processing, and then the two parts of data are output to the same or different data sources through different sql processing.

The above-mentioned operation test method has at least the following problems:

aiming at the problem that the user is not friendly enough, the user needs to operate the source and sink data sources to verify whether the own processing logic accords with expectations or not;

for a real-time computing platform, the platform needs to perform data management to acquire physical address information of source and sink and connect the corresponding source and sink, and if a data source is unavailable, testing cannot be performed.

Optionally, in this embodiment, the target test data is used to replace the target data source, so that the problem in the connection process of the target data source can be shielded, the operation is faster and convenient to test, and the problem can be at least partially solved. The target test data may be data in a text file, that is, the target test data may be text test data, or may be other types of data, which is not limited in this embodiment. In this embodiment, text test data is taken as an example for explanation.

The text test data can be uploaded to the platform as a resource, for example, in a text file. In conducting the unit test, the user may select which text test data to replace which source data source.

For the target terminal device, the target user may select text test data corresponding to the target data source based on an operation of the target user or a default configuration, that is, the target test data, and the selected target test data may be used to replace the target data source.

Step S206, the target identification information of the target operation process and the target test data is sent to the target server, so that the target server processes the target test data by using the target processing operator to obtain the target test result of the target operation.

The target terminal device may send the target workflow and the target identification information of the target test data to a target server, for example, a target computing service. The target server may integrate the target job according to the target identification information of the target job flow and the target test data, for example, replace the target data source with the target test data, and integrate the target job flow into the implementation graph of the target job.

The integrated target jobs may be tested by a real-time computing engine. The real-time computing engine may process the target test data using the target processing operator to obtain a test result output by the target processing operator, that is, a target test result of the target job.

Through the steps S202 to S206, a trigger operation performed on the target visualization interface is detected, where the trigger operation is used to trigger testing of a target job corresponding to a target job flow in the target visualization interface, and the target job is a job processed on a target data source by using a target processing operator; responding to the trigger operation, and selecting target test data corresponding to the target data source, wherein the target test data is used for replacing the target data source; the target operation process and the target identification information of the target test data are sent to the target server, so that the target server processes the target test data by using a target processing operator to obtain a target test result of the target operation, the problems of low test efficiency and poor user experience caused by complex test process in an operation test mode in the related technology are solved, the operation running rapidity is improved, the operation test efficiency is improved, and the user experience is also improved.

As an alternative embodiment, selecting the target test data corresponding to the target data source includes:

and S11, selecting target test data for replacing the target data source through a target configuration page, wherein the target configuration page is a page which is popped up after the trigger operation is detected and is used for configuring the test parameters of the target operation.

In order to improve the convenience of user operation, the configuration of the test parameters of the target operation can be carried out through the configuration page. After detecting the trigger operation, the target terminal device may pop up a target configuration page on the target visualization interface, where the configuration page may be a page for configuring test parameters of the target job, for example, a unit test page.

The target user can fill out which text test data replaces which source data source on the target configuration page. The target terminal device may detect a selection operation performed by the target object on the target configuration page, where the selection operation is used to select text test data that replaces the target data source. In response to the selection operation, the target terminal device may determine target test data for replacing the target data source.

Through this embodiment, select the test data that is used for replacing source data source through the configuration page, can improve the convenience of user operation, promote user's use and experience.

As an optional embodiment, after detecting the triggering operation performed on the target visualization interface, the method further includes:

s21, configuring target test parameters of the target operation through the target configuration page, wherein the target test parameters comprise at least one of the following: the read rate of the target test data, and the computational resources to run the target job.

In addition to selecting test data for replacing the target data source, the target user may also configure other test parameters of the target job, e.g., target test parameters, via the target configuration page. The target test parameters may be one or more of, and may include, but are not limited to, at least one of: the read rate of the target test data, and the computational resources to run the target job.

The target terminal device may detect a configuration operation performed by the target object on the target configuration page, where the configuration operation is used to configure target test parameters of the target job. In response to the configuration operation, the target terminal device may determine target test parameters for the target job.

For example, on the unit test page, the user may also perform some special configuration of the unit test (e.g., select the rate at which data is read), may also select the resources to run, etc. The operating resource is understood to be a computing resource, and may be regarded as a Central Processing Unit (CPU) in Linux, for example, 1 core 4G.

By the embodiment, the test parameters of the unit test are configured by the configuration page, so that the convenience and flexibility of unit test configuration can be improved.

As an optional embodiment, after sending the target identification information of the target workflow and the target test data to the target server, the method further includes:

s31, receiving target display information sent by a target server, wherein the target display information is used for describing a target test result;

and S32, displaying the target display information through the target display page so as to display the target test result on the target display page.

After obtaining the target test result, the target server may send the target display information to the target terminal device. The target display information may be used to describe a target test result, which may be the target test result itself, or may be information obtained after performing a certain processing on the target test result, where the performed processing may be multiple, for example, adding a summary description, and the like, which is not limited in this embodiment.

After receiving the target display information sent by the target server, the target terminal device may display the target display information through the target display page to display the target test result on the target display page. The target presentation page may be a text box or other presentation pages.

Through this embodiment, carry out the show of test result through the show page, can make things convenient for the user to look over the test result of operation, improve the convenience that information acquisition.

The above-described job testing method is described below with reference to an alternative example. When a user performs a unit test, after the user clicks a unit test button on a visual interface, a page (similar to the target configuration page) pops up, and the page guides the user to fill in the following contents: replacing the source data source with which file test data; some special configurations of unit testing (e.g., text read rate); selecting resources to run, etc. After the user clicks the "run" button in the page, the job test can be run successfully. After the operation test is completed, a text box (target display page) appears on the user interface, and the result of the operation, that is, the data output by the processing operator, is displayed in real time.

According to another aspect of the embodiment of the application, a job testing method is further provided. Alternatively, in the present embodiment, the job testing method described above may be applied to a hardware environment formed by the terminal 102 and the server 104 as shown in fig. 1. The description is already given and will not be repeated herein.

Taking the server 104 to execute the job testing method in this embodiment as an example, fig. 4 is a schematic flowchart of another optional job testing method according to an embodiment of the present application, and as shown in fig. 4, the flowchart of the method may include the following steps:

step S402, a target operation flow of a target operation and target identification information of target test data are obtained, wherein the target operation is an operation for processing the first data source by using a target processing operator, and the target test data is used for replacing the first data source.

The job testing method in this embodiment can be applied to the network architecture with the job testing function, for example, a real-time computing network architecture. Has already been described and will not be described in detail herein. The job testing method in this embodiment may be executed by the target terminal device or the real-time computing service, and in this embodiment, the job testing method is described as an example executed by the real-time computing service.

The real-time computing service may obtain target workflow and target identification information of the target test data of the target job, e.g., receive the target workflow and target identification information transmitted by the target terminal device through a communication connection with the real-time computing service and the target terminal device. The first data source (i.e., the target data source), the target job flow and the target test data are similar to those described above, and are not described herein again.

Step S404, a target job is constructed by using the target job flow and the target identification information, wherein the first data source in the target job is replaced by the target test data.

The real-time computing service may construct a target job using the target workflow and the target identification information, i.e., convert the target workflow into an implementation graph of the target job executable by a target computing engine, which may be a real-time computing engine.

Before the target job is constructed by using the target job flow and the target identification information, the data source information of the first data source in the target job flow can be replaced by the target identification information, so that the updated target job flow is obtained. Through the steps, the first data source can be replaced by the target test data.

Step S406, testing the target operation through the target calculation engine to obtain a target test result of the target operation, wherein the target test result is obtained after the target test data is processed by using a target processing operator.

For the constructed target job, the real-time computing service may test the target job through the target computing engine. If the real-time computing service and the target computing engine are located on the same device (e.g., target server), the target computing engine may be controlled directly by the device to test the target job.

Alternatively, if the real-time computing service and the target computing engine are located in different devices (for example, the real-time computing service is located in a target server, and the target computing engine is located in another server or a target terminal device), the built target job may be sent to the device where the target computing engine is located by the real-time computing service, and the device controls the target computing engine to test the target job.

The target calculation engine may read the target test data according to the target identification information, and process the target test data using the target processing operator to obtain a result output by the target processing operator, that is, a target test result of the target job.

It should be noted that the target test data may serve as a local data source, and correspondingly, the target job may be run in a local mode, so that the test flow of the target job may be started more quickly. The target terminal device may not perform step S206 but directly perform steps S402 to S406, or perform step S206 and then perform step S406, that is, the target terminal device tests the target job by using the target computing engine to obtain a target test result of the target job.

Through the above steps S402 to S406, a target job flow of a target job and target identification information of target test data are obtained, where the target job is a job for processing a first data source by using a target processing operator, and the target test data is used to replace the first data source; constructing a target operation by using the target operation flow and the target identification information, wherein a first data source in the target operation is replaced by target test data; the target operation is tested through the target computing engine, and a target test result of the target operation is obtained, wherein the target test result is obtained after target test data are processed through a target processing operator, so that the problems of low test efficiency and poor user experience caused by complex test process in an operation test mode in the related technology are solved, the operation running rapidity is improved, the operation test efficiency is improved, and the user experience is also improved.

As an alternative embodiment, after the target job is tested by the target computing engine to obtain the target test result of the target job, the method further includes:

s41, receiving a target test result input by the target calculation engine through the target link through the target input interface;

s42, storing the target test result.

The output data source of the target job may be predefined by the user, for example, by configuring a registered data source associated with the second data source (i.e., the output data source of the target job) via a target configuration page, the target test result may be written to the registered data source.

Alternatively, if the second data source of the target job is not associated with a registered data source, the write location of the target test result may be determined according to default configuration information. For example, the target test results may be saved to the real-time computing service via httpsink.

Here, httpsink is a way for outputting by a flink computing engine, and is output to a certain interface through an http (HyperText Transfer Protocol) link, where the interface may be fixed, and different test results may be output to different interfaces.

For the target test result, the target computing engine may output the target test result to the target input interface through a target link (http link). The real-time computing service can receive a target test result input by the target computing engine through the target link through the target input interface and store the target test result.

Through the embodiment, the test result input through the target link is received and stored through the specific interface, so that the user operation can be simplified, and the use experience of the user is improved.

As an alternative embodiment, before constructing the target job by using the target job flow and the target identification information, the method further includes:

s51, replacing the data source information of the first data source in the target work flow with target identification information;

s52, under the condition that the second data source of the target operation is not associated with the registered data source, replacing the data source information of the second data source in the target operation flow with the target output information to obtain an updated target operation flow;

the second data source is a data source in which the test result of the target operation is written, and the target output information is used for indicating that the test result of the target operation is output to the target input interface through the target link.

Before the target job is constructed, except that the data source information of the first data source in the target job flow is replaced by the target identification information, if the second data source of the target job is not associated with the registered data source, the data source information of the second data source in the target job flow can be replaced by default configuration information, so that the updated target job flow is obtained. Here, the second data source is a data source to which the test result of the target job is written, that is, an output data source of the target job.

The default configuration information may be target output information indicating that the test result of the target job is to be output into the target input interface through the target link, for example, the target output information may be the aforementioned httpsink.

For example, in unit testing, the real-time computing service may replace the source data source with file test data, replace the sink data source with httpsink, and save the data into the real-time computing service through httpsink.

Through the embodiment, the data source information of the output data source in the standard work flow is replaced by the default output information, so that the accuracy of writing the test result can be ensured.

and S71, sending target display information for describing the target test result to the target terminal equipment so as to display the target test result on the target terminal equipment.

After the target test result is obtained, the real-time computing service may send the target display information to the target terminal device to display the target test result on the target terminal device. The description has already been made in the foregoing embodiments, and the description is not repeated here.

The above-described job testing method is described below with reference to an alternative example. Considering that the user generally only cares about the processing logic in the middle of the job, in this example, a text file is used to replace the source data source (acting as the aforementioned target data source, the first data source), the result of the processing operator is sent to the real-time computing service through http and stored, and finally the result is displayed to the user.

As shown in fig. 5, the flow of the job testing method in this alternative example may include the following steps:

step S502, a user drags an operator in a visual interface of the real-time computing platform and connects the operator with the operator, operation logic is built, unit test is clicked, and a unit test page is popped up.

Step S504, on the unit test page, the user fills in which source data source is replaced by which file test data, and selects the rate of reading data, running resources, and the like.

In step S506, the user clicks the "run" button in the unit test page to trigger the job running.

And step S508, replacing the source data source with file test data, replacing the sink data source with httpsink by the real-time computing service, and storing the data in the real-time computing service through the httpsink.

The real-time computing service can obtain the operation implementation diagram shown in fig. 6 after replacing the source data source with the file test data and replacing the sink data source with the httpsink. In fig. 6, the input of the job is text test data, which is provided by the user. The processing logic of the job comprises a plurality of responsible sql processing, and the test result of the job is saved in the real-time computing service through httpsink.

In step S510, the real-time computing service displays the test result of the job to the user through the development interface (acting as the same target display page, e.g., the user debugging page) of the user.

By the method, the user can visually see the result of the operation only by providing one sample data, and the logic of the processing operator is more conveniently verified.

It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.

Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (e.g., a ROM (Read-Only Memory)/RAM (Random Access Memory), a magnetic disk, an optical disk) and includes several instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the methods according to the embodiments of the present application.

According to another aspect of the embodiments of the present application, there is also provided a job testing apparatus for implementing the job testing method. Fig. 7 is a block diagram of an alternative job testing apparatus according to an embodiment of the present application, and as shown in fig. 7, the apparatus may include:

a detecting unit 702, configured to detect a trigger operation performed on a target visualization interface, where the trigger operation is used to trigger a test on a target job corresponding to a target job flow in the target visualization interface, and the target job is a job that processes a target data source using a target processing operator;

a selecting unit 704, connected to the detecting unit 702, configured to select, in response to the trigger operation, target test data corresponding to the target data source, where the target test data is used to replace the target data source;

and the first sending unit 706 is connected to the selecting unit 704, and is configured to send the target identification information of the target job flow and the target test data to the target server, so that the target server processes the target test data by using a target processing operator to obtain a target test result of the target job.

It should be noted that the detecting unit 702 in this embodiment may be configured to execute the step S202, the selecting unit 704 in this embodiment may be configured to execute the step S204, and the first sending unit 706 in this embodiment may be configured to execute the step S206.

Detecting a trigger operation executed on a target visual interface through the module, wherein the trigger operation is used for triggering the test of a target job corresponding to a target job flow in the target visual interface, and the target job is a job processed on a target data source by using a target processing operator; responding to the trigger operation, and selecting target test data corresponding to the target data source, wherein the target test data is used for replacing the target data source; the target operation process and the target identification information of the target test data are sent to the target server, so that the target server processes the target test data by using a target processing operator to obtain a target test result of the target operation, the problems of low test efficiency and poor user experience caused by complex test process in an operation test mode in the related technology are solved, the operation running rapidity is improved, the operation test efficiency is improved, and the user experience is also improved.

As an alternative embodiment, the selecting unit 704 includes:

and the selecting module is used for selecting the target test data for replacing the target data source through a target configuration page, wherein the target configuration page is a page which is popped up after the trigger operation is detected and is used for configuring the test parameters of the target operation.

As an alternative embodiment, the apparatus further comprises:

the configuration unit is used for selecting target test parameters of the target operation through the target configuration page after detecting the triggering operation executed on the target visual interface, wherein the target test parameters include at least one of the following parameters: the read rate of the target test data, and the computational resources to run the target job.

As an alternative embodiment, the apparatus further comprises:

the first receiving unit is used for receiving target display information sent by a target server after target identification information of a target operation process and target test data is sent to the target server, wherein the target display information is used for describing a target test result;

and the display unit is used for displaying the target display information through the target display page so as to display the target test result on the target display page.

According to another aspect of the embodiments of the present application, there is also provided a job testing apparatus for implementing the job testing method. Fig. 8 is a block diagram of another alternative job testing apparatus according to an embodiment of the present application, and as shown in fig. 8, the apparatus may include:

an obtaining unit 802, configured to obtain a target job flow of a target job and target identification information of target test data, where the target job is a job for processing a first data source by using a target processing operator, and the target test data is used to replace the first data source;

a constructing unit 804, connected to the obtaining unit 802, configured to construct a target job by using the target job flow and the target identification information, where a first data source in the target job is replaced with target test data;

and the testing unit 806 is connected to the constructing unit 804, and is configured to test the target job through the target computing engine to obtain a target test result of the target job, where the target test result is obtained by processing the target test data using a target processing operator.

It should be noted that the obtaining unit 802 in this embodiment may be configured to execute the step S402, the constructing unit 804 in this embodiment may be configured to execute the step S404, and the testing unit 806 in this embodiment may be configured to execute the step S406.

Acquiring a target operation flow of target operation and target identification information of target test data through the modules, wherein the target operation is the operation of processing a first data source by using a target processing operator, and the target test data is used for replacing the first data source; constructing a target operation by using the target operation flow and the target identification information, wherein a first data source in the target operation is replaced by target test data; the target operation is tested through the target computing engine, and a target test result of the target operation is obtained, wherein the target test result is obtained after target test data are processed through a target processing operator, so that the problems of low test efficiency and poor user experience caused by complex test process in an operation test mode in the related technology are solved, the operation running rapidity is improved, the operation test efficiency is improved, and the user experience is also improved.

As an alternative embodiment, the apparatus further comprises:

the second receiving unit is used for receiving a target test result input by the target calculation engine through a target link through the target input interface after the target calculation engine tests the target operation to obtain the target test result of the target operation;

and the storage unit is used for storing the target test result.

As an alternative embodiment, the apparatus further comprises:

the first replacing unit is used for replacing the data source information of the first data source in the target operation flow with the target identification information before the target operation is constructed by using the target operation flow and the target identification information;

the second replacing unit is used for replacing the data source information of the second data source in the target operation flow with the target output information under the condition that the second data source of the target operation is not associated with the registered data source, so as to obtain an updated target operation flow;

As an alternative embodiment, the apparatus further comprises:

and the second sending unit is used for sending target display information for describing the target test result to the target terminal equipment after the target operation is tested by the target computing engine to obtain the target test result of the target operation, so that the target test result is displayed on the target terminal equipment.

It should be noted here that the modules described above are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the disclosure of the above embodiments. It should be noted that the modules described above as a part of the apparatus may be operated in a hardware environment as shown in fig. 1, and may be implemented by software, or may be implemented by hardware, where the hardware environment includes a network environment.

According to another aspect of the embodiments of the present application, there is also provided an electronic device for implementing the above job testing method, where the electronic device may be a server, a terminal, or a combination thereof.

Fig. 9 is a block diagram of an alternative electronic device according to an embodiment of the present application, as shown in fig. 9, including a processor 902, a communication interface 904, a memory 906, and a communication bus 908, where the processor 902, the communication interface 904, and the memory 906 communicate with each other via the communication bus 908, where,

a memory 906 for storing a computer program;

the processor 902, when executing the computer program stored in the memory 906, implements the following steps:

s1, detecting a trigger operation executed on the target visual interface, wherein the trigger operation is used for triggering the test of a target job corresponding to a target job flow in the target visual interface, and the target job is a job processed on a target data source by using a target processing operator;

s2, responding to the trigger operation, selecting target test data corresponding to the target data source, wherein the target test data is used for replacing the target data source;

and S3, sending the target identification information of the target operation process and the target test data to the target server, so that the target server processes the target test data by using a target processing operator to obtain a target test result of the target operation.

Optionally, the processor 902, when executing the computer program stored in the memory 906, implements the following steps:

s1, acquiring a target operation flow of a target operation and target identification information of target test data, wherein the target operation is an operation for processing a first data source by using a target processing operator, and the target test data is used for replacing the first data source;

s2, constructing a target job by using the target job flow and the target identification information, wherein the first data source in the target job is replaced by target test data;

and S3, testing the target operation through the target calculation engine to obtain a target test result of the target operation, wherein the target test result is obtained after the target test data is processed by using a target processing operator.

Alternatively, the communication bus may be a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 9, but this does not indicate only one bus or one type of bus. The communication interface is used for communication between the electronic equipment and other equipment.

The memory may include RAM, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory. Alternatively, the memory may be at least one memory device located remotely from the processor.

As an example, the memory 906 may include, but is not limited to, the detection unit 702, the selection unit 704, and the first sending unit 706 in the job testing apparatus. In addition, other module units in the job testing apparatus may also be included, but are not limited to these, and are not described in detail in this example.

As another example, the memory 906 may include, but is not limited to, the obtaining unit 802, the building unit 804, and the testing unit 806 in the job testing apparatus. In addition, other module units in the job testing apparatus may also be included, but are not limited to these, and are not described in detail in this example.

The Processor may be a general-purpose Processor such as a CPU, an NP (Network Processor), or the like. The processor may also be a DSP (Digital Signal Processing), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other Programmable logic device, discrete Gate or transistor logic device, or discrete hardware components.

In addition, the electronic device further includes: and the display is used for displaying the target test result of the target operation, such as the target display information.

Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment is not described herein again.

It can be understood by those skilled in the art that the structure shown in fig. 9 is only an illustration, and the device implementing the above-mentioned operation testing method may be a terminal device, and the terminal device may be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a Mobile Internet Device (MID), a PAD, and the like. Fig. 9 does not limit the structure of the electronic device. For example, the electronic device may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in FIG. 9, or have a different configuration than shown in FIG. 9.

Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disk, ROM, RAM, magnetic or optical disk, and the like.

According to still another aspect of an embodiment of the present application, there is also provided a storage medium. Optionally, in this embodiment, the storage medium may be configured to execute a program code of any job testing method in this embodiment of the present application.

Optionally, in this embodiment, the storage medium may be located on at least one of a plurality of network devices in a network shown in the above embodiment.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps:

Optionally, the specific example in this embodiment may refer to the example described in the above embodiment, which is not described again in this embodiment.

Optionally, in this embodiment, the storage medium may include, but is not limited to: various media capable of storing program codes, such as a U disk, a ROM, a RAM, a removable hard disk, a magnetic disk, or an optical disk.

The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.

The integrated unit in the above embodiments, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or a part of or all or part of the technical solution contributing to the prior art may be embodied in the form of a software product stored in a storage medium, and including instructions for causing one or more computer devices (which may be personal computers, servers, network devices, or the like) to execute all or part of the steps of the method described in the embodiments of the present application.

In the above embodiments of the present application, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the several embodiments provided in the present application, it should be understood that the disclosed client may be implemented in other manners. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, and may also be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution provided in the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The foregoing is only a preferred embodiment of the present application and it should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims

1. An operation testing method, comprising:

detecting a trigger operation executed on a target visual interface, wherein the trigger operation is used for triggering a target job corresponding to a target job flow in the target visual interface to be tested, and the target job is a job processed on a target data source by using a target processing operator;

responding to the trigger operation, and selecting target test data corresponding to the target data source, wherein the target test data is used for replacing the target data source;

and sending the target operation flow and the target identification information of the target test data to a target server so that the target server processes the target test data by using the target processing operator to obtain a target test result of the target operation.

2. The method of claim 1, wherein selecting the target test data corresponding to the target data source comprises:

and selecting the target test data for replacing the target data source through a target configuration page, wherein the target configuration page is a page which is popped up after the trigger operation is detected and is used for configuring the test parameters of the target operation.

3. The method of claim 2, wherein after detecting the triggering operation performed on the target visualization interface, the method further comprises:

configuring target test parameters of the target operation through the target configuration page, wherein the target test parameters comprise at least one of the following: the reading rate of the target test data, and the computing resources for running the target operation.

4. The method of any of claims 1 to 3, wherein after sending the target workflow and the target identification information of the target test data to a target server, the method further comprises:

receiving target display information sent by the target server, wherein the target display information is used for describing the target test result;

and displaying the target display information through a target display page so as to display the target test result on the target display page.

5. An operation testing method, comprising:

acquiring a target operation flow of target operation and target identification information of target test data, wherein the target operation is the operation of processing a first data source by using a target processing operator, and the target test data is used for replacing the first data source;

constructing the target job by using the target job flow and the target identification information, wherein the first data source in the target job is replaced by the target test data;

and testing the target operation through a target computing engine to obtain a target test result of the target operation, wherein the target test result is obtained after the target test data is processed by using the target processing operator.

6. The method of claim 5, wherein after testing the target job by the target compute engine, resulting in the target test result for the target job, the method further comprises:

receiving the target test result input by the target computing engine through a target link through a target input interface;

and storing the target test result.

7. The method of claim 6, wherein prior to building the target job using the target job flow and the target identification information, the method further comprises:

replacing the data source information of the first data source in the target operation flow with the target identification information;

under the condition that a second data source of the target operation is not associated with a registered data source, replacing data source information of the second data source in the target operation flow with target output information to obtain an updated target operation flow;

the second data source is a data source into which the test result of the target job is written, and the target output information is used for indicating that the test result of the target job is output to the target input interface through the target link.

8. The method of any of claims 5 to 7, wherein after testing the target job by the target compute engine, resulting in the target test result for the target job, the method further comprises:

and sending target display information for describing the target test result to target terminal equipment so as to display the target test result on the target terminal equipment.

9. An operation testing device, comprising:

the system comprises a detection unit, a data processing unit and a data processing unit, wherein the detection unit is used for detecting a trigger operation executed on a target visual interface, the trigger operation is used for triggering the test of a target job corresponding to a target job flow in the target visual interface, and the target job is a job processed on a target data source by using a target processing operator;

the selecting unit is used for responding to the trigger operation and selecting target test data corresponding to the target data source, wherein the target test data is used for replacing the target data source;

and the first sending unit is used for sending the target operation process and the target identification information of the target test data to a target server so that the target server processes the target test data by using the target processing operator to obtain a target test result of the target operation.

10. An operation testing device, comprising:

the system comprises an acquisition unit, a storage unit and a processing unit, wherein the acquisition unit is used for acquiring a target operation flow of target operation and target identification information of target test data, the target operation is operation for processing a first data source by using a target processing operator, and the target test data is used for replacing the first data source;

a construction unit, configured to construct the target job by using the target job flow and the target identification information, where the first data source in the target job is replaced with the target test data;

and the test unit is used for testing the target operation through a target calculation engine to obtain a target test result of the target operation, wherein the target test result is obtained after the target test data is processed by using the target processing operator.

11. An electronic device comprising a processor, a communication interface, a memory and a communication bus, wherein said processor, said communication interface and said memory communicate with each other via said communication bus,

the memory for storing a computer program;

the processor for performing the method steps of any one of claims 1 to 8 by running the computer program stored on the memory.

12. A computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to carry out the method steps of any one of claims 1 to 8 when executed.