CN117056366A

CN117056366A - Data extraction method, device, equipment and storage medium

Info

Publication number: CN117056366A
Application number: CN202310981529.5A
Authority: CN
Inventors: 朱宽; 饶勇军
Original assignee: BOE Technology Group Co Ltd; Beijing BOE Technology Development Co Ltd; Beijing Zhongxiangying Technology Co Ltd
Current assignee: BOE Technology Group Co Ltd; Beijing BOE Technology Development Co Ltd; Beijing Zhongxiangying Technology Co Ltd
Priority date: 2023-08-04
Filing date: 2023-08-04
Publication date: 2023-11-14

Abstract

The application provides a data extraction method, a device, equipment and a storage medium, wherein the method comprises the following steps: displaying a data extraction task formed by a plurality of functional controls on a data extraction page of an application tool, wherein the plurality of functional controls are used for representing a plurality of processing modes in a data extraction flow, and connecting lines among the plurality of functional controls are used for representing the flow direction of a data flow in the data extraction flow; determining an execution mode of the data extraction task based on the operation on the data extraction page; wherein the execution mode indicates a server employed to execute the data extraction task; and responding to the data extraction instruction, and executing the data extraction task based on the server indicated by the execution mode to perform data extraction. According to the technical scheme, the data extraction task is created anytime and anywhere based on the web browser based on the keyle tool.

Description

Data extraction method, device, equipment and storage medium

Technical Field

The present application relates to the field of software development technologies, and in particular, to a data extraction method, apparatus, device, and storage medium.

Background

Kettle is a source opening tool, and different steps are assembled and spliced in a dragging mode to complete a specific data extraction task.

In the related art, the creation of the data extraction task needs to be completed by a graphics submodule Spoon of a computer installation keyle only supporting the version of the Windows system, and the keyle is provided with timing task management logic.

However, when using a database-type repository, if the desired database is installed in a server, or container, the entire drawing process of the edit canvas can be quite stuck, greatly increasing the creation time of the data extraction task. In addition, the Kettle self-contained timing task management logic starts timing after the execution of the data extraction task is completed, the interval between the two data extraction tasks is the sum of the execution time and the timing time, so that the time deviation can be caused, and the scene requirement with harsh time interval can not be met.

Disclosure of Invention

The embodiment of the application provides a data extraction method, a device, equipment and a storage medium. The technical scheme is as follows:

according to an aspect of the embodiment of the present application, there is provided a data extraction method, including:

displaying a data extraction task formed by a plurality of functional controls on a data extraction page of an application tool, wherein the plurality of functional controls are used for representing a plurality of processing modes in a data extraction flow, and connecting lines among the plurality of functional controls are used for representing the flow direction of a data flow in the data extraction flow, and the application tool comprises a keyle tool;

Determining an execution mode of the data extraction task based on the operation on the data extraction page; wherein the execution mode indicates a server employed to execute the data extraction task;

and responding to the data extraction instruction, and executing the data extraction task based on the server indicated by the execution mode to perform data extraction.

In one possible implementation manner, the data extraction page of the application tool displays a data extraction task composed of the plurality of functional controls, including:

when the dragging operation of the functional control is detected, the functional control is displayed on a working sub-page of the data extraction page; the function control comprises any one of an input sub-function control, an output sub-function control and a conversion sub-function control;

and on the working sub-page, connecting at least two functional controls based on the operation of at least two functional controls, and displaying the data extraction task.

In one possible implementation manner, after determining the execution mode of the data extraction task based on the operation on the data extraction page, the method further includes:

when the operation on the log parameters is detected, determining the log category matched with the data extraction task; the log category comprises detailed logs and row-level logs.

In one possible implementation, the execution mode includes a local mode, a server mode, and a cluster mode;

wherein the local mode instructs the adoption of a local server to perform the data extraction task; the server mode indicates that the target server is adopted to execute the data extraction task; the cluster mode indicates that the data extraction task is performed with a plurality of servers in the cluster.

In one possible implementation manner, the responding to the data extraction instruction, the server based on the execution mode instruction executes the data extraction task to perform data extraction, and the method includes:

when any executing data extraction task is detected, displaying sub-pages in the state of the data extraction page, and displaying step measurement data matched with the executing data extraction task; wherein the step metric data includes the step being performed and the progress of the step being performed.

In one possible implementation manner, the server, responsive to the data extraction instruction, performs the data extraction task based on the execution mode instruction, and performs data extraction, and further includes:

when any one of the data extraction tasks which is finished being executed is detected, sub pages are displayed in the state of the data extraction page, and log data matched with the data extraction task which is finished being executed is displayed.

In one possible implementation manner, before the determining the execution mode of the data extraction task based on the operation on the data extraction page, the method further includes:

when detecting that at least one historical data extraction task is operated on a resource library sub-page of the data extraction page, displaying composition information of each historical data extraction task on the working sub-page; the composition information comprises each function control of each historical data extraction task and the connecting line between every two function controls.

In one possible implementation, the method further includes:

displaying at least one of the historical data extraction tasks on a timing page of the application tool when an operation on a timing parameter is detected, wherein the application tool comprises a xxl-job tool;

when detecting that any one of the historical data extraction tasks is operated, combining the matched historical data extraction task and the timing parameter, and displaying the timing data extraction task; wherein the timing parameter is used to determine a matched execution start time of the timing data extraction task.

In one possible implementation, the method further includes:

In the data management page, when detecting the operation of adding the data source connection information or the server connection information, displaying the added data source connection information or the server connection information; and/or

And deleting the data source connection information or the server connection information when the operation of deleting the data source connection information or the server connection information is detected in the data management page.

According to a second aspect of an embodiment of the present application, there is provided a data extraction apparatus, the apparatus comprising:

the first display module is used for displaying a data extraction task formed by a plurality of functional controls on a data extraction page of an application tool, wherein the plurality of functional controls are used for representing a plurality of processing modes in a data extraction flow, and connecting lines among the plurality of functional controls are used for representing the flow direction of a data flow in the data extraction flow, and the application tool comprises a keyle tool;

the determining module is used for determining an execution mode of the data extraction task based on the operation of the data extraction page; wherein the execution mode indicates a server employed to execute the data extraction task;

and the data extraction module is used for responding to the data extraction instruction, executing the data extraction task based on the server indicated by the execution mode and carrying out data extraction.

In one possible implementation, the first display module includes:

the first display sub-module is used for displaying the functional control when the dragging operation of the functional control is detected, and the functional control is displayed on the working sub-page of the data extraction page; the function control comprises any one of an input sub-function control, an output sub-function control and a conversion sub-function control;

and the connection sub-module is used for connecting at least two functional controls and displaying the data extraction task on the basis of operating at least two functional controls on the working sub-page.

In one possible implementation, the apparatus further includes:

the log module is used for determining a log category matched with the data extraction task when detecting that the log parameter is operated; the log category comprises detailed logs and row-level logs.

In one possible implementation, the data extraction module further includes:

the first matching sub-module is used for displaying sub-pages in the state of the data extraction page when any executing data extraction task is detected, and displaying step measurement data matched with the executing data extraction task; wherein the step metric data includes the step being performed and the progress of the step being performed.

In one possible implementation, the data extraction module further includes:

and the second matching sub-module is used for displaying sub-pages in the state of the data extraction page and displaying log data matched with the data extraction task of which the execution is finished when any one of the data extraction tasks of which the execution is finished is detected.

In one possible implementation, the apparatus further includes:

the composition module is used for displaying the composition information of each historical data extraction task on the working sub-page when detecting that at least one historical data extraction task is operated on the resource library sub-page of the data extraction page; the composition information comprises each function control of each historical data extraction task and the connecting line between every two function controls.

In one possible implementation, the apparatus further includes:

a timing module for displaying at least one of the historical data extraction tasks on a timing page of the application tool when an operation on a timing parameter is detected, wherein the application tool comprises a xxl-job tool;

the combination module is used for combining the matched historical data extraction tasks and the timing parameters when detecting to operate any one of the historical data extraction tasks, and displaying the timing data extraction tasks; wherein the timing parameter is used to determine a matched execution start time of the timing data extraction task.

In one possible implementation, the apparatus further includes:

the adding module is used for displaying the added data source connection information or the server connection information when detecting the operation of adding the data source connection information or the server connection information on the data management page; and/or

And the deleting module is used for deleting the data source connection information or the server connection information when detecting the operation of deleting the data source connection information or the server connection information on the data management page.

According to a third aspect of embodiments of the present application, there is provided an electronic device comprising a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, the processor implementing a method of data extraction as described above when the computer program is executed.

According to a fourth aspect of embodiments of the present application, there is provided a computer readable storage medium comprising a stored computer program, wherein the computer program when run controls a device on which the computer readable storage medium is located to perform the method of data extraction as described above.

According to a fifth aspect of embodiments of the present application, there is provided a computer program product which, when executed by a processor, implements a method of data extraction as described above.

The technical scheme provided by the embodiment of the application has the beneficial effects that:

in the embodiment of the application, the user accesses the webpage of the factory operation platform based on the web browser, in the webpage, based on the triggering operation of the keyle tool, the data of the keyle tool jumped to from the webpage is extracted, the data extraction page creates the data extraction task, the data extraction task can be created anytime and anywhere based on the triggering operation of the web browser, the data extraction task is created and the execution mode for executing the data extraction task is determined, the data extraction task can be created anytime and anywhere, the execution service is deployed in the background without the support of a terminal, the clamping of the terminal is reduced, and the stability is good; in the webpage, based on triggering operation on the xxl-job tool, the timing page of the xxl-job tool jumped from the webpage is timed on the basis of timing operation on the data extraction task, so that the execution starting time and the execution duration of the data extraction task are preset or adjusted, the execution time of the data extraction task is counted more accurately, and the execution efficiency of the data extraction task is improved; the data extraction tasks share one data source, and timely update and maintenance of the data source connection information and/or the server connection information are performed anytime and anywhere based on the triggering operation of the data management page.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of an implementation environment of a data extraction method according to an embodiment of the present application;

fig. 2 is a flow chart of a data extraction method according to an embodiment of the present application;

FIG. 3 is a schematic flow chart of an example of a data extraction method according to an embodiment of the present application;

FIG. 4 is a schematic diagram of a data extraction page according to an embodiment of the present application;

FIG. 5 is a schematic structural diagram of a first configuration operation sub-interface according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a status display sub-page according to an embodiment of the present application;

FIG. 7 is a schematic diagram of another configuration of a status display sub-page according to an embodiment of the present application;

FIG. 8 is a schematic diagram of a data management page according to an embodiment of the present application;

FIG. 9 is a schematic flow chart of timing data extraction tasks according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of a data extraction device according to an embodiment of the present application;

fig. 11 is a schematic structural view of a terminal according to an embodiment of the present application;

fig. 12 is a schematic structural diagram of a server according to an embodiment of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail with reference to the accompanying drawings.

It should be noted that the terms "first," "second," and the like in the description and the claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein. The implementations described in the following exemplary examples do not represent all implementations consistent with the application. Rather, they are merely examples of apparatus and methods consistent with aspects of the application as detailed in the accompanying claims.

It should be noted that, the information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data for analysis, stored data, presented data, etc.), and signals related to the present application are all authorized by the user or are fully authorized by the parties, and the collection, use, and processing of the related data is required to comply with the relevant laws and regulations and standards of the relevant countries and regions.

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the application. Rather, they are merely examples of apparatus and methods consistent with aspects of the application as detailed in the accompanying claims.

Fig. 1 is a schematic diagram of an implementation environment of a data extraction method according to an embodiment of the present application. As shown in fig. 1, includes a terminal 101 and a server 102. A client, for example, a web browser or the like, which provides a service by the server 102 is installed in the terminal 101. The user corresponding to the terminal 101 may implement functions such as data transmission, message interaction, etc. through the client. The terminal 101 may be an electronic device such as a smart phone, a personal computer, a laptop portable computer, and a tablet computer. The server 102 may be a server, a server cluster including a plurality of servers, or a cloud processing center.

The terminal 101 and the server 102 are connected via a wired network or a wireless network.

In some embodiments, the wireless network or the wired network uses standard communication techniques and/or protocols. The network is typically the Internet, but may be any network including, but not limited to, a local area network (Local Area Network, LAN), metropolitan area network (Metropolitan Area Network, MAN), wide area network (Wide Area Network, MAN), a mobile, wireline or wireless network, a private network, or any combination of virtual private networks. In some embodiments, data exchanged over the network is represented using techniques and/or formats including HyperText Mark-up Language (HTML), extensible markup Language (Extensible Markup Language, XML), and the like. All or some of the links may also be encrypted using conventional encryption techniques such as secure socket layer (Secure Socket Layer, SSL), transport layer security (Transport Layer Security, TLS), virtual private network (Virtual Private Network, VPN), internet protocol security (Internet Protocol Security, IPsec), and the like. In other embodiments, custom and/or dedicated data communication techniques may also be used in place of or in addition to the data communication techniques described above.

Fig. 2 is a flow chart of a data extraction method according to an embodiment of the present application, as shown in fig. 2, in the embodiment of the present application, a web page of a factory operation platform is connected to an application tool, for example, the web page may be respectively linked to a keyle tool and a xxl-job tool, and an example of accessing the keyle tool and the xxl-job tool in the factory operation platform based on the web browser is described. The method comprises the following steps:

in step 201, a data extraction task composed of a plurality of functional controls is displayed on a data extraction page of an application tool, where the plurality of functional controls are used to represent a plurality of processing modes in a data extraction flow, and connection lines between the plurality of functional controls are used to represent a flow direction of a data flow in the data extraction flow, where the application tool includes a keyle tool.

Wherein data extraction is the process of extracting data from a data source. The data extraction page is a page for configuring a data extraction flow of how to extract data from a data source and which part of data is extracted, wherein each selectable processing mode in the data extraction flow is displayed on the data extraction page, and each processing mode is at least matched with one functional control, for example, the data extraction flow comprises: input data, conversion of the input data, and output of the conversion result.

In some embodiments, a user accesses a web page of a factory operation platform based on a web browser installed in a terminal, and jumps to a data extraction page of a keyle tool when detecting that the keyle tool is triggered in the web page; the terminal may have any type of operating system installed therein, for example: windows operating system, linux operating system, etc.

In step 202, determining an execution mode of the data extraction task based on the operation on the data extraction page; wherein the execution mode indicates a server employed to perform the data extraction task.

The data extraction page also provides a configuration function of an execution mode, and after the data extraction task is set based on the data extraction page, the operation of the data extraction page can determine which server is used for executing the data extraction task.

In step 203, in response to the data extraction instruction, the server based on the execution mode instruction executes the data extraction task to perform data extraction.

Wherein after determining the execution mode, a data extraction instruction is generated.

According to the embodiment of the application, a user accesses a webpage of a factory operation platform based on a web browser, when the triggering operation of a key tool is detected in the webpage, a data extraction page of the key tool jumped to from the webpage is created, and based on the triggering operation of the data extraction page, a data extraction task is created and an execution mode for executing the data extraction task is determined; in addition, since the web browser is suitable for a terminal installed with any type of operating system, the embodiment of the application can realize the task of creating data extraction across operating systems, and is not limited to a certain terminal or a certain operating system.

The embodiment shown in fig. 2 is a simplified flow chart of the embodiment of the present application, and the technical solution of the present application will be further described based on fig. 3. Fig. 3 is a schematic flow chart of an example of a data extraction method according to an embodiment of the present application, in which a web browser applied to a terminal is taken as an example, a web page of a factory operation platform is respectively connected to application tools, for example, the web page may be respectively linked to a keyle tool and a xxl-job tool, and accessing the keyle tool and the xxl-job tool in the factory operation platform based on the web browser is described. The method comprises the following steps:

in step 301, when a drag operation is detected to be performed on the functional control, displaying the functional control on a working sub-page of the data extraction page; the function controls comprise any one of an input sub-function control, an output sub-function control and a conversion sub-function control.

In some embodiments, the data extraction page includes a function sub-page, a resource library sub-page, and a work sub-page, where the function sub-page, the resource library sub-page, and the work sub-page may skip each other, for example, skip from the function sub-page to the resource library sub-page based on a trigger operation on the function sub-page, and so on.

In some embodiments, as shown in fig. 4, at least one folder for storing data extraction tasks is displayed on a sub-page of a resource library, when a triggering operation is detected on the folder, a folder list is displayed, a new control, a rename control and the like are displayed in the folder list, and when a triggering operation is detected on the new control, a sub-folder of the folder is newly created; the function sub-page comprises an input sub-function control, an output sub-function control and a conversion sub-function control, wherein the input sub-function control comprises a surface input control. The export sub-controls include a table export control, a text file export control, and an Excel export control. The conversion sub-functionality controls include a map value control, a cut string control, an add constant control, and a remove duplicate records control.

In some embodiments, when a drag operation is detected on any one of the function controls in the function sub-page, the function control is jumped from the function sub-page to the working sub-page, and the function control is displayed on the working sub-page, as shown in fig. 4, when a drag operation is detected on the table input control, the function sub-page is jumped to the working sub-page, the table input control is displayed on the working sub-page, after drag operations on the cut string control, the duplicate record removal control and the Excel control are sequentially detected, the function sub-page is jumped to the working sub-page, and the cut string control, the duplicate record removal control and the Excel control are displayed on the working sub-page.

In some embodiments, by performing a drag operation on one or more functional controls of the functional sub-page, the functional controls may jump from the functional sub-page to the working sub-page, and the functional controls performing the drag operation are displayed on the working sub-page, so that each processing mode in the data extraction task may be created without a code form.

In step 302, at the work sub-page, based on operating at least two functionality controls, the at least two functionality controls are connected and a data extraction task is displayed.

In some embodiments, in the working sub-page, when triggering operation is detected on any two functional controls, the two functional controls are connected, and a connecting line is displayed between the two functional controls. After all the required functional controls are displayed on the working sub page, connection can be added between the functional controls, or after one functional control is newly displayed on the working sub page, the newly displayed functional control is connected with any one or more functional controls existing before through a connecting line.

In step 303, determining an execution mode of the data extraction task based on the operation of the data extraction page; wherein the execution mode indicates a server employed to perform the data extraction task.

In some embodiments, in the data extraction page, when a trigger operation on the save control is detected, the data extraction task displayed on the working sub-page is saved into a resource library linked with the resource library sub-page. When triggering operation is detected to any one or more historical data extraction tasks of the sub-pages of the resource library, composition information of each historical data extraction task is displayed on the sub-pages of the work, wherein the composition information comprises conversion information, function controls, connecting lines among the function controls, names of the historical data extraction tasks and the like, and the user can intuitively know the composition information of each historical data extraction task displayed on the sub-pages of the work and the included steps. As shown in fig. 4, the name of the history data extraction task is "test2", the conversion information includes a conversion id and a conversion name (representing the name of the result file), and when a trigger operation is detected on the name of the result file, the result file is downloaded; each result file is configured with a unique conversion id for the user to distinguish between different result files or for the server to distinguish between different result files.

It should be noted that, based on the triggering operation performed on the sub-pages of the resource library of the data extraction page, the number of the historical data extraction tasks executed in parallel is determined, which is not particularly limited in the embodiment of the present application.

In some embodiments, before the step 302, the method may further include: when detecting that at least one historical data extraction task is operated on a resource library sub-page of the data extraction page, displaying composition information of each historical data extraction task on a working sub-page; the composition information comprises each function control of each historical data extraction task and a connecting line between every two function controls.

According to the embodiment of the application, before a new data extraction task is created, a user performs searching operation on the historical data extraction task in the resource library linked with the sub-page of the resource library, and if the matched historical data extraction task is found, the new data extraction task is not required to be created, so that the time is saved, and the efficiency is improved.

In some embodiments, the execution modes in step 302 include a local mode, a server mode, and a cluster mode; the local mode indicates that a local server is adopted to execute a data extraction task; the server mode indicates that a target server is adopted to execute a data extraction task; the cluster mode indicates that a plurality of servers in the cluster are employed to perform data extraction tasks.

In some embodiments, in server mode, one server is selected from a plurality of independently operating servers in a server list as the target server. In the cluster mode, a main server and a plurality of sub servers are selected from a plurality of servers in a cluster list, wherein the main server is used for controlling the operation of each sub server, namely, the main server is used for distributing tasks to each sub server and summarizing the operation results of each sub server.

In some embodiments, when the execution mode is a local mode, it indicates that the data extraction task is executed based on the local server. When the execution mode is a server mode or a cluster mode, the data extraction task is executed based on other servers. The local mode or the server mode can be selected when the data amount corresponding to the data extraction task is smaller than a preset threshold, and the cluster mode can be selected when the data amount corresponding to the data extraction task is larger than the preset threshold. It should be noted that, the specific value of the preset threshold may be preset or adjusted according to the actual requirement, and the embodiment of the present application is not limited specifically.

In some embodiments, in the data extraction page, when a trigger operation is detected on the operation control, the operation control jumps from the working sub-page to the first setting operation sub-page, and in the first setting operation sub-page, when a trigger operation is detected on any execution mode, the execution mode is determined to be the execution mode for executing the data extraction task. As shown in fig. 5, in the first setting operation sub-page, when a triggering operation is detected for the local mode, the id of the local server in the local mode is displayed in a display frame corresponding to the server id.

According to the embodiment of the application, the user is allowed to perform the triggering operation on the first setting operation sub-page based on the data volume of the data extraction task, the proper execution mode is determined, and the execution efficiency of the data extraction task is improved.

In some embodiments, after the step 302, the method may further include: when the operation on the log parameters is detected, determining the log category matched with the data extraction task; the log category comprises detailed logs and row-level logs.

In some embodiments, running the sub-page in the first setting, when detecting that a trigger operation is performed on a display frame corresponding to a log category, displaying a drop-down list corresponding to the log category, when detecting that any one of the log categories in the drop-down list is selected, determining the log category as a log category matched with the data extraction task, and displaying the log category in the display frame corresponding to the log category. Wherein a plurality of log categories are included in the drop down list, such as: a plurality of log categories such as "none", "error", "minimum", "basic", "detailed", "debug", and "line level". As shown in fig. 5, when the trigger operation for "detailed" in the drop-down list is detected, the "detailed" is displayed in the display frame corresponding to the log category.

In step 304, in response to the data extraction instruction, data extraction is performed based on the server in the execution mode performing the data extraction task.

In some embodiments, in the first setting operation sub-page, when detecting that the determination control is triggered, closing the first setting operation sub-page, and performing data extraction based on the data extraction task executed by the server in the execution mode.

In some embodiments, the step 304 may include: when any executing data extraction task is detected, displaying sub pages in the state of the data extraction page, and displaying step measurement data matched with the executing data extraction task; wherein the step metric data includes the step being performed and the progress of the step being performed.

In some embodiments, in the data extraction page, when triggering operation is detected on the display log control, the data extraction page jumps to the state display sub-page, and when triggering operation is detected on the hidden control, the state display sub-page is closed. As shown in fig. 6, when it is detected that any one of the data extraction tasks is being executed, the data extraction page is jumped to the status display sub-page, and step variable data including a plurality of parameters corresponding to each of the steps being executed such as "step name", "number of copied recording lines", "read", "write", "input", "output", "update", "reject", "activate", "error", "time", and "speed" is displayed on the status display sub-page. According to the embodiment of the application, the data contained in the step variable data is comprehensive and rich, and based on the step variable data, the user can intuitively know the current step and the conditions of each parameter of the step, so that the execution strategy is optimized.

In some embodiments, the step 304 may include: when any one data extraction task of which the execution is finished is detected, displaying the sub-page in the state of the data extraction page, and displaying log data matched with the data extraction task of which the execution is finished.

In some embodiments, as shown in FIG. 7, when it is determined that the data extraction task has been performed to end, log data is displayed on the status sub-page; the log data includes a plurality of parameters such as "time", "step name", "parameter value", and the like. According to the embodiment of the application, based on the log data, a user can intuitively know the completion condition of the data extraction task, and can also find problems from the log data and optimize the execution strategy.

In some embodiments, in the process of executing the data extraction task, multiple steps in the data extraction task can be executed in parallel, so that the execution efficiency of the data extraction task is improved. For example: in the table input step, after 10 lines of data are input (100 lines are required to be input in total), the 10 lines of data can enter the next step first, and the next step is not needed to be carried out after all the data are input, so that the execution time of the data extraction task is saved, and the execution efficiency of the data extraction task is improved.

In some embodiments, at the data management page, when an operation to add data source connection information or server connection information is detected, the added data source connection information or server connection information is displayed; and/or displaying the added data source connection information or the server connection information when the operation of adding the data source connection information or the server connection information is detected in the data management page.

In some embodiments, in steps 301-302, when the data source connection information is abnormal or does not exist in the process of creating the data extraction task, jumping from the data extraction page to a data source connection information abnormal sub-page for prompting a user, closing the data source connection information abnormal page when detecting that the user triggers the data source connection information abnormal page, and jumping from the data extraction page to the data management page when detecting that the triggering operation is performed on the resource library control in the data extraction page; the data management page comprises a data source list sub-page, a server list sub-page and a cluster list sub-page, and the data source list sub-page, the server list sub-page and the cluster list sub-page can mutually jump in pairs. In the data management page, when triggering operation is detected to be carried out on the data source list control, jumping to a sub-page of the data source list from the data management page, and when triggering operation is detected to be carried out on the newly added data source connection control, adding new data source connection information in the data source list; when triggering operation is detected to be carried out on an editing control corresponding to any one data source in a data source list sub page of the database management page, editing is carried out on the data source connection information corresponding to the data source, so that the data source connection information corresponding to the data source is recovered to be normal; when triggering operation is detected to be carried out on the deletion control corresponding to any one data source, deleting the data source connection information corresponding to the data source. As shown in fig. 8, in the data source list sub-page, a plurality of data sources and data source connection information of each data source are displayed, where the data source connection information includes a plurality of parameters such as "data source connection name", "IP address", "data source name", "domain number", "data source type", "connection mode", and "update time".

In some embodiments, in steps 303-304, when only one data extraction task is executing, executing until the data source connection information is located, acquiring the data source according to the data source connection information; when a plurality of data extraction tasks are executed in parallel, one data extraction task or a plurality of data extraction tasks are executed until the connection information of the data sources is located, matched data sources are respectively obtained, wherein the data sources obtained by different data extraction tasks can be the same or different, the fact that the plurality of data extraction tasks share one data source is achieved, and the problem that in the related art, after one data source is used by any one data extraction task, other data extraction tasks cannot use the data source is solved.

In some embodiments, in steps 303-304, if the execution mode is a server mode, in the process of executing the data extraction task, when a certain server is abnormal, for example, a certain server cannot be connected, a page is skipped from the data extraction page to a server connection information abnormal sub-page for prompting a user, when a trigger operation is detected on the server connection information abnormal sub-page, the server connection information abnormal sub-page is closed, in the data extraction page, when a trigger operation is detected on a resource library control in a resource library sub-page, the page is skipped from the data extraction page to a data management page, when a trigger operation is detected on a server list control, the page is skipped from the data management page to the server list sub-page, when a trigger operation is detected on a newly added server connection control, a new server and server connection information of the server are added in the server list, when a trigger operation is detected on an edit control corresponding to a certain server in the server list, the server connection information of the server is edited, when a trigger operation is detected on a corresponding server in the server list, and the server connection information is deleted.

In some embodiments, in steps 303-304, if the execution mode is a cluster mode, in the process of executing the data extraction task, when a certain server is abnormal, the data extraction page jumps to a server connection information abnormal sub-page for prompting a user, when triggering operation on the server connection information abnormal sub-page is detected, the server connection information abnormal sub-page is closed, in the data extraction page, when triggering operation on a resource library control linked with a resource library sub-page is detected, the data extraction page jumps to a data management page, when triggering operation on a server list control is detected, the data management page jumps to a cluster list sub-page, when triggering operation on a newly increased cluster server connection control is detected, a new server and server connection information of the server are added in the cluster list, when triggering operation on an editing control corresponding to a certain server in the cluster list is detected, the server connection information of the server is edited, and when triggering operation on deletion control corresponding to the certain server in the cluster list is detected, the server connection information is deleted.

In some real-time examples, when triggering operation of the closing control is detected on the data extraction page, the data extraction page is closed, and a webpage of the factory operation platform is displayed.

In some embodiments, from any one of the steps 301 to 304, the method can skip from the data extraction page to the data management page based on the triggering operation of the resource library control in the data extraction page, and skip from the data management page to any one of the corresponding data source list sub-page, the server list sub-page or the cluster list sub-page based on the triggering operation of any one of the data source list control, the server list control or the cluster list, thereby realizing updating and maintaining of the data source and the server at any time and any place.

In some embodiments, even if no data extraction tasks are being created or executed, the data extraction page may be entered into the data management page for timely updating and maintenance of the data source and server based on triggering operations of the repository controls of the data extraction page.

According to the embodiment of the application, a user accesses a webpage of a factory operation platform based on a web browser, jumps to a data extraction page of the factory tool from the webpage based on the triggering operation of the keyle tool in the webpage, creates a data extraction task and determines an execution mode for executing the data extraction task based on the triggering operation of the data extraction page, through the process, the data extraction task can be created anytime and anywhere, an execution service is deployed in the background without the support of a terminal, the clamping of the terminal is reduced, and the stability is good; based on the triggering operation of the resource library control in the data extraction page, the data extraction page is jumped to the data management page, and based on the triggering operation of the data management page, the data source and the server can be updated and maintained in time, so that a plurality of data extraction tasks share one data source.

After the step 301, the data extraction page may be closed based on the triggering operation of the data extraction page, the web page of the factory operation platform may be displayed, the web page may be skipped to the timing page of the xxl-job tool based on the triggering operation of the xxl-job tool in the web page, the execution start time and the execution duration of the data extraction task may be set based on the triggering operation of the timing page, so as to obtain the timing data extraction task, and then the timing data extraction task may be executed. The following describes the technical scheme of the present application further based on fig. 9. Fig. 9 is a schematic flow chart of an example of a data extraction method according to an embodiment of the present application, in which a web browser applied to a terminal is taken as an example, a web page of a factory operation platform is respectively connected to application tools, for example, the web page may be respectively linked to a keyle tool and a xxl-job tool, and accessing the keyle tool and the xxl-job tool in the factory operation platform based on the web browser is described. The method comprises the following steps:

in step 901, at least one historical data extraction task is displayed on a timing page of the application tool when an operation on a timing parameter is detected, wherein the application tool comprises a xxl-job tool.

The timing parameters comprise the execution duration and the execution starting time of the historical data extraction task.

In some embodiments, a web browser-based access to a web page of a factory operating platform, in which when a trigger operation is detected for xxl-job, a jump is made from the web page to a timing page of xxl-job tool, a history timing data extraction task (historically saved timing data extraction task) is displayed on the timing page, when a trigger operation is detected for a newly added timing data extraction task, a jump is made from the timing page to a timing parameter sub-page, and when a trigger operation is detected for a timing parameter, at least one history data extraction task is displayed.

In step 902, when detecting that any one of the historical data extraction tasks is operated, combining the matched historical data extraction task and the timing parameter, and displaying the timing data extraction task; the timing parameters are used for determining the execution starting time of the matched timing data extraction task.

In some embodiments, in the timing parameter sub-page, when triggering operation is detected on any one of the historical data extraction tasks, the timing parameter determined in the timing parameter sub-page is combined with the historical data extraction task, when triggering operation is detected on the save control, the combined timing parameter and the historical data extraction task are saved, so as to obtain the timing data extraction task, the timing parameter sub-page is closed, and the timing data extraction task is displayed in the timing page.

In step 903, in the timing page, when a trigger operation for the history data extraction task is detected, the second setting operation sub-page is skipped from the timing page, and when a trigger operation for the execution mode is detected, the execution mode for executing the history data extraction task is determined.

In some embodiments, when a trigger operation is detected on the running control on the timing page, the running control jumps from the timing page to the second setting running sub-page, and when a trigger operation is detected on any one of the local mode/server mode/cluster mode, an execution mode corresponding to the trigger operation is determined as an execution mode for executing the history extraction task.

In step 904, when a trigger operation on the determination control is detected in the second setting operation sub-page, the second setting operation sub-page is closed, and data extraction is performed based on the historical data extraction task executed by the server in the execution mode.

In some embodiments, the second setting sub-page further includes a log category control, and when a trigger operation is detected on the log category control, a log category matching the timing data extraction task is determined.

In some real-time examples, in the timing page, when the triggering operation of the closing control is detected, the timing page is closed, and the webpage of the factory operation platform is displayed.

In some embodiments, from any one of the steps 901-904, based on the triggering operation on the resource library control in the timing page, the method jumps from the timing page to the data management page, and is used for updating and maintaining the data source connection information and/or the server connection information anytime and anywhere based on the triggering operation on the database management page.

In some embodiments, even if no timing data extraction task is executing, a trigger operation can be performed on the resource library control of the timing page, and the data management page is entered from the timing page, so as to timely update and maintain the data source connection information and/or the server connection information based on the trigger operation on the database management page.

According to the embodiment of the application, the web browser is used for accessing the web page of the factory operation platform, when the triggering operation of the xxl-job tool is detected in the web page, the web page jumps to the timing page of the xxl-job tool, and the timing operation is performed on the data extraction task based on the timing page, so that the execution starting time and the execution time of the data extraction task are preset or adjusted, the next data extraction task can be executed after waiting for a period of time after the execution of the last data extraction task is finished, the execution time of the data extraction task can be counted more accurately, and the execution efficiency of the data extraction task is improved; and based on the triggering operation of the resource library control in the timing page, jumping from the timing page to the data management page, and realizing timely updating and maintenance of the data source connection information and/or the server connection information based on the triggering operation of the data management page.

Fig. 10 is a schematic structural diagram of a data extraction device 1000 according to an embodiment of the present application, where the device includes:

the first display module 1001 is configured to display, on a data extraction page of an application tool, a data extraction task that is composed of a plurality of functional controls, where the plurality of functional controls are used to represent a plurality of processing modes in a data extraction flow, and connection lines between the plurality of functional controls are used to represent a flow direction of a data flow in the data extraction flow, where the application tool includes a keyle tool;

a determining module 1002, configured to determine an execution mode of a data extraction task based on an operation on the data extraction page; wherein, the execution mode indicates a server adopted for executing the data extraction task;

the data extraction module 1003 is configured to perform data extraction based on the server indicated by the execution mode executing the data extraction task in response to the data extraction instruction.

In one possible implementation, the first display module 1001 includes:

the first display sub-module is used for displaying the functional control on the working sub-page of the data extraction page when the dragging operation of the functional control is detected; the function control comprises any one of an input sub-function control, an output sub-function control and a conversion sub-function control;

And the connection sub-module is used for connecting at least two functional controls and displaying data extraction tasks on the basis of operating the at least two functional controls on the working sub-page.

In one possible implementation, the apparatus further includes:

In one possible implementation, the execution modes include a local mode, a server mode, and a cluster mode;

the local mode indicates that a local server is adopted to execute a data extraction task; the server mode indicates that a target server is adopted to execute a data extraction task; the cluster mode indicates that a plurality of servers in the cluster are employed to perform data extraction tasks.

In one possible implementation, the data extraction module 1003 further includes:

and the second matching sub-module is used for displaying the sub-page in the state of the data extraction page when any one of the data extraction tasks which are finished being executed is detected, and displaying log data matched with the data extraction tasks which are finished being executed.

In one possible implementation, the apparatus further includes:

the composition module is used for displaying composition information of each historical data extraction task on a working sub-page when detecting that at least one historical data extraction task is operated on the sub-page of the resource library of the data extraction page; the composition information comprises each function control of each historical data extraction task and a connecting line between every two function controls.

In one possible implementation, the apparatus further includes:

a timing module for displaying at least one historical data extraction task on a timing page of the application tool when an operation on a timing parameter is detected, wherein the application tool comprises a xxl-job tool;

the combination module is used for combining the matched historical data extraction tasks and timing parameters when detecting that any one historical data extraction task is operated, and displaying the timing data extraction tasks; the timing parameters are used for determining the execution starting time of the matched timing data extraction task.

In one possible implementation, the apparatus further includes:

According to the embodiment of the application, a user accesses a webpage of a factory operation platform based on a web browser, in the webpage, based on the triggering operation of a keyle tool, a data extraction page of the keyle tool jumped to from the webpage is created based on the triggering operation of the data extraction page, and an execution mode for executing the data extraction task is determined, so that the data extraction task can be created anytime and anywhere, execution services are deployed in the background without the support of a terminal, the clamping of the terminal is reduced, and the stability is good; in the webpage, based on triggering operation on the xxl-job tool, the timing page of the xxl-job tool jumped from the webpage is timed on the basis of timing operation on the data extraction task, so that the execution starting time and the execution duration of the data extraction task are preset or adjusted, the execution time of the data extraction task is counted more accurately, and the execution efficiency of the data extraction task is improved; the data extraction tasks share one data source, and timely update and maintenance of the data source connection information and/or the server connection information are performed anytime and anywhere based on the triggering operation of the data management page.

It should be noted that: the data extraction device provided in the above embodiment only uses the division of the above functional modules to illustrate when executing the corresponding steps, and in practical application, the above functional allocation may be performed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the data extraction device and the data extraction method provided in the foregoing embodiments belong to the same concept, and specific implementation processes of the data extraction device and the data extraction method are detailed in the method embodiments and are not repeated herein.

Embodiments of the present application also provide an electronic device comprising a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, the processor implementing the method as above when executing the computer program.

Taking an electronic device as an example of a terminal, fig. 11 is a schematic structural diagram of a terminal provided in an embodiment of the present application, referring to fig. 11, a terminal 1100 may be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, motion picture expert compression standard audio plane 3), an MP4 (Moving Picture Experts Group Audio Layer IV, motion picture expert compression standard audio plane 4) player, a notebook computer, or a desktop computer. Terminal 1100 may also be referred to by other names of user devices, portable terminals, laptop terminals, desktop terminals, and the like.

Generally, the terminal 1100 includes: a processor 1101 and a memory 1102.

The processor 1101 may include one or more processing cores, such as a 4-core processor, an 11-core processor, and the like. The processor 1101 may be implemented in at least one hardware form of DSP (Digital Signal Processing ), FPGA (Field-Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array ). The processor 1101 may also include a main processor, which is a processor for processing data in an awake state, also called a CPU (Central Processing Unit ), and a coprocessor; a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 1101 may be integrated with a GPU (Graphics Processing Unit, image processor) for taking care of rendering and rendering of content that the display screen is required to display. In some embodiments, the processor 1101 may also include an AI (Artificial Intelligence ) processor for processing computing operations related to machine learning.

Memory 1102 may include one or more computer-readable storage media, which may be non-transitory. Memory 1102 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 1102 is used to store at least one program code for execution by processor 1101 to implement the processes performed by a terminal in the data extraction method provided by the method embodiments of the present application.

In some embodiments, the terminal 1100 may further optionally include: a peripheral interface 1103 and at least one peripheral. The processor 1101, memory 1102, and peripheral interface 1103 may be connected by a bus or signal lines. The individual peripheral devices may be connected to the peripheral device interface 1103 by buses, signal lines or circuit boards. Specifically, the peripheral device includes: at least one of radio frequency circuitry 1104, a display screen 1105, a camera assembly 1106, audio circuitry 1107, and a power supply 1108.

A peripheral interface 1103 may be used to connect I/O (Input/Output) related at least one peripheral device to the processor 1101 and memory 1102. In some embodiments, the processor 1101, memory 1102, and peripheral interface 1103 are integrated on the same chip or circuit board; in some other embodiments, either or both of the processor 1101, memory 1102, and peripheral interface 1103 may be implemented on a separate chip or circuit board, as embodiments of the application are not limited in this respect.

The Radio Frequency circuit 1104 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 1104 communicates with a communication network and other communication devices via electromagnetic signals. The radio frequency circuit 1104 converts an electrical signal into an electromagnetic signal for transmission, or converts a received electromagnetic signal into an electrical signal. In some embodiments, the radio frequency circuit 1104 includes: antenna systems, RF transceivers, one or more amplifiers, tuners, oscillators, digital signal processors, codec chipsets, subscriber identity module cards, and so forth. The radio frequency circuitry 1104 may communicate with other terminals via at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: metropolitan area networks, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and/or WiFi (Wireless Fidelity ) networks. In some embodiments, the radio frequency circuitry 1104 may also include NFC (Near Field Communication, short-range wireless communication) related circuitry, which is not limiting of the application.

The display screen 1105 is used to display a UI (User Interface, user page). The UI may include graphics, text, icons, video, and any combination thereof. When the display 1105 is a touch display, the display 1105 also has the ability to collect touch signals at or above the surface of the display 1105. The touch signal may be input to the processor 1101 as a control signal for processing. At this time, the display screen 1105 may also be used to provide virtual buttons and/or virtual keyboards, also referred to as soft buttons and/or soft keyboards. In some embodiments, the display 1105 may be one and disposed on the front panel of the terminal 1100; in other embodiments, the display 1105 may be at least two, respectively disposed on different surfaces of the terminal 1100 or in a folded design; in other embodiments, the display 1105 may be a flexible display disposed on a curved surface or a folded surface of the terminal 1100. Even more, the display 1105 may be arranged in a non-rectangular irregular pattern, i.e., a shaped screen. The display 1105 may be made of LCD (Liquid Crystal Display ), OLED (Organic Light-Emitting Diode) or other materials.

The camera assembly 1106 is used to capture images or video. In some embodiments, the camera assembly 1106 includes a front camera and a rear camera. Typically, the front camera is disposed on the front panel of the terminal and the rear camera is disposed on the rear surface of the terminal. In some embodiments, the at least two rear cameras are any one of a main camera, a depth camera, a wide-angle camera and a tele camera, so as to realize that the main camera and the depth camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize a panoramic shooting and Virtual Reality (VR) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 1106 may also include a flash. The flash lamp can be a single-color temperature flash lamp or a double-color temperature flash lamp. The dual-color temperature flash lamp refers to a combination of a warm light flash lamp and a cold light flash lamp, and can be used for light compensation under different color temperatures.

The audio circuit 1107 may include a microphone and a speaker. The microphone is used for collecting sound waves of users and environments, converting the sound waves into electric signals, and inputting the electric signals to the processor 1101 for processing, or inputting the electric signals to the radio frequency circuit 1104 for voice communication. For purposes of stereo acquisition or noise reduction, a plurality of microphones may be provided at different portions of the terminal 1100, respectively. The microphone may also be an array microphone or an omni-directional pickup microphone. The speaker is used to convert electrical signals from the processor 1101 or the radio frequency circuit 1104 into sound waves. The speaker may be a conventional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, not only the electric signal can be converted into a sound wave audible to humans, but also the electric signal can be converted into a sound wave inaudible to humans for ranging and other purposes. In some embodiments, the audio circuit 1107 may also include a headphone jack.

A power supply 1108 is used to power the various components in terminal 1100. The power supply 1108 may be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 1108 includes a rechargeable battery, the rechargeable battery may support wired or wireless charging. The rechargeable battery may also be used to support fast charge technology.

In some embodiments, terminal 1100 also includes one or more sensors 1109. The one or more sensors 1109 include, but are not limited to: acceleration sensor 1110, gyroscope sensor 1111, pressure sensor 1112, optical sensor 1113, and proximity sensor 1114.

The acceleration sensor 1110 may detect the magnitudes of accelerations on three coordinate axes of a coordinate system established with the terminal 1100. For example, the acceleration sensor 1110 may be used to detect components of gravitational acceleration in three coordinate axes. The processor 1101 may control the display screen 1105 to display a user page in a landscape view or a portrait view according to the gravitational acceleration signal acquired by the acceleration sensor 1110. Acceleration sensor 1110 may also be used for the acquisition of motion data of a game or user.

The gyro sensor 1111 may detect a body direction and a rotation angle of the terminal 1100, and the gyro sensor 1111 may collect a 3D motion of the user on the terminal 1100 in cooperation with the acceleration sensor 1110. The processor 1101 may implement the following functions based on the data collected by the gyro sensor 1111: motion sensing (e.g., changing UI according to a tilting operation by a user), image stabilization at shooting, game control, and inertial navigation.

Pressure sensor 1112 may be disposed on a side frame of terminal 1100 and/or on an underlying layer of display 1105. When the pressure sensor 1112 is disposed at a side frame of the terminal 1100, a grip signal of the terminal 1100 by a user may be detected, and the processor 1101 performs a left-right hand recognition or a shortcut operation according to the grip signal collected by the pressure sensor 1112. When the pressure sensor 1112 is disposed at the lower layer of the display screen 1105, the processor 1101 realizes control of the operability control on the UI page according to the pressure operation of the user on the display screen 1105. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

The optical sensor 1113 is used to collect the intensity of ambient light. In one embodiment, the processor 1101 may control the display brightness of the display screen 1105 based on the intensity of ambient light collected by the optical sensor 1113. Specifically, when the intensity of the ambient light is high, the display luminance of the display screen 1105 is turned up; when the ambient light intensity is low, the display luminance of the display screen 1105 is turned down. In another embodiment, the processor 1101 may also dynamically adjust the shooting parameters of the camera assembly 1106 based on the intensity of ambient light collected by the optical sensor 1113.

A proximity sensor 1114, also referred to as a distance sensor, is typically provided at the front panel of the terminal 1100. Proximity sensor 1114 is used to collect the distance between the user and the front of terminal 1100. In one embodiment, when the proximity sensor 1114 detects that the distance between the user and the front face of the terminal 1100 gradually decreases, the processor 1101 controls the display 1105 to switch from the bright screen state to the off screen state; when the proximity sensor 1114 detects that the distance between the user and the front surface of the terminal 1100 gradually increases, the display screen 1105 is controlled by the processor 1101 to switch from the off-screen state to the on-screen state.

Those skilled in the art will appreciate that the structure shown in fig. 11 is not limiting and that terminal 1100 may include more or fewer components than shown, or may combine certain components, or may employ a different arrangement of components.

Taking an electronic device as a server as an example, fig. 12 is a schematic structural diagram of a server according to an embodiment of the present application, where the server 1200 may have a relatively large difference due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 1201 and one or more memories 1202, where at least one computer program is stored in the one or more memories 1202, and the at least one computer program is loaded and executed by the one or more processors 1201 to implement the data extraction method described above. Of course, the server 1200 may also have a wired or wireless network interface, a keyboard, an input/output interface, etc. for performing input/output, and the server 1200 may also include other components for implementing device functions, which are not described herein.

Embodiments of the present application also provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program, wherein the computer program when run controls a device in which the computer-readable storage medium is located to perform the data extraction method as described above. Alternatively, the computer readable storage medium may be a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a Compact-Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

According to a fifth aspect of embodiments of the present application, there is provided a computer program product which when executed by a processor implements a method of data extraction as described above.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The foregoing description of the preferred embodiments of the present application is not intended to limit the application, but rather, the application is to be construed as limited to the appended claims.

Claims

1. A method of data extraction, comprising:

displaying a data extraction task formed by a plurality of functional controls on a data extraction page of an application tool, wherein the plurality of functional controls are used for representing a plurality of processing modes in a data extraction flow, and connecting lines among the plurality of functional controls are used for representing the flow direction of a data flow in the data extraction flow, and the application tool comprises a key tool;

2. The method according to claim 1, wherein displaying the data extraction task composed of the plurality of functionality controls on the data extraction page of the application tool comprises:

when the dragging operation of the functional control is detected, displaying the functional control on a working sub-page of the data extraction page; the function control comprises any one of an input sub-function control, an output sub-function control and a conversion sub-function control;

3. The method of claim 1, further comprising, after the determining the execution mode of the data extraction task based on the operation on the data extraction page:

when the operation on the log parameters is detected, determining the log category matched with the data extraction task; the log category comprises a detailed log and a row-level log.

4. The method of claim 1, wherein the execution modes include a local mode, a server mode, and a cluster mode;

wherein the local mode indicates that a local server is adopted to execute the data extraction task; the server mode indicates that the target server is adopted to execute the data extraction task; the cluster mode indicates that the data extraction task is performed with a plurality of servers in a cluster.

5. The method of claim 1, wherein the responding to the data extraction instruction, the server based on the execution mode instruction executes the data extraction task to perform data extraction, and the method comprises:

6. The method of claim 5, wherein the responding to the data extraction instruction, the server based on the execution mode instruction executes the data extraction task to perform data extraction, further comprising:

and when any data extraction task of which the execution is finished is detected, displaying a sub page in the state of the data extraction page, and displaying log data matched with the data extraction task of which the execution is finished.

7. The method of claim 1, further comprising, prior to the determining the execution mode of the data extraction task based on the operation on the data extraction page:

when operation on at least one historical data extraction task is detected on a resource library sub-page of the data extraction page, the composition information of each historical data extraction task is displayed on the working sub-page; the composition information comprises each function control of each historical data extraction task and the connecting line between every two function controls.

8. The method of claim 7, wherein the method further comprises:

when detecting that any one of the historical data extraction tasks is operated, combining the matched historical data extraction tasks with the timing parameters, and displaying the timing data extraction tasks; the timing parameters are used for determining the execution starting time of the matched timing data extraction task.

9. The method according to claim 1, wherein the method further comprises:

in a data management page, when detecting the operation of adding data source connection information or server connection information, displaying the added data source connection information or server connection information; and/or

10. A data extraction apparatus, comprising:

the determining module is used for determining an execution mode of the data extraction task based on the operation on the data extraction page; wherein the execution mode indicates a server employed to execute the data extraction task;

11. An electronic device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, the processor implementing the method of any one of claims 1 to 9 when the computer program is executed.

12. A computer readable storage medium, characterized in that the computer readable storage medium comprises a stored computer program, wherein the computer program, when run, controls a device in which the computer readable storage medium is located to perform the method according to any one of claims 1 to 9.

13. A computer program product, characterized in that the computer program product, when executed by a processor, implements the method according to any one of claims 1 to 9.