CN112783578A

CN112783578A - Method, device and equipment for starting task flow and storage medium

Info

Publication number: CN112783578A
Application number: CN202110277386.0A
Authority: CN
Inventors: 温贵涛; 许文彬
Original assignee: Tianmian Information Technology Shenzhen Co ltd
Current assignee: Tianmian Information Technology Shenzhen Co ltd
Priority date: 2021-03-15
Filing date: 2021-03-15
Publication date: 2021-05-11

Abstract

The invention relates to the field of data processing, and provides a method, a device, equipment and a storage medium for starting a task flow. The method comprises the following steps: the method comprises the steps of establishing a mapping relation between a task flow and configuration information of the task flow according to a preset template engine, receiving a task flow starting request sent by a user, obtaining relevant parameters carried in the request, generating the task flow of a to-be-processed state corresponding to the request based on the relevant parameters, reading the relevant parameters of the task flow of the to-be-processed state by using the template engine, splicing the relevant parameters and preset default configuration information into target configuration information, generating a workflow of the task flow of the to-be-processed state according to the target configuration information and the mapping relation, and starting a new process to execute a plurality of task flows after a starter of the template engine receives the information of the plurality of task flows corresponding to the workflow. The invention can decouple the initiating process and the running process of the task through the template engine.

Description

Method, device and equipment for starting task flow and storage medium

Technical Field

The present invention relates to the field of data processing, and in particular, to a method, an apparatus, a device, and a storage medium for starting a task flow.

Background

A plurality of task flows exist in the modeling of the federal learning, the task flows have no fixed regulation, the task flows can be freely combined according to different data or requirements, and the relevance between the starting flow of the task flow and the running flow of the task flow is strong possibly caused by the task, so that the flexibility and the expandability of the structure of the federal learning project are poor.

Disclosure of Invention

In view of the above, the present invention provides a method, an apparatus, a device and a storage medium for starting a task flow, and aims to solve the technical problem in the prior art that the relevance between the starting of the task flow and the running of the task flow is strong.

In order to achieve the above object, the present invention provides a method for starting a task flow, including:

according to a preset template engine, establishing a mapping relation between a task flow and configuration information of the task flow;

receiving a task flow starting request sent by a user, acquiring related parameters carried in the request, and generating a task flow of a to-be-processed state corresponding to the request based on the related parameters;

responding to the request, reading related parameters of the task flow of the state to be processed by using the template engine, and splicing the related parameters and preset default configuration information into target configuration information;

and generating a workflow of the task flow in the state to be processed according to the target configuration information and the mapping relation, and starting a new process to execute the plurality of task flows after a starter of the template engine receives the information of the plurality of task flows corresponding to the workflow.

Preferably, the configuration information of the task flow includes configuration information of a workflow, configuration information of a task flow, and configuration information of a method flow, the configuration information of the workflow is used to describe a flow relationship, the configuration information of the task flow includes common configuration information and component configuration information, the configuration information of the method flow includes a parameter list and a flow attribute list, one workflow includes at least one task flow, and one task flow includes at least one method flow.

Preferably, after the task flow of the to-be-processed state corresponding to the request is generated based on the relevant parameter, the method further includes:

and storing the task flow of the state to be processed to a preset storage path in a preset format for the template engine to read.

Preferably, after extracting the feature index data of the standard format data based on the extraction rule, the method further includes:

and judging whether the characteristic index data needs to be sent to a preset storage path or not according to the message header of the original data corresponding to the characteristic index data, and if so, sending the characteristic index data to the preset storage path.

Preferably, the splicing the related parameters and preset default configuration information into target configuration information includes:

and judging whether a parameter item with blank parameters exists in the related parameters, and filling the parameter of default configuration information corresponding to the parameter item into the parameter item when the parameter item with blank parameters exists.

In order to achieve the above object, the present invention further provides a task flow starting device, including:

a building module: the system comprises a task flow and a template engine, wherein the task flow is used for setting a mapping relation between the task flow and configuration information of the task flow according to the preset template engine;

a receiving module: the task flow starting device is used for receiving a task flow starting request sent by a user, acquiring related parameters carried in the request, and generating a task flow of a to-be-processed state corresponding to the request based on the related parameters;

splicing modules: the template engine is used for responding to the request, reading related parameters of the task flow of the state to be processed, and splicing the related parameters and preset default configuration information into target configuration information;

a starting module: and the workflow is used for generating the task flow of the state to be processed according to the target configuration information and the mapping relation, and after the initiator of the template engine receives the information of a plurality of task flows corresponding to the workflow, a new process is started to execute the plurality of task flows.

In order to achieve the above object, the present invention also provides an electronic device, including:

at least one processor; and the number of the first and second groups,

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores a program executable by the at least one processor, the program being executable by the at least one processor to enable the at least one processor to perform the steps of:

In order to achieve the above object, the present invention further provides a computer-readable storage medium, which stores a start-up program of a task flow, and when the start-up program of the task flow is executed by a processor, the computer-readable storage medium implements any step of the start-up method of the task flow as described above.

According to the starting method, the starting device, the starting equipment and the storage medium of the task flow, the initiating flow and the running flow of the task are decoupled through the template engine, so that the flexibility, the robustness and the like of the whole project are improved, the flexibility and the expandability are provided for the project construction of the federal study, and in the specific development process, developers do not need to know the specific business flow and only need to concentrate on the realization of the function.

Drawings

FIG. 1 is a flow chart diagram of a preferred embodiment of a method for initiating a task flow of the present invention;

FIG. 2 is a block diagram of a preferred embodiment of a starting device for task flow according to the present invention;

FIG. 3 is a diagram of an electronic device according to a preferred embodiment of the present invention;

the implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The invention provides a method for starting a task flow. Referring to fig. 1, a method flow diagram of an embodiment of a method for starting a task flow according to the present invention is shown. The method may be performed by an electronic device, which may be implemented by software and/or hardware. The starting method of the task flow comprises the following steps:

step S10: and establishing a mapping relation between the task flow and the configuration information of the task flow according to a preset template engine.

The application scenario of the scheme may be a scenario in which a certain task flow (e.g., loading a data set, aligning data, screening features, calculating feature values, modeling, etc.) needs to be started in a multi-party joint learning modeling process to describe the scheme, and it should be noted that an actual application scenario of the scheme is not limited to the scenario.

Federal learning is an encrypted distributed machine learning technique that encrypts model parameters, and participating parties can update the model parameters by exchanging the encrypted model parameters, thereby creating a virtual common model whose effect is almost equal to that of modeling by aggregating data on one machine. For example, member A has user characteristic data and member B has product characteristic data and annotation data. Two members cannot easily merge data of two parties due to data privacy and the like, and the two parties are assumed to respectively establish a task model, each task can be classified or predicted, and the tasks are approved by respective users when the data are obtained, and the key point is how to establish a high-quality model between A and B. Due to incomplete data or insufficient data (insufficient data volume to build a good model), the model at each end may not be built or the effect is not ideal, and the problem can be solved by adopting federal learning.

In the modeling of federal learning, a plurality of processes exist, the processes have no fixed regulation, and the processes can be freely combined according to different data or requirements, so that certain flexibility and extensibility are needed for starting the task process.

In this embodiment, a template engine is pre-constructed, and a mapping relationship between Task flows and configuration information of the Task flows is established, where the Task flows include a Job workflow, a Task workflow, and a Function method flow. One Job workflow includes several tasks, and one Task includes several Function method flows, where a Function method flow is the smallest unit in flow control.

Further, the configuration information of the workflow is used to describe the process relationship, for example, which tasks a certain workflow contains, which parent nodes of a certain task have, and which child nodes have.

The configuration information of the task flow comprises common configuration information and component configuration information, wherein the common configuration information is automatically generated by the system and comprises project basic configuration information, operation environment configuration information and federal member configuration information. The component configuration information is input by the system to infer life and user input, and includes input stream configuration information, output stream configuration information, component module configuration information, and component running configuration information. The configuration information of the method flow includes a parameter list and a flow attribute list.

Step S20: receiving a task flow starting request sent by a user, acquiring related parameters carried in the request, and generating a task flow of a to-be-processed state corresponding to the request based on the related parameters.

In this embodiment, when a user needs to start a certain task flow, the user may open an application program used for starting the task flow on the terminal and initiate a start request, the user may select a corresponding model on an interactive interface of the terminal and well fill in parameters that need to be customized, and after receiving the start request sent by the user, the terminal analyzes the customized parameters carried in the request acquisition request, where the relevant parameters may be parameters corresponding to a workflow, a task flow, or a method flow, and the relevant parameters may be parameters that are entered by the user when initiating the task flow request, or may be parameters that are acquired by the application program from an address specified by the request after the user initiates the task flow start request. It is understood that the task flow starting request may be actively initiated by the user through the human-computer interaction interface, or may be actively pushed to the user by the application program which starts the task flow.

In one embodiment, the method further comprises: and storing the task flow of the state to be processed to a preset storage path in a preset format for the template engine to read.

The task flow of the to-be-processed state can be stored in a storage medium in a certain format for being read by a template engine, and can be a series of conventional tasks in a modeling process, such as feature screening, feature value calculation and the like. Initiating a task refers to a series of processes after a user clicks operation, including warehousing of a workflow and warehousing of a task flow.

Step S30: and responding to the request, reading the relevant parameters of the task flow of the state to be processed by using the template engine, and splicing the relevant parameters and preset default configuration information into target configuration information.

In this embodiment, the template engine is provided with a timing monitor, which can monitor whether a task flow in a to-be-processed state exists in a preset storage path at regular time, and when the task flow in the to-be-processed state exists, the template is started to read related parameters of the task flow in the to-be-processed state, and the related parameters and default configuration information are spliced into target configuration information.

In an embodiment, the splicing the related parameters and preset default configuration information into target configuration information includes:

When the user does not select or fill in some parameters of the task flow in the to-be-processed state, namely blank parameter items exist in related parameters of the task flow in the to-be-processed state, default parameters of the system can be used.

Each task flow maintains one own default parameter, when target configuration information is spliced, the default configuration parameters of the task flow in the system are obtained through reflection, at the moment, two task parameters are available, namely the user-defined configuration parameters and the system default configuration parameters, the user-defined configuration parameters are used for covering the same nodes on the system default configuration parameters, and the assembling and splicing of the target configuration information of the task flow can be completed.

Step S40: and generating a workflow of the task flow in the state to be processed according to the target configuration information and the mapping relation, and starting a new process to execute the plurality of task flows after a starter of the template engine receives the information of the plurality of task flows corresponding to the workflow.

In this embodiment, according to the target configuration information and the mapping relationship between the task flow and the configuration information of the task flow, the workflow of the task flow in the to-be-processed state is spliced, for example:

data exists in data nodes in the configuration, namely the Task needs data input, the input of the Task comes from a different Task, namely the use _ previews in the Meta of the Task is marked as True, and the Function method for reading the data set is written into the workflow.

According to the Task name under the normal _ data _ set node, a data set which is output by the corresponding Task and is of the type normal _ data _ set can be obtained as a parameter of the Function method.

According to the model _ for _ loading under the model node, the method for generating a model is known, so that a model output method is added to the Function list, the type of the output model is the model _ for _ loading, and the name of the model is the Task name of the Task.

The workflow comprises a plurality of task flows, and the combination mode of the task flows can be specifically executed by the template engine according to the established engine rules. And after the starter of the template engine receives the information of the plurality of task flows corresponding to the workflow, starting a new process to execute the plurality of task flows.

Referring to fig. 2, a functional block diagram of the starting apparatus 100 of the task flow of the present invention is shown.

The starting device 100 of the task flow of the present invention can be installed in an electronic device. According to the implemented functions, the starting apparatus 100 of the task flow may include a building module 110, a receiving module 120, a splicing module 130, and a starting module 140. The module of the present invention, which may also be referred to as a unit, refers to a series of computer program segments that can be executed by a processor of an electronic device and that can perform a fixed function, and that are stored in a memory of the electronic device.

In the present embodiment, the functions regarding the respective modules/units are as follows:

the establishing module 110 is configured to establish a mapping relationship between a task flow and configuration information of the task flow according to a preset template engine.

The receiving module 120 is configured to receive a task flow starting request sent by a user, acquire related parameters carried in the request, and generate a task flow in a to-be-processed state corresponding to the request based on the related parameters.

In one embodiment, the receiving module is further configured to: and storing the task flow of the state to be processed to a preset storage path in a preset format for the template engine to read.

And the splicing module 130 is configured to respond to the request, read, by using the template engine, a relevant parameter of the task flow in the to-be-processed state, and splice the relevant parameter and preset default configuration information into target configuration information.

The starting module 140 is configured to generate a workflow of the task flow in the to-be-processed state according to the target configuration information and the mapping relationship, and start a new process to execute the plurality of task flows after the starter of the template engine receives information of the plurality of task flows corresponding to the workflow.

Fig. 3 is a schematic diagram of an electronic device 1 according to a preferred embodiment of the invention.

The electronic device 1 includes but is not limited to: memory 11, processor 12, display 13, and network interface 14. The electronic device 1 is connected to a network through a network interface 14 to obtain raw data. The network may be a wireless or wired network such as an Intranet (Intranet), the Internet (Internet), a Global System for Mobile communications (GSM), Wideband Code Division Multiple Access (WCDMA), a 4G network, a 5G network, Bluetooth (Bluetooth), Wi-Fi, or a communication network.

The memory 11 includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the storage 11 may be an internal storage unit of the electronic device 1, such as a hard disk or a memory of the electronic device 1. In other embodiments, the memory 11 may also be an external storage device of the electronic device 1, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like equipped with the electronic device 1. Of course, the memory 11 may also comprise both an internal memory unit and an external memory device of the electronic device 1. In this embodiment, the memory 11 is generally used for storing an operating system installed in the electronic device 1 and various types of application software, such as a program code of the start program 10 of the task flow. Further, the memory 11 may also be used to temporarily store various types of data that have been output or are to be output.

Processor 12 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 12 is typically used for controlling the overall operation of the electronic device 1, such as performing data interaction or communication related control and processing. In this embodiment, the processor 12 is configured to run the program code stored in the memory 11 or process data, for example, the program code of the start program 10 running a task flow.

The display 13 may be referred to as a display screen or display unit. In some embodiments, the display 13 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an Organic Light-Emitting Diode (OLED) touch screen, or the like. The display 13 is used for displaying information processed in the electronic device 1 and for displaying a visual work interface, e.g. displaying the results of data statistics.

The network interface 14 may optionally comprise a standard wired interface, a wireless interface (e.g. WI-FI interface), the network interface 14 typically being used for establishing a communication connection between the electronic device 1 and other electronic devices.

Fig. 3 only shows the electronic device 1 with components 11-14 and the start-up procedure 10 of the task flow, but it is to be understood that not all shown components are required to be implemented, and that more or less components may be implemented instead.

Optionally, the electronic device 1 may further comprise a user interface, the user interface may comprise a Display (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface may further comprise a standard wired interface and a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an Organic Light-Emitting Diode (OLED) touch screen, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the electronic device 1 and for displaying a visualized user interface, among other things.

The electronic device 1 may further include a Radio Frequency (RF) circuit, a sensor, an audio circuit, and the like, which are not described in detail herein.

In the above embodiment, the processor 12 may implement the following steps when executing the start program 10 of the task flow stored in the memory 11:

The storage device may be the memory 11 of the electronic device 1, or may be another storage device communicatively connected to the electronic device 1.

For detailed description of the above steps, please refer to the functional block diagram of the embodiment of the starting apparatus 100 related to the task flow in fig. 2 and the flowchart of the embodiment of the starting method related to the task flow in fig. 1.

In addition, an embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium may be non-volatile or volatile. The computer readable storage medium may be any one or any combination of hard disks, multimedia cards, SD cards, flash memory cards, SMCs, Read Only Memories (ROMs), Erasable Programmable Read Only Memories (EPROMs), portable compact disc read only memories (CD-ROMs), USB memories, etc. The computer-readable storage medium includes a storage data area and a storage program area, the storage data area stores data created according to the use of the blockchain node, the storage program area stores a start program 10 of the task flow, and when executed by the processor, the start program 10 of the task flow implements the following operations:

The specific implementation of the computer-readable storage medium of the present invention is substantially the same as the specific implementation of the above-mentioned task flow starting method, and will not be described herein again.

It should be noted that the above-mentioned numbers of the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments. And the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, apparatus, article, or method that includes the element.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention essentially or contributing to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) as described above and includes several instructions for enabling a terminal device (such as a mobile phone, a computer, an electronic device, or a network device) to execute the method according to the embodiments of the present invention.

The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

Claims

1. A method for starting a task flow, the method comprising:

2. The method for starting task flow according to claim 1, wherein the configuration information of task flow includes configuration information of workflow, configuration information of task flow and configuration information of method flow, the configuration information of workflow is used for describing flow relationship, the configuration information of task flow includes common configuration information and component configuration information, the configuration information of method flow includes parameter list and flow attribute list, wherein one workflow includes at least one task flow, and one task flow includes at least one method flow.

3. The method for starting task flow according to claim 2, wherein the common configuration information includes project basic configuration information, operation environment configuration information, and federal member configuration information, and the component configuration information includes input stream configuration information, output stream configuration information, component module configuration information, and component operation configuration information.

4. The method for starting task flow according to claim 1, wherein after the task flow of the to-be-processed state corresponding to the request is generated based on the relevant parameter, the method further comprises:

5. The method for starting a task flow according to claim 1, wherein the splicing the related parameters and preset default configuration information into target configuration information comprises:

6. An apparatus for initiating a task flow, the apparatus comprising:

7. An electronic device, characterized in that the electronic device comprises:

at least one processor; and the number of the first and second groups,

the memory stores a program executable by the at least one processor, the program being executable by the at least one processor to cause the at least one processor to perform the steps of:

8. The electronic device of claim 7, wherein the configuration information of the task flow comprises configuration information of a workflow, configuration information of a task flow and configuration information of a method flow, the configuration information of the workflow is used for describing a flow relationship, the configuration information of the task flow comprises common configuration information and component configuration information, and the configuration information of the method flow comprises a parameter list and a flow attribute list, wherein one workflow comprises at least one task flow and one task flow comprises at least one method flow.

9. The electronic device of claim 7, wherein the stitching the relevant parameters and preset default configuration information into target configuration information comprises:

10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a startup program of a task flow, which when executed by a processor, implements the steps of the startup method of a task flow according to any one of claims 1 to 5.