CN112527277B - Visualized calculation task arrangement method and device, electronic equipment and storage medium - Google Patents
Visualized calculation task arrangement method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112527277B CN112527277B CN202011485303.9A CN202011485303A CN112527277B CN 112527277 B CN112527277 B CN 112527277B CN 202011485303 A CN202011485303 A CN 202011485303A CN 112527277 B CN112527277 B CN 112527277B
- Authority
- CN
- China
- Prior art keywords
- view
- data service
- task
- state
- calculation task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/30—Creation or generation of source code
- G06F8/34—Graphical or visual programming
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention relates to the technical field of job scheduling, and provides a visualized calculation task scheduling method, a visualized calculation task scheduling device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a computing task, acquiring a service view according to the computing task, acquiring a physical deployment state of a data service unit, generating a first operation view according to the service view and the physical deployment state, judging whether the physical deployment state changes, and if the physical deployment state is detected to change, operating the computing task according to the first operation view; if the physical deployment state is detected to change, acquiring the changed physical deployment state, generating a second operation view according to the service view and the changed physical deployment state, and executing the calculation task according to the second operation view. The method and the device solve the technical problem that when the physical deployment state changes in the prior art, the corresponding operation layout diagram cannot be adaptively generated according to the change of the physical deployment state.
Description
Technical Field
The invention relates to the technical field of job scheduling of big data systems, in particular to a visualized calculation task scheduling method, a visualized calculation task scheduling device, electronic equipment and a storage medium.
Background
In the prior art, in the process of job scheduling, one job node in the job scheduling graph corresponds to a job instance in physical deployment one by one, when the physical deployment state of a data service unit changes, for example, when the running environment changes or the data service unit increases or decreases, the job scheduling graph also needs to be correspondingly changed, but in the prior art, the job scheduling graph cannot be adaptively changed according to the change of the physical deployment state of the data service unit, so that the complexity of operation and maintenance and the production risk are greatly increased.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a method, apparatus, electronic device, and storage medium for visual computing task orchestration to achieve visual computing task orchestration of changes in physical deployment state of an adaptive data service unit.
A first aspect of the present application provides a visualized calculation task orchestration method, the visualized calculation task orchestration method comprising:
acquiring a calculation task;
obtaining a service view according to the calculation task, wherein the service view shows the dependency relationship among sub-calculation tasks in the calculation task;
Acquiring physical deployment states of all data service units, generating a first operation view according to the service view and the physical deployment states of all the data service units, wherein the physical deployment states of the data service units comprise a calculation task state and a no calculation task state, the first operation view shows the relation between the service view and the data service units, one data service unit is in a state with a calculation task and indicates that the data service unit is executing a calculation task, and one data service unit is in a state without a calculation task and indicates that the data service unit is not executing a calculation task;
judging whether the physical deployment state of all the data service units changes;
if the physical deployment state of all the data service units is detected to be unchanged, executing the computing task according to the first operation view; if the change of the physical deployment state of at least one data service unit is detected, acquiring the physical deployment state of all the data service units, generating a second operation view according to the service view and the physical deployment state of all the data service units, and executing the calculation task according to the second operation view, wherein the second operation view displays the relation between the service view and the data service units.
Preferably, the obtaining a service view according to the computing task includes:
splitting the computing task to obtain at least one sub-computing task;
obtaining a first service view according to the dependency relationship between the at least one sub-computing task, wherein the first service view shows the dependency relationship between the at least one sub-computing task;
judging whether the first service view has defects or not;
when the first service view is defective, modifying the first service view so that the first service view is not defective;
and when the first service view is not defective, saving the first service view as the service view.
Preferably, the service view includes a rule service view or a workflow service view, the rule service view includes a next service view, the workflow service view does not include a next service view, and the next service view of the rule service view is used for displaying the processing sequence of the sub-computing tasks.
Preferably, the generating the first operation view according to the service view and the physical deployment state of all the data service units includes:
When one node of the service view is executed by one data service unit, generating the node in the first operation view as a single data service unit type node which represents a sub-calculation task of a calculation task in the service view, wherein the single data service unit type node displays the name of the sub-calculation task and the name of the one data service unit executing the sub-calculation task;
when one node of the service view is executed by a plurality of data service units, generating the node as a multi-data service unit type node in the first operation view, wherein the multi-data service unit type node displays the name of the sub-calculation task and the names of the plurality of data service units executing the sub-calculation task;
and when one node of the service view is not executed by the data service unit, generating the node as a common type node in the first operation view, wherein the common type node displays the sub-calculation task name.
Preferably, the determining whether the physical deployment status of all the data service units changes includes:
detecting whether at least one data service unit is switched from a state with a computing task to a state without a computing task or from a state without a computing task to a state with a computing task, and detecting whether the data service unit is increased or decreased;
And when detecting that the physical deployment state of the data service unit is increased or decreased, or at least one data service unit is switched from the state with the computing task to the state without the computing task, or is switched from the state without the computing task to the state with the computing task, determining that the physical deployment state of the at least one data service unit is changed.
Preferably, the determining whether the physical deployment status of all the data service units changes further includes:
detecting whether the running environment of the computing task changes, wherein the running environment comprises a test environment and a production environment;
and when detecting that the running environment changes, determining that the physical deployment state of the data service unit changes.
Preferably, performing the computing task according to the first running view includes:
judging whether the first operation view has operation conditions, when the first operation view has operation conditions, distributing a data service unit for the calculation task according to the first operation view, and executing the calculation task data by using the data service unit.
A second aspect of the present application provides a visualized calculation task orchestration device, the visualized calculation task orchestration device comprising:
The computing task acquisition module is used for acquiring computing tasks;
the business view production module is used for obtaining a business view according to the calculation task, and the business view shows the dependency relationship among sub-calculation tasks in the calculation task;
the state acquisition module is used for acquiring the physical deployment state of all the data service units, generating a first operation view according to the service view and the physical deployment state of all the data service units, wherein the physical deployment state of the data service units comprises a calculation task state and a no calculation task state, the first operation view shows the relation between the service view and the data service units, one data service unit is in a calculation task state and represents that the data service unit is executing a calculation task, and the other data service unit is in a no calculation task state and represents that the data service unit is not executing a calculation task;
the change detection module is used for judging whether the physical deployment state of all the data service units is changed or not;
the operation module is used for executing the calculation task according to the first operation view if detecting that the physical deployment state of all the data service units is not changed; if the change of the physical deployment state of at least one data service unit is detected, acquiring the physical deployment state of all the data service units, generating a second operation view according to the service view and the physical deployment state of all the data service units, and executing the calculation task according to the second operation view, wherein the second operation view displays the relation between the service view and the data service units.
A third aspect of the present application provides an electronic apparatus comprising:
a memory storing at least one instruction; a kind of electronic device with high-pressure air-conditioning system
And the processor executes the instructions stored in the memory to realize the visualized calculation task scheduling method.
A fourth aspect of the application provides a computer storage medium having stored thereon computer readable instructions which when executed by a processor implement the visualized computing task orchestration method.
In the application, a computing task is acquired, a service view is obtained according to the computing task, the service view shows the dependency relationship among sub-computing tasks in the computing task, the physical deployment state of a data service unit is acquired, a first operation view is generated according to the service view and the physical deployment state of the data service unit, the physical deployment state of the data service unit comprises a computing task state and a no-computing task state, the first operation view shows the relationship between the service view and the data service unit, whether the physical deployment state of the data service unit changes is judged, and if the physical deployment state of the data service unit is detected to change, the computing task is executed according to the first operation view; if the physical deployment state of the data service unit is detected to change, the physical deployment state of the changed data service unit is obtained, a second operation view is generated according to the service view and the physical deployment state of the changed data service unit, and the calculation task is executed according to the second operation view, wherein the second operation view shows the relation between the service view and the physical deployment state of the changed data service unit.
Drawings
FIG. 1 is a flow chart of a method of visual computing task orchestration in an embodiment of the present application.
FIG. 2 is a block diagram of a visualized computing task orchestration device according to one embodiment of the present application.
Fig. 3 is a schematic diagram of an electronic device according to an embodiment of the application.
Fig. 4 is a schematic diagram of a service view in an embodiment of the present application.
Fig. 5 is a schematic diagram of an operational view in an embodiment of the present application.
Detailed Description
In order that the above-recited objects, features and advantages of the present application will be more clearly understood, a more particular description of the application will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application, and the described embodiments are merely some, rather than all, embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the description of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.
Preferably, the visualized computing task orchestration method of the present invention is applied in one or more electronic devices. The electronic device is a device capable of automatically performing numerical calculation and/or information processing according to a preset or stored instruction, and its hardware includes, but is not limited to, a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a programmable gate array (Field-Programmable Gate Array, FPGA), a digital processor (Digital Signal Processor, DSP), an embedded device, and the like.
The electronic device may be a computing device such as a desktop computer, a notebook computer, a tablet computer, a cloud server, and the like. The device can perform man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch pad or voice control equipment and the like.
Example 1
FIG. 1 is a flow chart of a method of visualized computing task orchestration in one embodiment of the invention. The order of the steps in the flow diagrams may be changed, and some steps may be omitted, according to different needs.
Referring to fig. 1, the visualized calculation task scheduling method specifically includes the following steps:
step S11, obtaining a calculation task.
In at least one embodiment of the invention, the computing tasks are stored in a job orchestration system, which may be deployed in a distributed cluster, which may be a cluster of execution machines.
For example, the computing task may be a card-making file processing job that may be stored in a job orchestration system that may be deployed in a distributed cluster, such as in a HADOOP system, that may be a cluster of executors, such as an Internet Data Center (IDC).
And step S12, obtaining a service view according to the calculation task, wherein the service view displays the dependency relationship among sub-calculation tasks in the calculation task.
In at least one embodiment of the present invention, the obtaining a service view according to the computing task includes:
dividing the computing task to obtain at least one sub-computing task;
obtaining a first service view according to the dependency relationship between the at least one sub-computing task, wherein the first service view shows the dependency relationship between the at least one sub-computing task;
judging whether the first service view has defects or not;
when the first service view is defective, modifying the first service view so that the first service view is not defective;
and when the first service view is not defective, saving the first service view as the service view.
For example, when the computing task is a card making file processing job, obtaining the service view according to the computing task includes:
dividing the card making file processing operation into three sub-calculation tasks of an operation segmentation node, a sub-operation node and an operation summarization node;
and drawing according to the sequence of the job segmentation node, the sub-job operation node and the job summarizing node in a dragging mode to obtain a first service view of processing the card-making file.
Specifically, the defects of the first service view include, but are not limited to: the first service view is annular.
For example, when the job summary node points to the job segmentation node in the first service view processed by the card making file, the first service view processed by the card making file has a ring shape, and the defect of the service view processed by the card making file is judged, and the first service view processed by the card making file is modified. And when the first service view processed by the card making file is modified and has no defect, storing the first service view as the card making file processing service view.
In at least one embodiment of the present invention, the service view includes a rule service view or a workflow service view, the rule service view includes a next-level service view, and the workflow service view does not include a next-level service view, and the next-level service view of the rule service view is used for displaying a processing sequence of the sub-computing tasks.
Specifically, the rule service view is predefined in the system by service personnel.
For example, when a business person clicks on a rule business view, a next level business view of the rule business view may be obtained, which may be a rule business view and/or a workflow business view; when a business person clicks the workflow business view, the next stage business view is not obtained.
For example, in the card making file processing service view, the job segmentation node and the job summarization node may be workflow service views, and when a service person clicks the job segmentation node or the job summarization node, the next service view cannot be obtained; the sub-job operation node may be a regular service view, and when a service person clicks the sub-job operation node, a next service view, that is, a service view of each sub-job, may be obtained.
For another example, referring to FIG. 4, FIG. 4 is a business view of an example job. Wherein, the start of the job instance is denoted by "start"; s401, executing a first task of the job instance; s402, executing a first rule task of the job instance; the job instance ends.
Step S13, obtaining the physical deployment state of all the data service units, generating a first operation view according to the service view and the physical deployment state of all the data service units, wherein the physical deployment state of the data service units comprises a calculation task state and a no calculation task state, the first operation view displays the relation between the service view and the data service units, one data service unit is in a calculation task state and indicates that the data service unit is executing a calculation task, and the other data service unit is in a no calculation task state and indicates that the data service unit is not executing a calculation task.
In at least one embodiment of the present invention, the physical deployment state of the data service unit may be acquired at a preset time node or a preset frequency or a preset time interval.
For example, the preset time node may be 16 points, the preset frequency may be once per day, and the preset time interval may be every four hours.
In at least one embodiment of the invention, the data traffic unit may be a data traffic unit (dsu, data-service-unit) of a distributed cluster. The data traffic unit (dsu) is part of a distributed cluster, comprising a complete set of systems that can provide all services.
For example, when the distributed cluster has 2000 thousands of users, the distributed cluster is divided into 200 thousands of users, and each 200 thousands of users deploy one data service unit, 10 data service units need to be deployed, and the 10 data service units together form a distributed cluster.
In at least one embodiment of the present invention, the generating a first operation view according to the service view and the physical deployment status of all the data service units includes:
when a node of the service view is executed by a data service unit, generating the node in the first operation view as a single data service unit type node which represents a sub-calculation task of a calculation task in the service view, wherein the single data service unit type node displays the name of the sub-calculation task and the name of the data service unit executing the sub-calculation task;
when one node of the service view is executed by a plurality of data service units, generating the node as a multi-data service unit type node in the first operation view, wherein the multi-data service unit type node displays the name of the sub-calculation task and the names of the plurality of data service units executing the sub-calculation task;
And when one node of the service view is not executed by the data service unit, generating the node as a common type node in the first operation view, wherein the common type node displays the sub-calculation task name.
In other embodiments of the present invention, when a node type of the service view does not exist, the node type is defined, and a relationship between the node type and the job instance is defined.
For example, when the job instance is a card making file processing job, the generating a card making file processing operation view according to the card making file processing operation view and the physical deployment state of the data service unit includes:
the operation segmentation nodes of the card making file processing service view are not executed in the data service unit, and in the card making file processing operation view, the operation segmentation nodes correspond to common type nodes;
the sub-operation nodes of the card making file processing service view are executed by a plurality of data service units, and in the card making file processing operation view, the sub-operation nodes correspond to the type nodes of the plurality of data service units;
the operation summarizing flow of the card making file processing service view is executed by a data service unit, and in the card making file processing operation view, the operation summarizing node corresponds to a single data service unit type node;
And generating a card-making file processing operation view according to the card-making file processing operation segmentation node and the corresponding common type node, the sub-operation node and the corresponding multi-data service unit type node, and the operation summarizing node and the corresponding single-data service unit type node.
Referring to FIG. 5, FIG. 5 is an operational view taken from the service view and physical deployment state of the data service unit shown in FIG. 4, wherein a job instance start is denoted by "start"; s510, dividing the operation instance; s520, performing, by the first data service unit, a portion of the first task; s521, performing another part of the first task by the second data traffic unit; s530, executing a part of the first rule task by the first data service unit; s531 executing another part of the first rule task by the second data service unit; s540, summarizing the first rule tasks executed by the first data service unit and the second data service unit; the job instance ends.
In other embodiments of the present invention, when a computing task is invoked, if a service view of the computing task is not modified, generating an operation view according to the service view of the computing task and a physical deployment state of a corresponding data service unit; or if the service view of the computing task is modified, generating a corresponding operation view according to the modified service view of the computing task and the physical deployment state of all data service units.
In other embodiments of the present invention, in response to the received instruction, a preview run view is generated from the service view and the physical deployment status of all data service units.
Specifically, in response to a received preview instruction sent by a service person, the preview operation view is generated according to the service view and the physical deployment state of the data service unit when the instruction is sent.
And step S14, judging whether the physical deployment state of all the data service units is changed.
In at least one embodiment of the present invention, the determining whether the physical deployment status of all the data service units changes includes:
detecting whether at least one data service unit is switched from a state with a computing task to a state without a computing task or from a state without a computing task to a state with a computing task, and detecting whether the data service unit is increased or decreased;
and when detecting that the data service units are increased or decreased, or at least one data service unit is switched from the state with the computing task to the state without the computing task, or the state without the computing task is switched from the state with the computing task, determining that the physical deployment state of the at least one data service unit is changed.
In at least one embodiment of the present invention, the determining whether the physical deployment state of the data service unit changes further includes:
detecting whether the running environment of the computing task changes, wherein the running environment comprises a test environment and a production environment;
and when detecting that the running environment changes, determining that the physical deployment state of the data service unit changes.
For example, for one node job23 of the business view, if the test environment data business unit number is 2, the codes are test100, test101, respectively, and the production environment data business unit number is 3, the codes are prd100, prd101, prd102, respectively, when the running environment changes from the test environment to the production environment, the node number of the running view changes from test100$job23, test101$job23 to prd100$job23, prd101$job23, prd $102 $job23.
For another example, for one node job23 of the service view, if the number of data service units of the test environment is 2, the codes are respectively test100 and test101, the nodes of the running view are respectively test100$job23 and test101$job23, when the number of production data service units of the test environment is 3, the nodes of the expanded service view are changed to test100$job23, test101$job23 and test102, and the nodes of the running view corresponding to the expanded service view are changed to test100$job23, test101$job23 and test102$job23.
For another example, for one node job23 of the service view, if the number of data service units of the test environment is 3, the codes are respectively test100, test101 and test102, the nodes of the operation view are respectively test100$job23, test101$job23 and test102$job23, when the number of data service units of the production data of the test environment is 2, the nodes of the service view after the capacity is changed to test100 and test101, and the nodes of the operation view corresponding to the service view after the capacity is changed to test100$job23 and test101$job23.
Step S15, if the physical deployment state of all the data service units is detected not to change, executing the calculation task according to the first operation view; if the change of the physical deployment state of at least one data service unit is detected, acquiring the physical deployment state of all the data service units, generating a second operation view according to the service view and the physical deployment state of all the data service units, and executing the calculation task according to the second operation view, wherein the second operation view displays the relation between the service view and the data service units.
In at least one embodiment of the invention, performing the computing task according to the first operational view includes:
Judging whether the first operation view has operation conditions, when the first operation view has operation conditions, distributing a data service unit for the calculation task according to the first operation view, and executing the calculation task data by using the data service unit.
In at least one embodiment of the invention, performing the computing task according to the second operational view includes:
judging whether the second operation view has operation conditions, and when the second operation view has operation conditions, distributing a data service unit for the calculation task according to the second operation view, and executing the calculation task data by using the data service unit.
For example, when the computing task is the card making file processing job, judging whether the card making file processing operation view has operation conditions, and when the card making file processing operation view has operation conditions, distributing the data service unit for the computing task according to the card making file processing operation view, and executing the card making file processing job by using the data service unit.
It should be noted that, to ensure the privacy and security of the data and output results of the above-mentioned processing procedure, the data and output results of the processing procedure may be stored in a blockchain, such as the computing task, the service view, the running view, and the like.
According to the method, a service view is obtained according to a calculation task, the service view shows the dependency relationship among sub-calculation tasks in the calculation task, the physical deployment state of a data service unit is obtained, a first operation view is generated according to the service view and the physical deployment state of the data service unit, the physical deployment state of the data service unit comprises a calculation task state and a no calculation task state, the first operation view shows the relationship between the service view and the data service unit, whether the physical deployment state of the data service unit changes is judged, and if the physical deployment state of the data service unit is detected to change, the calculation task is executed according to the first operation view; if the physical deployment state of the data service unit is detected to change, acquiring the physical deployment state of the changed data service unit, generating a second operation view according to the service view and the physical deployment state of the changed data service unit, and executing the calculation task according to the second operation view, wherein the second operation view displays the relation between the service view and the physical deployment state of the changed data service unit, so that the visualized calculation task arrangement of the self-adaptive physical structure change is realized, the technical problem that the corresponding operation arrangement diagram cannot be adaptively generated according to the change of the physical deployment state of the data service unit when the physical deployment state of the data service unit is changed in the prior art is solved, and the complexity of operation and maintenance and the production risk are greatly reduced.
Example 2
Fig. 2 is a block diagram of a visualized computing task orchestration device 30 according to an embodiment of the present invention.
In some embodiments, the visualized computing task orchestration device 30 runs in an electronic device. The visualized computing task orchestration device 30 may comprise a number of functional modules consisting of program code segments. Program code for each program segment of the visual computing task orchestration device 30 may be stored in memory and executed by at least one processor to visualize computing task orchestration functions.
In this embodiment, the visualized calculation task orchestration device 30 may be divided into multiple functional modules according to the functions it performs. Referring to fig. 2, the visualized computation task orchestration device 30 may include a computation task acquisition module 301, a service view generation module 302, a state acquisition module 303, a change detection module 304, and a running module 305. The module referred to herein is a series of computer readable instructions capable of being executed by at least one processor and of performing a fixed function, stored in a memory. The functions of the modules in some embodiments will be described in detail in the following embodiments.
The calculation task acquisition module 301 acquires a calculation task.
In at least one embodiment of the invention, the computing tasks are stored in a job orchestration system, which may be deployed in a distributed cluster, which may be a cluster of execution machines.
The service view generating module 302 obtains a service view according to the computing task, where the service view shows the dependency relationship between sub-computing tasks in the computing task.
In at least one embodiment of the present invention, the service view generating module 302 obtains a service view according to the computing task includes:
dividing the computing task to obtain at least one sub-computing task;
obtaining a first service view according to the dependency relationship between the at least one sub-computing task, wherein the first service view shows the dependency relationship between the at least one sub-computing task;
judging whether the first service view has defects or not;
when the first service view is defective, modifying the first service view so that the first service view is not defective;
and when the first service view is not defective, saving the first service view as the service view.
Specifically, the defects of the first service view include, but are not limited to: the first service view is annular.
In at least one embodiment of the present invention, the service view includes a rule service view or a workflow service view, the rule service view includes a next-level service view, and the workflow service view does not include a next-level service view, and the next-level service view of the rule service view is used for displaying a processing sequence of the sub-computing tasks.
Specifically, the rule service view is predefined in the system by service personnel.
The physical deployment state obtaining module 303 of the data service unit obtains the physical deployment states of all the data service units, and generates a first operation view according to the service view and the physical deployment states of all the data service units, where the physical deployment states of the data service units include a calculation task state and a no calculation task state, the first operation view shows a relationship between the service view and the data service units, one data service unit indicates that the data service unit is executing a calculation task in a state with a calculation task, and one data service unit indicates that the data service unit is not executing a calculation task in a state without a calculation task.
In at least one embodiment of the present invention, the physical deployment state of the data service unit may be acquired at a preset time node or a preset frequency or a preset time interval.
In at least one embodiment of the invention, the data traffic unit may be a data traffic unit (dsu, data-service-unit) of a distributed cluster. The data traffic unit (dsu) is part of a distributed cluster, comprising a complete set of systems that can provide all services.
In at least one embodiment of the present invention, the state obtaining module 303 generates the first operation view according to the service view and the physical deployment state of all the data service units, where the generating includes:
when a node of the service view is executed by a data service unit, generating the node in the first operation view as a single data service unit type node which represents a sub-calculation task of a calculation task in the service view, wherein the single data service unit type node displays the name of the sub-calculation task and the name of the data service unit executing the sub-calculation task;
when one node of the service view is executed by a plurality of data service units, generating the node as a multi-data service unit type node in the first operation view, wherein the multi-data service unit type node displays the name of the sub-calculation task and the names of the plurality of data service units for executing the sub-calculation task;
And when one node of the service view is not executed by the data service unit, generating the node as a common type node in the first operation view, wherein the common type node displays the sub-calculation task name.
In other embodiments of the present invention, when a node type of the service view does not exist, the node type is defined, and a relationship between the node type and the job instance is defined.
In other embodiments of the present invention, when a computing task is invoked, if a service view of the computing task is not modified, the state acquisition module 303 generates a running view according to the service view of the computing task and a physical deployment state of a corresponding data service unit; or if the service view of the computing task is modified, the state acquisition module 303 generates a corresponding running view according to the modified service view of the computing task and the physical deployment state of the corresponding data service unit.
In other embodiments of the present invention, in response to the received instruction, a preview run view is generated from the service view and the physical deployment status of all data service units.
The change detection module 304 determines whether the physical deployment status of all the data service units has changed.
In at least one embodiment of the present invention, the determining, by the change detection module 304, whether the physical deployment status of all the data service units has changed includes:
detecting whether at least one data service unit is switched from a state with a computing task to a state without a computing task or from a state without a computing task to a state with a computing task, and detecting whether the data service unit is increased or decreased;
and when detecting that the data service units are increased or decreased, or at least one data service unit is switched from the state with the computing task to the state without the computing task, or the state without the computing task is switched from the state with the computing task, determining that the physical deployment state of the at least one data service unit is changed.
In at least one embodiment of the present invention, the determining, by the change detection module 304, whether the physical deployment status of the data service unit changes further includes:
detecting whether the running environment of the computing task changes, wherein the running environment comprises a test environment and a production environment;
And when detecting that the running environment changes, determining that the physical deployment state of the data service unit changes.
If the operation module 305 detects that the physical deployment states of all the data service units are not changed, executing the computing task according to the first operation view; if the change of the physical deployment state of at least one data service unit is detected, acquiring the physical deployment state of all the data service units, generating a second operation view according to the service view and the physical deployment state of all the data service units, and executing the calculation task according to the second operation view, wherein the second operation view displays the relation between the service view and the data service units.
In at least one embodiment of the invention, performing the computing task according to the first operational view includes:
judging whether the first operation view has operation conditions, when the first operation view has operation conditions, distributing a data service unit for the calculation task according to the first operation view, and executing the calculation task data by using the data service unit.
In at least one embodiment of the invention, performing the computing task according to the second operational view includes:
judging whether the second operation view has operation conditions, and when the second operation view has operation conditions, distributing a data service unit for the calculation task according to the second operation view, and executing the calculation task data by using the data service unit.
It should be noted that, to ensure the privacy and security of the data and output results of the above-mentioned processing procedure, the data and output results of the processing procedure may be stored in a blockchain, such as the computing task, the service view, the running view, and the like.
According to the method, a service view is obtained according to a calculation task, the service view shows the dependency relationship among sub-calculation tasks in the calculation task, the physical deployment state of a data service unit is obtained, a first operation view is generated according to the service view and the physical deployment state of the data service unit, the physical deployment state of the data service unit comprises a calculation task state and a no calculation task state, the first operation view shows the relationship between the service view and the data service unit, whether the physical deployment state of the data service unit changes is judged, and if the physical deployment state of the data service unit is detected to change, the calculation task is executed according to the first operation view; if the physical deployment state of the data service unit is detected to change, acquiring the physical deployment state of the changed data service unit, generating a second operation view according to the service view and the physical deployment state of the changed data service unit, and executing the calculation task according to the second operation view, wherein the second operation view displays the relation between the service view and the physical deployment state of the changed data service unit, so that the visualized calculation task arrangement of the self-adaptive physical structure change is realized, the technical problem that the corresponding operation arrangement diagram cannot be adaptively generated according to the change of the physical deployment state of the data service unit when the physical deployment state of the data service unit is changed in the prior art is solved, and the complexity of operation and maintenance and the production risk are greatly reduced.
Example 3
Fig. 3 is a schematic diagram of an electronic device 6 according to an embodiment of the invention.
The electronic device 6 comprises a memory 61, a processor 62 and computer readable instructions stored in the memory 61 and executable on the processor 62. The steps of the embodiment of the visualized calculation task orchestration method described above, such as steps S11-S15 shown in fig. 1, are implemented when the processor 62 executes the computer readable instructions. Alternatively, the processor 62, when executing the computer readable instructions, performs the functions of the modules/units of the visual computing task orchestration device embodiments described above, e.g., modules 301-305 of FIG. 2.
Illustratively, the computer readable instructions may be partitioned into one or more modules/units that are stored in the memory 61 and executed by the processor 62 to complete the present invention. The one or more modules/units may be a series of computer readable instructions capable of performing a particular function, the instruction describing the execution of the computer readable instructions at the electronic device 6. For example, the computer readable instructions may be divided into a computing task acquisition module 301, a business view generation module 302, a status acquisition module 303, a change detection module 304, and a running module 305 of fig. 2, each module having specific functions, see embodiment 2.
In this embodiment, the electronic device 6 may be a computing device such as a desktop computer, a notebook computer, a palm computer, a server, or a cloud terminal device. It will be appreciated by those skilled in the art that the schematic diagram is merely an example of the electronic device 6 and does not constitute a limitation of the electronic device 6, and may include more or less components than illustrated, or may combine certain components, or different components, e.g., the electronic device 6 may also include input-output devices, network access devices, buses, etc.
The processor 62 may be a central processing module (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor 62 may be any conventional processor or the like, the processor 62 being a control center of the electronic device 6, with various interfaces and lines connecting the various parts of the entire electronic device 6.
The memory 61 may be used to store the computer readable instructions and/or modules/units, and the processor 62 may implement the various functions of the electronic device 6 by executing or executing the computer readable instructions and/or modules/units stored in the memory 61 and invoking data stored in the memory 61. The memory 61 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data created according to the use of the electronic device 6, or the like. In addition, the memory 61 may include volatile memory, and may also include nonvolatile memory, such as a hard disk, memory, plug-in hard disk, smart Media Card (SMC), secure Digital (SD) Card, flash Card (Flash Card), at least one disk storage device, flash memory device, or other storage device.
The modules/units integrated by the electronic device 6 may be stored in a computer readable storage medium if implemented in the form of software functional modules and sold or used as a stand alone product. Based on such understanding, the present invention may also be implemented by implementing all or part of the above-described embodiment method flow, or by instructing the relevant hardware by means of a computer readable instruction, where the computer readable instruction may be stored in a computer readable storage medium, where the computer readable instruction, when executed by a processor, may implement the steps of each of the above-described method embodiments. Wherein the computer readable instructions comprise computer readable instruction code which may be in the form of source code, object code, executable files, or in some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer readable instruction code, a recording medium, a USB flash disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), and so forth.
The blockchain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm and the like. The Blockchain (Blockchain), which is essentially a decentralised database, is a string of data blocks that are generated by cryptographic means in association, each data block containing a batch of information of network transactions for verifying the validity of the information (anti-counterfeiting) and generating the next block. The blockchain may include a blockchain underlying platform, a platform product services layer, an application services layer, and the like.
In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be other manners of division when actually implemented.
In addition, each functional module in each embodiment of the present invention may be integrated in the same processing module, or each module may exist alone physically, or two or more modules may be integrated in the same module. The integrated modules may be implemented in hardware or in hardware plus software functional modules.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it is evident that the word "comprising" does not exclude other modules or steps, and that the singular does not exclude a plurality. Several of the modules set forth in the present invention may be embodied by one and the same item of hardware or software. The terms first, second, etc. are used to denote a name, but not any particular order.
Finally, it should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made to the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention.
Claims (9)
1. A method of visualized computing task orchestration, the method comprising:
acquiring a calculation task;
obtaining a service view according to the calculation task, wherein the service view shows the dependency relationship among sub-calculation tasks in the calculation task;
acquiring physical deployment states of all data service units, generating a first operation view according to the service view and the physical deployment states of all the data service units, wherein the physical deployment states of the data service units comprise a calculation task state and a no calculation task state, the first operation view shows the relation between the service view and the data service units, one data service unit is in a state with a calculation task and indicates that the data service unit is executing a calculation task, and one data service unit is in a state without a calculation task and indicates that the data service unit is not executing a calculation task;
the generating a first operation view according to the service view and the physical deployment state of all the data service units includes: generating the node in the first operation view as a single data service unit type node when one node of the service view is executed by one data service unit, the node representing a sub-calculation task of a calculation task in the service view, the single data service unit type node displaying the sub-calculation task name and the name of the one data service unit executing the sub-calculation task, generating the node in the first operation view as a multiple data service unit type node when one node of the service view is executed by a plurality of data service units, the multiple data service unit type node displaying the sub-calculation task name and the names of the plurality of data service units executing the sub-calculation task, and generating the node in the first operation view as a common type node displaying the sub-calculation task name when one node of the service view is not executed by a data service unit;
Judging whether the physical deployment state of all the data service units changes;
if the physical deployment state of all the data service units is detected to be unchanged, executing the computing task according to the first operation view; if the change of the physical deployment state of at least one data service unit is detected, acquiring the physical deployment state of all the data service units, generating a second operation view according to the service view and the physical deployment state of all the data service units, and executing the calculation task according to the second operation view, wherein the second operation view displays the relation between the service view and the data service units.
2. The visualized computing task orchestration method of claim 1, wherein the obtaining a business view from the computing task comprises:
splitting the computing task to obtain at least one sub-computing task;
obtaining a first service view according to the dependency relationship between the at least one sub-computing task, wherein the first service view shows the dependency relationship between the at least one sub-computing task;
Judging whether the first service view has defects or not;
when the first service view is defective, modifying the first service view so that the first service view is not defective;
and when the first service view is not defective, saving the first service view as the service view.
3. A visualized computing task orchestration method according to claim 1 or 2, wherein the business view comprises a rule business view or a workflow business view, the rule business view comprising a next level business view, the workflow business view not comprising a next level business view, the next level business view of the rule business view being used to demonstrate the processing order of the sub-computing tasks.
4. The method of claim 1, wherein determining whether the physical deployment status of all of the data service units has changed comprises:
detecting whether at least one data service unit is switched from a state with a computing task to a state without a computing task or from a state without a computing task to a state with a computing task, and detecting whether the data service unit is increased or decreased;
And when detecting that the physical deployment state of the data service unit is increased or decreased, or at least one data service unit is switched from the state with the computing task to the state without the computing task, or is switched from the state without the computing task to the state with the computing task, determining that the physical deployment state of the at least one data service unit is changed.
5. The method of claim 1, wherein determining whether the physical deployment status of all of the data service units has changed further comprises:
detecting whether the running environment of the computing task changes, wherein the running environment comprises a test environment and a production environment;
and when detecting that the running environment changes, determining that the physical deployment state of the data service unit changes.
6. The visualized computing task orchestration method according to claim 1, wherein performing the computing task according to the first run view comprises:
judging whether the first operation view has operation conditions, when the first operation view has operation conditions, distributing a data service unit for the calculation task according to the first operation view, and executing the calculation task data by using the data service unit.
7. A visualized calculation task orchestration device for implementing the visualized calculation task orchestration method according to any one of claims 1 to 6, wherein the visualized calculation task orchestration device comprises:
the computing task acquisition module is used for acquiring computing tasks;
the business view production module is used for obtaining a business view according to the calculation task, and the business view shows the dependency relationship among sub-calculation tasks in the calculation task;
the state acquisition module is used for acquiring the physical deployment state of all the data service units, generating a first operation view according to the service view and the physical deployment state of all the data service units, wherein the physical deployment state of the data service units comprises a calculation task state and a no calculation task state, the first operation view shows the relation between the service view and the data service units, one data service unit is in a calculation task state and represents that the data service unit is executing a calculation task, and the other data service unit is in a no calculation task state and represents that the data service unit is not executing a calculation task;
The change detection module is used for judging whether the physical deployment state of all the data service units is changed or not;
the operation module is used for executing the calculation task according to the first operation view if detecting that the physical deployment state of all the data service units is not changed; if the change of the physical deployment state of at least one data service unit is detected, acquiring the physical deployment state of all the data service units, generating a second operation view according to the service view and the physical deployment state of all the data service units, and executing the calculation task according to the second operation view, wherein the second operation view displays the relation between the service view and the data service units.
8. An electronic device, the electronic device comprising:
a memory storing at least one instruction; a kind of electronic device with high-pressure air-conditioning system
A processor executing instructions stored in the memory to implement the visualized computing task orchestration method according to any one of claims 1 to 6.
9. A computer storage medium having stored thereon computer readable instructions which when executed by a processor implement the visualized computing task orchestration method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011485303.9A CN112527277B (en) | 2020-12-16 | 2020-12-16 | Visualized calculation task arrangement method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011485303.9A CN112527277B (en) | 2020-12-16 | 2020-12-16 | Visualized calculation task arrangement method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112527277A CN112527277A (en) | 2021-03-19 |
| CN112527277B true CN112527277B (en) | 2023-08-18 |
Family
ID=75000596
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011485303.9A Active CN112527277B (en) | 2020-12-16 | 2020-12-16 | Visualized calculation task arrangement method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112527277B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108595610A (en) * | 2018-04-20 | 2018-09-28 | 中国计量大学 | A kind of electri forklift operating mode visualization system based on Internet of Things |
| CN109857377A (en) * | 2019-01-07 | 2019-06-07 | 平安科技（深圳）有限公司 | A kind of API method of combination and device |
| CN111857928A (en) * | 2020-07-23 | 2020-10-30 | 北京达佳互联信息技术有限公司 | Page task access method, device and system, electronic equipment and storage medium |
| CN112035238A (en) * | 2020-09-11 | 2020-12-04 | 曙光信息产业(北京)有限公司 | Task scheduling processing method and device, cluster system and readable storage medium |
-
2020
- 2020-12-16 CN CN202011485303.9A patent/CN112527277B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108595610A (en) * | 2018-04-20 | 2018-09-28 | 中国计量大学 | A kind of electri forklift operating mode visualization system based on Internet of Things |
| CN109857377A (en) * | 2019-01-07 | 2019-06-07 | 平安科技（深圳）有限公司 | A kind of API method of combination and device |
| CN111857928A (en) * | 2020-07-23 | 2020-10-30 | 北京达佳互联信息技术有限公司 | Page task access method, device and system, electronic equipment and storage medium |
| CN112035238A (en) * | 2020-09-11 | 2020-12-04 | 曙光信息产业(北京)有限公司 | Task scheduling processing method and device, cluster system and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112527277A (en) | 2021-03-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11704618B2 (en) | Application mapping and alerting based on data dependencies | |
| US8606659B2 (en) | Identification of discrepancies in actual and expected inventories in computing environment having multiple provisioning orchestration server pool boundaries | |
| US11652767B2 (en) | Systems and methods for improved processing of message queries | |
| CN109299169B (en) | Data visualization method, system, terminal and computer readable storage medium | |
| CN112559535B (en) | Multithreading-based asynchronous task processing method, device, equipment and medium | |
| CN112148700A (en) | Log data processing method and device, computer equipment and storage medium | |
| JP6963062B1 (en) | Information processing device and information processing method | |
| CN112631751A (en) | Task scheduling method and device, computer equipment and storage medium | |
| CN110597719B (en) | Image clustering method, device and medium for adaptation test | |
| US7925755B2 (en) | Peer to peer resource negotiation and coordination to satisfy a service level objective | |
| CN112817562A (en) | Service processing method and device | |
| US20170344926A1 (en) | System impact based logging with resource finding remediation | |
| US20140181701A1 (en) | Model-Based User Interface | |
| CN112527277B (en) | Visualized calculation task arrangement method and device, electronic equipment and storage medium | |
| CN112364005A (en) | Data synchronization method and device, computer equipment and storage medium | |
| CN112306648A (en) | Transaction calling method and device, electronic equipment and storage medium | |
| US8036921B2 (en) | System and method for optimization process repeatability in an on-demand computing environment | |
| CN115082249A (en) | Vehicle insurance pricing model testing method and device, server and storage medium | |
| US20140172955A1 (en) | Distributed mobile enterprise application platform | |
| CN112422331A (en) | Operation and maintenance operation node monitoring method and related equipment | |
| US11392424B2 (en) | Method and device for aiding decision-making for the allocation of computing means on a high performance computing infrastructure | |
| CN113342472A (en) | Micro-service cluster creating method and device, electronic equipment and readable storage medium | |
| CN112308440A (en) | Work order processing method and device, computer equipment and computer readable storage medium | |
| JP2023538941A (en) | Intelligent backup and restore of containerized environments | |
| CN113553180A (en) | Container scheduling method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |