CN112749085B - Cross-cluster task publishing method and device, electronic equipment and storage medium - Google Patents
Cross-cluster task publishing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112749085B CN112749085B CN202011618326.2A CN202011618326A CN112749085B CN 112749085 B CN112749085 B CN 112749085B CN 202011618326 A CN202011618326 A CN 202011618326A CN 112749085 B CN112749085 B CN 112749085B
- Authority
- CN
- China
- Prior art keywords
- cluster
- target
- task
- version
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000012545 processing Methods 0.000 claims description 13
- 238000013507 mapping Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 5
- 238000011161 development Methods 0.000 description 6
- 238000004519 manufacturing process Methods 0.000 description 5
- 238000012360 testing method Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000013515 script Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 125000000174 L-prolyl group Chemical group [H]N1C([H])([H])C([H])([H])C([H])([H])[C@@]1([H])C(*)=O 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3664—Environments for testing or debugging software
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/70—Software maintenance or management
- G06F8/71—Version control; Configuration management
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The application provides a cross-cluster task release method, a device, electronic equipment and a storage medium, comprising the following steps: when a task release request of a target user for a target cluster is received, determining whether the cluster environment of a source cluster of the target user is consistent with the cluster environment of the target cluster; if so, judging whether the target user has the user authority of the target cluster; and if the task has the user permission, determining a target version of the task published on the target cluster according to the version information of the task carried in the task publishing request. The application determines the version information of the task issued by the target user with the target cluster user authority as the target version under the condition of ensuring that the cluster environment of the source cluster is consistent with the cluster environment of the target cluster, and issues the task of the target version in the target cluster, thereby realizing the task issue of the cross-cluster and the target version of the task issued by the user authority, and greatly improving the accuracy and efficiency of issuing the version.
Description
Technical Field
The present application relates to the field of data development technologies, and in particular, to a method and apparatus for task publishing across clusters, an electronic device, and a storage medium.
Background
With the development of society and the advancement of science and technology, in the daily development process, developers often develop data processing in a test cluster environment and then issue the data in a production environment, and when the developers issue the processed data in the production environment, all scripts need to be transferred to the production cluster again, and then relevant workflow, scheduling and other configurations are carried out again, so that labor force is increased excessively in an intangible way.
At present, most of the ways of publishing data tasks which are used in the market are divided into two parts, one part is based on offline data tasks to realize data publishing, and the other part is based on environments appointed by dock or maven compiling and the like to carry out integral software publishing, however, the task publishing ways in the two cases still have certain defects, such as the way of publishing data tasks based on offline data tasks requires a large number of unified users of a development cluster, thus the singleness of task publishing is caused, and the way of publishing the integral software based on the appointed environments of dock or maven compiling and the like is limited by the publishing environment, when the task is forcedly published across environments, the problem that the task is incompatible in different environments can occur, and the efficiency of publishing the task in the cluster environment by developers can be greatly reduced.
Disclosure of Invention
In view of the above, the present application aims to provide a cross-cluster task publishing method, apparatus, electronic device and storage medium, which determines the version information of a task carried in a task publishing request published by a target user with target cluster user rights as a target version of a publishing task and publishes the task of the target version in the target cluster under the condition of ensuring that the cluster environment of a source cluster is consistent with the cluster environment of a target cluster, thereby not only realizing cross-cluster task publishing, reducing the compatibility problem of the task in different clusters, but also realizing that the target version of the task can be published through the user rights, greatly increasing the diversity of task publishing, reducing the labor force of developers, and improving the accuracy and efficiency of publishing versions.
The application mainly comprises the following aspects:
in a first aspect, an embodiment of the present application provides a method for task publishing across clusters, where the method includes:
when a task release request of a target user for a target cluster is received, determining whether the cluster environment of a source cluster of the target user is consistent with the cluster environment of the target cluster;
if so, judging whether the target user has the user authority of the target cluster;
and if the task has the user permission, determining a target version of the task published on the target cluster according to the version information of the task carried in the task publishing request, and publishing the task of the target version in the target cluster.
In one possible implementation, it is determined whether the cluster environment of the source cluster of the target user is consistent with the cluster environment of the target cluster by:
checking whether the configuration file in the source cluster is matched with the configuration file in the target cluster, and checking whether the related component environments in the source cluster are consistent with the related component environments in the target cluster.
In a possible implementation manner, the verifying whether the configuration file in the source cluster and the configuration file in the target cluster match includes:
and if the configuration files in the source cluster and the configuration files in the target cluster meet a preset matching rule, determining that the configuration files in the source cluster and the configuration files in the target cluster are matched.
In one possible implementation manner, the determining, according to the version information of the task carried in the task release request, a target version of the task released on the target cluster, and releasing the task of the target version in the target cluster includes:
judging whether the version information exists on the target cluster according to the version information of the task carried in the task release request;
if the task version information exists, determining that the version information on the target cluster is the latest version information, determining the task version corresponding to the version information as the target version, and issuing the task of the target version in the target cluster.
In one possible implementation manner, after the task of the target version is published in the target cluster, the task publishing method further includes:
and synchronously updating metadata information in the target version into the target cluster according to the target version of the task and a preset mapping rule.
In one possible implementation manner, the task issuing method further includes:
when receiving a rollback request of a target user for a task of a specified version in a target cluster, rollback processing is carried out on the task of the specified version;
and if the rollback processing is determined to be changed from the cluster environment of the target cluster to the cluster environment of the source cluster, deleting the appointed version of the task in the target cluster and the metadata information corresponding to the appointed version.
In a second aspect, an embodiment of the present application further provides a task publishing device across clusters, where the device includes:
the determining module is used for determining whether the cluster environment of the source cluster of the target user is consistent with the cluster environment of the target cluster or not when a task release request of the target user for the target cluster is received;
the judging module is used for judging whether the target user has the user authority of the target cluster or not if the target user has the user authority of the target cluster;
and the issuing module is used for determining a target version of the task issued on the target cluster according to the version information of the task carried in the task issuing request if the user authority is provided, and issuing the task of the target version in the target cluster.
In one possible implementation, it is determined whether the cluster environment of the source cluster of the target user in the module is consistent with the cluster environment of the target cluster by:
and verifying whether the configuration file in the source cluster is consistent with the configuration file in the target cluster, and verifying whether the related component environment in the source cluster is consistent with the related component environment in the target cluster.
In a third aspect, an embodiment of the present application further provides an electronic device, including: a processor, a memory and a bus, said memory storing machine readable instructions executable by said processor, said processor and said memory communicating over the bus when the electronic device is running, said machine readable instructions when executed by said processor performing the steps of the task issuing method as described above.
In a fourth aspect, embodiments of the present application also provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the task issuing method as described above.
Compared with the task publishing method in the prior art, the task publishing method and device for the cross-cluster are characterized in that under the condition that the cluster environment of the source cluster is consistent with the cluster environment of the target cluster, version information of the task carried in the task publishing request published by the target user with the target cluster user authority is determined to be the target version of the publishing task, and the task of the target version is published in the target cluster, so that task publishing of the cross-cluster is achieved, compatibility problems of the task in different clusters are reduced, the target version of the task can be published through the user authority, diversity of task publishing is greatly improved, labor force of developers is reduced, and accuracy and efficiency of publishing the version are improved.
In order to make the above objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 shows a flow chart of a cross-environment task publishing method provided by an embodiment of the application;
FIG. 2 illustrates a flowchart of another cross-environment task publishing method provided by an embodiment of the application;
FIG. 3 is a schematic structural diagram of a task publishing device across environments according to an embodiment of the present application;
fig. 4 shows a schematic structural diagram of an electronic device according to an embodiment of the present application.
In the figure:
300-task issuing device; 310-determining a module; 320-judging module; 330-a publication module; a 340-rollback module; 350 a processing module; 400-an electronic device; 410-a processor; 420-memory; 430-bus.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. The components of the embodiments of the present application generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the application, as presented in the figures, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. Based on the embodiments of the present application, every other embodiment obtained by a person skilled in the art without making any inventive effort falls within the scope of protection of the present application.
According to research, at present, most of ways of publishing data tasks which are used in the market are divided into two parts, one part is based on offline data tasks to realize data publishing, and the other part is based on environments appointed by dock or maven compiling and the like to carry out integral software publishing, however, the task publishing ways in the two cases still have certain defects, such as a way of realizing data task publishing based on offline data tasks, a unified user with most development clusters is required, so that the singleness of task publishing is caused, and in a way of carrying out integral software publishing based on environments appointed by dock or maven compiling and the like, the published tasks are limited by publishing environments, when the task is forcedly published across environments, the problem that the tasks are incompatible in different environments occurs, and the efficiency of the developer publishing the tasks in the cluster environment is greatly reduced.
Based on the above, the embodiment of the application provides a cross-environment task release method, which determines the target version of a task released on a target cluster by ensuring the version information of the task carried in a task release request released by a target user with target cluster user authority under the condition that the cluster environment of a source cluster is consistent with the cluster environment of the target cluster, and releases the task of the target version in the target cluster, thereby realizing the task release of the cross-cluster and the target version of the task through the user authority, and improving the accuracy and efficiency of releasing the version while reducing the labor force of developers.
Referring to fig. 1, fig. 1 is a flowchart of a cross-environment task publishing method according to an embodiment of the present application. As shown in fig. 1, the task publishing method across environments provided by the embodiment of the application includes the following steps:
s101, when a task release request of a target user for a target cluster is received, determining whether the cluster environment of a source cluster of the target user is consistent with the cluster environment of the target cluster.
In this step, when a target user (i.e., a developed staff) needs to issue a task for testing on a target cluster, it is required to first send a task issue request for the target cluster to the target cluster, and determine, through the task issue request, whether a cluster environment of a source cluster of the target user is consistent with a cluster environment of the target cluster, and only on the basis that the cluster environment of the source cluster of the target user is consistent with the cluster environment of the target cluster, task issue across environments and across clusters may be performed.
Here, the cluster environment of the source cluster may be specifically a test environment used by the target user in the task publishing process, and the cluster environment of the target cluster may be specifically a production environment used by the target user in the task publishing process.
If the cluster environment of the source cluster of the target user is inconsistent with the cluster environment of the target cluster, cross-environment task release and cross-cluster task release cannot be performed, and the target user (i.e. developer) is required to recheck the cluster environments of the two clusters, so that the cluster environments of the two clusters are consistent.
Further, it is determined whether the cluster environment of the source cluster of the target user is consistent with the cluster environment of the target cluster by:
checking whether the configuration file in the source cluster is matched with the configuration file in the target cluster, and checking whether the related component environments in the source cluster are consistent with the related component environments in the target cluster.
Here, the criterion for determining whether the cluster environment of the source cluster of the target user is consistent with the cluster environment of the target cluster is composed of two parts, wherein one part is to check the configuration file in the source cluster and the configuration file in the target cluster, and the other part is to check the related component environment in the source cluster and the related component environment in the target cluster.
The configuration files in the source cluster and the configuration files in the target cluster may specifically include node types in each workflow in the two clusters, a local storage directory in the cluster, an hdfs storage directory in the cluster, and user rights of the target user of the target cluster; the determining whether the related component environments in the source cluster are consistent with the related component environments in the target cluster may be specific to whether the components owned by the target cluster and the source cluster are consistent. Such as: a determination is made as to whether an environmental component, such as hive, spark, hdfs, is normally available.
Further, the verifying whether the configuration file in the source cluster matches the configuration file in the target cluster includes:
and if the configuration files in the source cluster and the configuration files in the target cluster meet a preset matching rule, determining that the configuration files in the source cluster and the configuration files in the target cluster are matched.
Here, the preset matching rule is a key value indicating between a configuration file in the target cluster and a configuration file in the source cluster, the configuration file in the target cluster corresponds to the configuration file in the source cluster one by one through the key value, and when the configuration file corresponding to the configuration file in the source cluster exists in the configuration file corresponding to the key value in the target cluster, the configuration file in the source cluster is determined to be matched with the configuration file in the target cluster.
If the log directory of the configuration file in the source cluster is logs=/tmp/test/dir, and according to the key value, logs=/tmp/pro/dir exists in the configuration file of the target cluster, it is indicated that the configuration file in the source cluster about logs is matched with the configuration file in the target cluster, that is, the logs are matched with the configuration file in the target cluster.
S102, if so, judging whether the target user has the user authority of the target cluster.
In this step, if the configuration file in the source cluster and the configuration file in the target cluster satisfy the matching rule, and after the related component environments in the source cluster are consistent with the related component environments in the target cluster, the user permission of the target user is obtained.
And the target user is provided with a target cluster corresponding to the target user, wherein different developers or the target user has different development authorities, and the operation authorities of the target cluster corresponding to the target user are different.
Specifically, if the target user exists in the cluster environment of the source cluster represented by the test environment, it is determined whether the user right of the target user exists in the cluster environment of the target cluster represented by the production environment.
And S103, if the user authority is provided, determining a target version of the task published on the target cluster according to the version information of the task carried in the task publication request, and publishing the task of the target version in the target cluster.
In the step, if the user limit exists in the cluster environment of the target cluster, a subsequent version is selected, a target version of the task published on the target cluster is determined according to the version information of the task carried in the task publishing request, and the task of the target version is published in the target cluster.
Thus, if the user-limited rights do not exist in the cluster environment of the target cluster, the target user is required to synchronize the user-limited rights in the cluster environment of the source cluster to the cluster environment of the target cluster, and version selection is performed.
Further, the task issuing method further comprises the following steps: and when receiving a rollback request of a target user for a task of a specified version in a target cluster, performing rollback processing on the task of the specified version.
Here, when receiving a rollback request of a target user for a task of a specified version in a target cluster, the current version is returned to the specified version according to the rollback request.
And if the rollback processing is determined to be changed from the cluster environment of the target cluster to the cluster environment of the source cluster, deleting the appointed version of the task in the target cluster and the metadata information corresponding to the appointed version.
Here, when the set version of the rollback in the rollback request is required to be changed from the cluster environment of the target cluster to the cluster environment of the incoming cluster, deleting the current version of the task in the current target cluster and the metadata information corresponding to the current version, and synchronizing the set version of the task and the metadata information corresponding to the set version.
If the set version in the rollback request does not need to be changed from the cluster environment of the target cluster to the cluster environment of the source cluster, the set version is scrolled.
According to the task release method for the cross-cluster, under the condition that the cluster environment of the source cluster is consistent with the cluster environment of the target cluster, the version information of the task carried in the task release request released by the target user with the target cluster user authority is determined to be the target version of the release task, and the task of the target version is released in the target cluster, so that the task release of the cross-cluster is realized, the compatibility problem of the task in different clusters is reduced, the target version of the task can be released through the user authority, the task release diversity is greatly improved, the labor force of a developer is reduced, the accuracy and the efficiency of the release version are improved, and the release of the cross-cluster version rollback and the release of the appointed version based on the user authority are supported.
Referring to fig. 2, fig. 2 is a flowchart of a cross-environment task publishing method according to another embodiment of the present application. As shown in fig. 2, the cross-environment task publishing method provided by the embodiment of the present application includes:
s201, when a task release request of a target user for a target cluster is received, determining whether the cluster environment of a source cluster of the target user is consistent with the cluster environment of the target cluster.
S202, if so, judging whether the target user has the user authority of the target cluster.
The descriptions of S201 to S202 may refer to the descriptions of S101 to S102, and the same technical effects can be achieved, which will not be described in detail.
And S203, if the user permission exists, judging whether the version information exists on the target cluster according to the version information of the task carried in the task release request.
In the step, if the user limit exists in the cluster environment of the target cluster, judging whether the version information exists in the target cluster according to the version information of the task carried in the task release request, and selecting the version information according to the judging result.
S204, if the target version information exists, determining that the version information on the target cluster is the latest version information, determining a task version corresponding to the version information as the target version, and issuing the task of the target version in the target cluster.
In the step, if the version information exists on the target cluster, determining that the version information on the target cluster is the latest version information, determining a task version corresponding to the version information as the target version, and issuing the task of the target version in the target cluster.
Here, if the version information does not exist on the target cluster, the version information is initialized, and an initial version is denoted as v 000001.
Further, after the task of the target version is published in the target cluster, the task publishing method further includes:
and synchronously updating metadata information in the target version into the target cluster according to the target version of the task and a preset mapping rule.
Here, firstly, a preset mapping rule between different target versions is obtained, secondly, the metadata comprises all information of the targets and all information such as scripts, files, shells, hql and the like is stored, so after corresponding users and required versions are configured, the metadata information in the target versions is synchronously updated to the target cluster by combining the preset mapping rule between the target versions, and the consistency of the configuration of the files in the target cluster and the configuration of metadata resources is ensured.
And if the file is a local file, copying by using an ftp server. If an error occurs in the middle, automatically sending a mail to the target user.
Compared with the task release method in the prior art, the task release method provided by the embodiment of the application determines the version information of the task carried in the task release request released by the target user with the target cluster user authority as the target version of the release task under the condition that the cluster environment of the source cluster is consistent with the cluster environment of the target cluster, and releases the task of the target version in the target cluster, thereby realizing task release across clusters, reducing the compatibility problem of the task in different clusters, realizing release of the target version of the task through the user authority, greatly increasing the diversity of task release, reducing the labor force of developers and improving the accuracy and efficiency of release versions.
Referring to fig. 3, fig. 3 is a schematic structural diagram of a task publishing device with cross environments according to an embodiment of the present application. As shown in fig. 3, the task issuing device 300 includes:
a determining module 310, configured to determine, when a task issuing request for a target user for a target cluster is received, whether a cluster environment of a source cluster of the target user is consistent with a cluster environment of the target cluster.
Further, it is determined whether the cluster environment of the source cluster of the target user in the determining module 310 is consistent with the cluster environment of the target cluster by:
checking whether the configuration file in the source cluster is matched with the configuration file in the target cluster, and checking whether the related component environments in the source cluster are consistent with the related component environments in the target cluster.
Further, the verifying whether the configuration file in the source cluster matches the configuration file in the target cluster includes:
and if the configuration files in the source cluster and the configuration files in the target cluster meet a preset matching rule, determining that the configuration files in the source cluster and the configuration files in the target cluster are matched.
And a judging module 320, configured to judge whether the target user has the user authority of the target cluster if the target user has the user authority of the target cluster.
And the issuing module 330 is configured to determine, if the user authority is provided, a target version of the task issued on the target cluster according to the version information of the task carried in the task issuing request, and issue the task of the target version in the target cluster.
Further, the determining, by the publishing module 330, a target version of the task published on the target cluster according to the version information of the task carried in the task publishing request, and publishing, in the target cluster, the task of the target version includes:
and judging whether the version information exists on the target cluster according to the version information of the task carried in the task release request.
If the task version information exists, determining that the version information on the target cluster is the latest version information, determining the task version corresponding to the version information as the target version, and issuing the task of the target version in the target cluster.
Further, after the task of the target version is published in the target cluster, the task publishing method further includes:
and synchronously updating metadata information in the target version into the target cluster according to the target version of the task and a preset mapping rule.
The rollback module 340. And the rollback processing module is used for rollback processing the task of the appointed version when receiving the rollback request of the target user for the task of the appointed version in the target cluster.
And the processing module 350 is configured to delete the specified version of the task and metadata information corresponding to the specified version in the target cluster currently if it is determined that the rollback processing needs to be changed from the cluster environment of the target cluster to the cluster environment of the source cluster.
Compared with the task issuing device in the prior art, the task issuing device 300 provided by the embodiment of the application determines the version information of the task carried in the task issuing request issued by the target user with the target cluster user authority as the target version of the issuing task under the condition that the cluster environment of the source cluster is consistent with the cluster environment of the target cluster, and issues the task of the target version in the target cluster, thereby realizing task issuing of the cross-cluster, reducing the compatibility problem of the task in different clusters, realizing issuing of the target version of the task through the user authority, greatly increasing the diversity of task issuing, reducing the labor force of developers and improving the accuracy and efficiency of issuing the version.
Referring to fig. 4, fig. 4 is a schematic structural diagram of an electronic device 400 according to an embodiment of the application. As shown in fig. 4, the electronic device 400 includes a processor 410, a memory 420, and a bus 430, the memory 420 storing machine-readable instructions executable by the processor 410, the processor 410 and the memory 420 in communication via the bus 430 when the electronic device 400 is running, the machine-readable instructions being executed by the processor 410 to perform the steps of the task issuing method according to any of the embodiments described above.
In particular, the machine-readable instructions, when executed by the processor 410, may perform the following:
when a task release request of a target user for a target cluster is received, determining whether the cluster environment of a source cluster of the target user is consistent with the cluster environment of the target cluster.
And if so, judging whether the target user has the user authority of the target cluster.
And if the task has the user permission, determining a target version of the task published on the target cluster according to the version information of the task carried in the task publishing request, and publishing the task of the target version in the target cluster.
According to the method and the device, under the condition that the cluster environment of the source cluster is consistent with the cluster environment of the target cluster, the version information of the task carried in the task release request released by the target user with the target cluster user authority is determined to be the target version of the release task, and the task of the target version is released in the target cluster, so that the task release of the cross-cluster task is realized, the compatibility problem of the task in different clusters is reduced, the target version of the task can be released through the user authority, the diversity of the task release is greatly increased, the labor force of a developer is reduced, and the accuracy and the efficiency of the release of the version are improved.
The embodiment of the present application further provides a computer readable storage medium, where a computer program is stored, where the computer program may execute the steps of the task issuing method in the method embodiments shown in fig. 1 and fig. 2 when the computer program is executed by the processor 410, and the specific implementation manner may refer to the method embodiments and will not be described herein.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.
In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be other manners of division in actual implementation, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Finally, it should be noted that: the above examples are only specific embodiments of the present application, and are not intended to limit the scope of the present application, but it should be understood by those skilled in the art that the present application is not limited thereto, and that the present application is described in detail with reference to the foregoing examples: any person skilled in the art may modify or easily conceive of the technical solution described in the foregoing embodiments, or perform equivalent substitution of some of the technical features, while remaining within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application, and are intended to be included in the scope of the present application. Therefore, the protection scope of the application is subject to the protection scope of the claims.
Claims (8)
1. The task issuing method across clusters is characterized by comprising the following steps of:
when a task release request of a target user for a target cluster is received, determining whether the cluster environment of a source cluster of the target user is consistent with the cluster environment of the target cluster;
if so, judging whether the target user has the user authority of the target cluster;
if the task has the user permission, determining a target version of the task published on the target cluster according to the version information of the task carried in the task publishing request, and publishing the task of the target version in the target cluster;
determining whether a cluster environment of a source cluster of the target user is consistent with a cluster environment of the target cluster by:
verifying whether the configuration file in the source cluster is matched with the configuration file in the target cluster, and verifying whether the related component environments in the source cluster are consistent with the related component environments in the target cluster;
the configuration files in the source cluster and the configuration files in the target cluster comprise node types in each workflow in the two clusters, a local storage directory in the cluster, an hdfs storage directory in the cluster and user rights of the target user of the target cluster;
the verifying whether the related component environments in the source cluster are consistent with the related component environments in the target cluster includes: and checking whether components owned by the source cluster and the target cluster are consistent.
2. The task publishing method of claim 1, wherein said verifying if the configuration file in the source cluster matches the configuration file in the target cluster comprises:
and if the configuration files in the source cluster and the configuration files in the target cluster meet a preset matching rule, determining that the configuration files in the source cluster and the configuration files in the target cluster are matched.
3. The task publishing method of claim 1, wherein the determining the target version of the task published on the target cluster according to the version information of the task carried in the task publishing request, and publishing the task of the target version in the target cluster, comprises:
judging whether the version information exists on the target cluster according to the version information of the task carried in the task release request;
if the task version information exists, determining that the version information on the target cluster is the latest version information, determining the task version corresponding to the version information as the target version, and issuing the task of the target version in the target cluster.
4. The task publishing method of claim 3, wherein after publishing the target version of the task in the target cluster, the task publishing method further comprises:
and synchronously updating metadata information in the target version into the target cluster according to the target version of the task and a preset mapping rule.
5. The task issuing method according to claim 1, characterized in that the task issuing method further comprises:
when receiving a rollback request of a target user for a task of a specified version in a target cluster, rollback processing is carried out on the task of the specified version;
and if the rollback processing is determined to be changed from the cluster environment of the target cluster to the cluster environment of the source cluster, deleting the appointed version of the task in the target cluster and the metadata information corresponding to the appointed version.
6. A cross-cluster task publication device, comprising:
the determining module is used for determining whether the cluster environment of the source cluster of the target user is consistent with the cluster environment of the target cluster or not when a task release request of the target user for the target cluster is received;
the judging module is used for judging whether the target user has the user authority of the target cluster or not if the target user has the user authority of the target cluster;
the issuing module is used for determining a target version of the task issued on the target cluster according to the version information of the task carried in the task issuing request if the task has the user permission, and issuing the task of the target version in the target cluster;
determining whether a cluster environment of a source cluster of the target user is consistent with a cluster environment of the target cluster by:
verifying whether the configuration file in the source cluster is matched with the configuration file in the target cluster, and verifying whether the related component environments in the source cluster are consistent with the related component environments in the target cluster;
the configuration files in the source cluster and the configuration files in the target cluster comprise node types in each workflow in the two clusters, a local storage directory in the cluster, an hdfs storage directory in the cluster and user rights of the target user of the target cluster;
the verifying whether the related component environments in the source cluster are consistent with the related component environments in the target cluster includes: and checking whether components owned by the source cluster and the target cluster are consistent.
7. An electronic device, comprising: a processor, a memory and a bus, said memory storing machine readable instructions executable by said processor, said processor and said memory communicating via the bus when the electronic device is running, said machine readable instructions when executed by said processor performing the steps of the task issuing method according to any of the preceding claims 1 to 5.
8. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, performs the steps of the task issuing method according to any of the preceding claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011618326.2A CN112749085B (en) | 2020-12-31 | 2020-12-31 | Cross-cluster task publishing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011618326.2A CN112749085B (en) | 2020-12-31 | 2020-12-31 | Cross-cluster task publishing method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112749085A CN112749085A (en) | 2021-05-04 |
| CN112749085B true CN112749085B (en) | 2023-10-24 |
Family
ID=75650181
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011618326.2A Active CN112749085B (en) | 2020-12-31 | 2020-12-31 | Cross-cluster task publishing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112749085B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB201607542D0 (en) * | 2016-04-29 | 2016-06-15 | Fujitsu Ltd | A computer-implemented method of executing a query in a network of data centres |
| WO2017041486A1 (en) * | 2015-09-10 | 2017-03-16 | 中兴通讯股份有限公司 | Cluster deployment implementation method and apparatus |
| US9632892B1 (en) * | 2015-03-30 | 2017-04-25 | EMC IP Holding Company LLC | NFS cluster failover |
| WO2017161984A1 (en) * | 2016-03-24 | 2017-09-28 | 中兴通讯股份有限公司 | Method, device and system for deploying data clusters, and computer storage medium |
| CN108076088A (en) * | 2016-11-10 | 2018-05-25 | 中国移动通信集团安徽有限公司 | Communication service system and its version dissemination method based on telecommunication network |
| CN108574720A (en) * | 2017-05-09 | 2018-09-25 | 北京金山云网络技术有限公司 | A kind of service loading method and device |
| CN109471799A (en) * | 2018-10-29 | 2019-03-15 | 北京城市网邻信息技术有限公司 | A kind of dispositions method of demand environment, device, storage medium and terminal |
| CN110209653A (en) * | 2019-06-04 | 2019-09-06 | 中国农业银行股份有限公司 | HBase data migration method and moving apparatus |
| US10503748B1 (en) * | 2016-11-14 | 2019-12-10 | Amazon Technologies, Inc. | ELT management service with publish/subscribe architecture |
| CN110572436A (en) * | 2019-08-12 | 2019-12-13 | 浙江讯盟科技有限公司 | Multi-place cross-cluster server deployment method and system |
| CN111581285A (en) * | 2020-04-30 | 2020-08-25 | 中国工商银行股份有限公司 | Data information synchronization method and device, electronic equipment and medium |
| CN111666080A (en) * | 2020-04-28 | 2020-09-15 | 平安科技（深圳）有限公司 | Micro-service cluster deployment method and device, computer equipment and storage medium |
-
2020
- 2020-12-31 CN CN202011618326.2A patent/CN112749085B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9632892B1 (en) * | 2015-03-30 | 2017-04-25 | EMC IP Holding Company LLC | NFS cluster failover |
| WO2017041486A1 (en) * | 2015-09-10 | 2017-03-16 | 中兴通讯股份有限公司 | Cluster deployment implementation method and apparatus |
| WO2017161984A1 (en) * | 2016-03-24 | 2017-09-28 | 中兴通讯股份有限公司 | Method, device and system for deploying data clusters, and computer storage medium |
| GB201607542D0 (en) * | 2016-04-29 | 2016-06-15 | Fujitsu Ltd | A computer-implemented method of executing a query in a network of data centres |
| CN108076088A (en) * | 2016-11-10 | 2018-05-25 | 中国移动通信集团安徽有限公司 | Communication service system and its version dissemination method based on telecommunication network |
| US10503748B1 (en) * | 2016-11-14 | 2019-12-10 | Amazon Technologies, Inc. | ELT management service with publish/subscribe architecture |
| CN108574720A (en) * | 2017-05-09 | 2018-09-25 | 北京金山云网络技术有限公司 | A kind of service loading method and device |
| CN109471799A (en) * | 2018-10-29 | 2019-03-15 | 北京城市网邻信息技术有限公司 | A kind of dispositions method of demand environment, device, storage medium and terminal |
| CN110209653A (en) * | 2019-06-04 | 2019-09-06 | 中国农业银行股份有限公司 | HBase data migration method and moving apparatus |
| CN110572436A (en) * | 2019-08-12 | 2019-12-13 | 浙江讯盟科技有限公司 | Multi-place cross-cluster server deployment method and system |
| CN111666080A (en) * | 2020-04-28 | 2020-09-15 | 平安科技（深圳）有限公司 | Micro-service cluster deployment method and device, computer equipment and storage medium |
| CN111581285A (en) * | 2020-04-30 | 2020-08-25 | 中国工商银行股份有限公司 | Data information synchronization method and device, electronic equipment and medium |
Non-Patent Citations (2)
| Title |
|---|
| 基于容器技术的PaaS云平台方案;齐磊;《电信科学》;1-5 * |
| 基于阿里云的Kubernetes容器云平台的设计与实现;陈金光;《万方》;1-50 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112749085A (en) | 2021-05-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11385890B2 (en) | Shared software libraries for computing devices | |
| JP6774499B2 (en) | Providing access to hybrid applications offline | |
| JP7158482B2 (en) | Method, computer-readable medium, and system for resolution of violations in client synchronization | |
| CN109558215B (en) | Backup method, recovery method and device of virtual machine and backup server cluster | |
| US10169614B2 (en) | Container update system | |
| US11321291B2 (en) | Persistent version control for data transfer between heterogeneous data stores | |
| US20140372998A1 (en) | App package deployment | |
| US11308223B2 (en) | Blockchain-based file handling | |
| US10747643B2 (en) | System for debugging a client synchronization service | |
| US10970193B2 (en) | Debugging a client synchronization service | |
| CN110162429A (en) | System repair, server and storage medium | |
| CN107526838B (en) | Method and device for database cluster capacity expansion | |
| CN113094085A (en) | Component integration method and device, computer-readable storage medium and electronic equipment | |
| CN112749085B (en) | Cross-cluster task publishing method and device, electronic equipment and storage medium | |
| CN111444148A (en) | Data transmission method and device based on MapReduce | |
| US9298487B2 (en) | Managing virtual machine images in a distributed computing environment | |
| CN113515303B (en) | Project transformation method, device and equipment | |
| US11675931B2 (en) | Creating vendor-neutral data protection operations for vendors' application resources | |
| US20120005162A1 (en) | Managing Copies of Data Structures in File Systems | |
| CN115964061A (en) | Plug-in updating method and device, electronic equipment and computer readable storage medium | |
| JP5943753B2 (en) | Virtual machine management system, virtual machine management method and program | |
| US10162626B2 (en) | Ordered cache tiering for program build files | |
| Li et al. | A hybrid disaster-tolerant model with DDF technology for MooseFS open-source distributed file system | |
| CN113761075A (en) | Method, device, equipment and computer readable medium for switching databases | |
| US10185759B2 (en) | Distinguishing event type |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |