CN111414226A - Method and system for establishing task sandbox - Google Patents
Method and system for establishing task sandbox Download PDFInfo
- Publication number
- CN111414226A CN111414226A CN201910010493.XA CN201910010493A CN111414226A CN 111414226 A CN111414226 A CN 111414226A CN 201910010493 A CN201910010493 A CN 201910010493A CN 111414226 A CN111414226 A CN 111414226A
- Authority
- CN
- China
- Prior art keywords
- task
- execution
- directory
- dependent
- configuration file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45562—Creating, deleting, cloning virtual machine instances
Abstract
The invention discloses a method and a system for establishing a task sandbox, wherein the method comprises the following steps: acquiring a configuration file, wherein the content of the configuration file comprises: the method comprises the steps of calculating a feature list, a sample acquisition data source, a dependent task code, a dependent feature list and version information of a dependent library; creating a task execution directory; generating a virtual environment under an execution directory according to a dependent library version and a dependent task code in a configuration file, and copying a dependency characteristic list appointed in the configuration file to the execution directory; in the virtual environment, the intermediate result and the final result of the execution are saved in the task execution directory. After the scheme is adopted, on one hand, the customization requirements of users are met; on the other hand, a context storage mode of an execution result is standardized, so that all generated environments automatically reserve complete backtracking and problem positioning information, and a user is not required to manually maintain the environment.
Description
Technical Field
The invention belongs to a computer processing system, and relates to a method and a system for establishing a task sandbox.
Background
Currently, when configuring an offline computing environment, the traditional solution is that an architecture engineer builds a set of computing environment as a whole, including the latest code, the latest version of the dependency library, and the latest configuration file, for use by others. This approach is not flexible on the one hand and does not meet the customized requirements of each user; on the other hand, the context of the execution result depends on the user, so that the use threshold is increased, the unified management is not facilitated, and the contexts are very important backtracking basis and problem positioning information.
Disclosure of Invention
The invention aims to provide a task sandbox establishing method and system.
The technical scheme adopted by the invention for solving the technical problems is as follows:
a task sandbox establishing method comprises the following steps:
acquiring a configuration file, wherein the content of the configuration file comprises: the method comprises the steps of calculating a feature list, a sample acquisition data source, a dependent task code, a dependent feature list and version information of a dependent library;
creating a task execution directory;
generating a virtual environment under an execution directory according to a dependent library version and a dependent task code in a configuration file, and copying a dependency characteristic list appointed in the configuration file to the execution directory;
in the virtual environment, the intermediate result and the final result of the execution are saved in the task execution directory.
Preferably, the creating of the task execution directory specifically includes:
and establishing a task execution directory for each task, wherein the task execution directory is provided with information related to the task.
Preferably, the information related to the task includes:
task execution code, dependency library execution code, configuration files used by the task, intermediate results and final results of task execution.
Preferably, the saving the intermediate result and the final result in the task execution directory further comprises: and packaging the execution catalog, uploading the backup and ensuring the traceability of the task.
Preferably, the virtual environment is a virtualenv environment generated based on python language.
A task sandbox creation system comprising:
a configuration file obtaining unit, configured to obtain a configuration file, where the content of the configuration file includes: the method comprises the steps of calculating a feature list, a sample acquisition data source, a dependent task code, a dependent feature list and version information of a dependent library;
the task execution catalog creating unit is used for creating a task execution catalog;
generating a virtual environment under an execution directory according to a dependent library version and a dependent task code in a configuration file, and copying a dependency characteristic list appointed in the configuration file to the execution directory;
and the data execution unit is used for storing the intermediate result and the final result of the execution in the task execution directory in the virtual environment.
Preferably, the creating of the task execution directory specifically includes:
and establishing a task execution directory for each task, wherein the task execution directory is provided with information related to the task.
Preferably, the information related to the task includes:
task execution code, dependency library execution code, configuration files used by the task, intermediate results and final results of task execution.
Preferably, the saving the intermediate result and the final result in the task execution directory further comprises: and packaging the execution catalog, uploading the backup and ensuring the traceability of the task.
Preferably, the virtual environment is a virtualenv environment generated based on python language.
After the scheme is adopted, on one hand, the customization requirements of users are met; on the other hand, a context storage mode of an execution result is standardized, so that all generated environments automatically reserve complete backtracking and problem positioning information, and a user is not required to manually maintain the environment.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
The present invention will be described in detail below with reference to the accompanying drawings so that the above advantages of the present invention will be more apparent. Wherein the content of the first and second substances,
FIG. 1 is a schematic diagram of an embodiment of a method for creating a task sandbox in accordance with the present invention.
Detailed Description
The following detailed description of the embodiments of the present invention will be provided with reference to the drawings and examples, so that how to apply the technical means to solve the technical problems and achieve the technical effects can be fully understood and implemented. It should be noted that, as long as there is no conflict, the embodiments and the features of the embodiments of the present invention may be combined with each other, and the technical solutions formed are within the scope of the present invention.
Additionally, the steps illustrated in the flow charts of the figures may be performed in a computer system such as a set of computer-executable instructions and, although a logical order is illustrated in the flow charts, in some cases, the steps illustrated or described may be performed in an order different than here.
The first embodiment is as follows:
the invention designs a set of full-isolation task sandbox scheme.
As shown in fig. 1, a method for establishing a task sandbox includes:
acquiring a configuration file, wherein the content of the configuration file comprises: the method comprises the steps of calculating a feature list, a sample acquisition data source, a dependent task code, a dependent feature list and version information of a dependent library;
creating a task execution directory;
generating a virtual environment under an execution directory according to a dependent library version and a dependent task code in a configuration file, and copying a dependency characteristic list appointed in the configuration file to the execution directory;
in the virtual environment, the intermediate result and the final result of the execution are saved in the task execution directory.
Preferably, the creating of the task execution directory specifically includes:
and establishing a task execution directory for each task, wherein the task execution directory is provided with information related to the task.
Preferably, the information related to the task includes:
task execution code, dependency library execution code, configuration files used by the task, intermediate results and final results of task execution.
Preferably, the saving the intermediate result and the final result in the task execution directory further comprises: and packaging the execution catalog, uploading the backup and ensuring the traceability of the task.
Preferably, the virtual environment is a virtualenv environment generated based on python language.
After the scheme is adopted, on one hand, the customization requirements of users are met; on the other hand, a context storage mode of an execution result is standardized, so that all generated environments automatically reserve complete backtracking and problem positioning information, and a user is not required to manually maintain the environment.
Example two:
the present invention is described with reference to examples, and specifically, the present invention includes:
and establishing a task execution directory for each task, wherein the directory automatically stores all information related to the task, such as task execution codes, execution codes of a dependent library, configuration files used by the task, intermediate results and final results of task execution, and the like, and the series of operations are toolized.
The method specifically comprises the following steps:
1. and writing a configuration file. The user can fill in the configuration file according to the actual requirement of the user, and the content of the configuration file comprises the following steps: a list of features to be computed, a sample acquisition data source, a version of the dependency library, etc.
2. And (5) initializing the environment. Creating a task execution directory, then generating a virutalenv environment under the execution directory according to information such as a dependent library version and a dependent task code configured by the configuration file, and copying an additional configuration such as a dependent feature list specified in the configuration file to the execution directory.
3. And executing the task. And using the virtualenv created in the step 2, and saving the intermediate result and the final result of the execution in the task execution directory.
4. A directory packed backup is performed. And packaging the execution directory, and uploading the execution directory to hdfs for backup processing. Ensuring the traceability of the task.
Therefore, on one hand, the customization requirements of users are met; on the other hand, a context storage mode of an execution result is standardized, so that all generated environments automatically reserve complete backtracking and problem positioning information, and a user is not required to manually maintain the environment.
Example three:
corresponding to the above embodiment, the present invention further provides a system for establishing a task sandbox, including:
a configuration file obtaining unit, configured to obtain a configuration file, where the content of the configuration file includes: the method comprises the steps of calculating a feature list, a sample acquisition data source, a dependent task code, a dependent feature list and version information of a dependent library;
the task execution catalog creating unit is used for creating a task execution catalog;
generating a virtual environment under an execution directory according to a dependent library version and a dependent task code in a configuration file, and copying a dependency characteristic list appointed in the configuration file to the execution directory;
and the data execution unit is used for storing the intermediate result and the final result of the execution in the task execution directory in the virtual environment.
Preferably, the creating of the task execution directory specifically includes:
and establishing a task execution directory for each task, wherein the task execution directory is provided with information related to the task.
Preferably, the information related to the task includes:
task execution code, dependency library execution code, configuration files used by the task, intermediate results and final results of task execution.
Preferably, the saving the intermediate result and the final result in the task execution directory further comprises: and packaging the execution catalog, uploading the backup and ensuring the traceability of the task.
Preferably, the virtual environment is a virtualenv environment generated based on python language.
After the scheme is adopted, on one hand, the customization requirements of users are met; on the other hand, a context storage mode of an execution result is standardized, so that all generated environments automatically reserve complete backtracking and problem positioning information, and a user is not required to manually maintain the environment.
It should be noted that for simplicity of description, the above method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects.
Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that changes may be made in the embodiments and/or equivalents thereof without departing from the spirit and scope of the invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A task sandbox establishing method is characterized by comprising the following steps:
acquiring a configuration file, wherein the content of the configuration file comprises: the method comprises the steps of calculating a feature list, a sample acquisition data source, a dependent task code, a dependent feature list and version information of a dependent library;
creating a task execution directory;
generating a virtual environment under an execution directory according to a dependent library version and a dependent task code in a configuration file, and copying a dependency characteristic list appointed in the configuration file to the execution directory;
in the virtual environment, the intermediate result and the final result of the execution are saved in the task execution directory.
2. The method for establishing a task sandbox according to claim 1, wherein creating a task execution directory specifically includes:
and establishing a task execution directory for each task, wherein the task execution directory is provided with information related to the task.
3. The method for creating a task sandbox as claimed in claim 2, wherein the information related to the task comprises:
task execution code, dependency library execution code, configuration files used by the task, intermediate results and final results of task execution.
4. The method of claim 1, wherein the storing the intermediate results and the final results in a task execution directory, further comprises: and packaging the execution catalog, uploading the backup and ensuring the traceability of the task.
5. The method for creating a task sandbox as recited in claim 1, wherein the virtual environment is a virtualenv environment generated based on python language.
6. A task sandbox creation system, comprising:
a configuration file obtaining unit, configured to obtain a configuration file, where the content of the configuration file includes: the method comprises the steps of calculating a feature list, a sample acquisition data source, a dependent task code, a dependent feature list and version information of a dependent library;
the task execution catalog creating unit is used for creating a task execution catalog;
generating a virtual environment under an execution directory according to a dependent library version and a dependent task code in a configuration file, and copying a dependency characteristic list appointed in the configuration file to the execution directory;
and the data execution unit is used for storing the intermediate result and the final result of the execution in the task execution directory in the virtual environment.
7. The system for establishing a task sandbox as claimed in claim 6, wherein the creating of the task execution directory specifically comprises:
and establishing a task execution directory for each task, wherein the task execution directory is provided with information related to the task.
8. The system for creating a task sandbox as recited in claim 7, wherein said information related to the task comprises:
task execution code, dependency library execution code, configuration files used by the task, intermediate results and final results of task execution.
9. The system for creating a task sandbox as in claim 6, wherein the intermediate results and the final results of the execution are stored in a task execution directory, further comprising: and packaging the execution catalog, uploading the backup and ensuring the traceability of the task.
10. The system for creating a task sandbox as recited in claim 1, wherein said virtual environment is a virtualenv environment generated based on python language.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910010493.XA CN111414226A (en) | 2019-01-07 | 2019-01-07 | Method and system for establishing task sandbox |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910010493.XA CN111414226A (en) | 2019-01-07 | 2019-01-07 | Method and system for establishing task sandbox |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111414226A true CN111414226A (en) | 2020-07-14 |
Family
ID=71490714
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910010493.XA Pending CN111414226A (en) | 2019-01-07 | 2019-01-07 | Method and system for establishing task sandbox |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111414226A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111708564A (en) * | 2020-08-20 | 2020-09-25 | 上海森亿医疗科技有限公司 | Multi-model management method, system, medium and server |
| CN116991563A (en) * | 2023-09-28 | 2023-11-03 | 之江实验室 | Queue generating method and device supporting rapid sandbox construction |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150100726A1 (en) * | 2013-10-03 | 2015-04-09 | Cleversafe, Inc. | Dispersed storage system with width dispersal control and methods for use therewith |
| CN105068873A (en) * | 2015-08-10 | 2015-11-18 | 北京思特奇信息技术股份有限公司 | Isomerous virtual resource and task scheduling method and system |
| CN106775730A (en) * | 2016-12-21 | 2017-05-31 | 深圳Tcl数字技术有限公司 | Program dissemination method and system |
| CN107220116A (en) * | 2017-05-25 | 2017-09-29 | 深信服科技股份有限公司 | Sandbox environment task processing method and system under a kind of NUMA architecture |
| CN107357583A (en) * | 2017-07-13 | 2017-11-17 | 上海市共进通信技术有限公司 | The system and method built automatically based on software code progress module |
| CN107608710A (en) * | 2017-08-31 | 2018-01-19 | 华为技术有限公司 | Software project structure task collocation method and device based on Jenkins instruments |
| CN107885587A (en) * | 2017-11-17 | 2018-04-06 | 清华大学 | A kind of executive plan generation method of big data analysis process |
| CN107977253A (en) * | 2016-10-25 | 2018-05-01 | 腾讯科技（北京）有限公司 | Program processing method, IC clients and IC platforms |
-
2019
- 2019-01-07 CN CN201910010493.XA patent/CN111414226A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150100726A1 (en) * | 2013-10-03 | 2015-04-09 | Cleversafe, Inc. | Dispersed storage system with width dispersal control and methods for use therewith |
| CN105068873A (en) * | 2015-08-10 | 2015-11-18 | 北京思特奇信息技术股份有限公司 | Isomerous virtual resource and task scheduling method and system |
| CN107977253A (en) * | 2016-10-25 | 2018-05-01 | 腾讯科技（北京）有限公司 | Program processing method, IC clients and IC platforms |
| CN106775730A (en) * | 2016-12-21 | 2017-05-31 | 深圳Tcl数字技术有限公司 | Program dissemination method and system |
| CN107220116A (en) * | 2017-05-25 | 2017-09-29 | 深信服科技股份有限公司 | Sandbox environment task processing method and system under a kind of NUMA architecture |
| CN107357583A (en) * | 2017-07-13 | 2017-11-17 | 上海市共进通信技术有限公司 | The system and method built automatically based on software code progress module |
| CN107608710A (en) * | 2017-08-31 | 2018-01-19 | 华为技术有限公司 | Software project structure task collocation method and device based on Jenkins instruments |
| CN107885587A (en) * | 2017-11-17 | 2018-04-06 | 清华大学 | A kind of executive plan generation method of big data analysis process |
Non-Patent Citations (2)
| Title |
|---|
| R. SUAREZ 等: "Using configuration and force sensing in assembly task planning and execution", 《PROCEEDINGS. IEEE INTERNATIONAL SYMPOSIUM ON ASSEMBLY AND TASK PLANNING》, pages 273 - 279 * |
| 李凌: "支持依赖关系的并行任务软件框架研究与实现", 《安徽广播电视大学学报》, pages 120 - 124 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111708564A (en) * | 2020-08-20 | 2020-09-25 | 上海森亿医疗科技有限公司 | Multi-model management method, system, medium and server |
| CN116991563A (en) * | 2023-09-28 | 2023-11-03 | 之江实验室 | Queue generating method and device supporting rapid sandbox construction |
| CN116991563B (en) * | 2023-09-28 | 2023-12-22 | 之江实验室 | Queue generating method and device supporting rapid sandbox construction |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5957084B2 (en) | Generate compiled code indicating register survival | |
| US9411576B2 (en) | Software version management for executing replacement actions | |
| US9086885B2 (en) | Reducing merge conflicts in a development environment | |
| CN110955431B (en) | Processing method and device of compiling environment | |
| CN111090433B (en) | Data processing method, device and storage medium | |
| US11074154B2 (en) | Identifying a source file for use in debugging compiled code | |
| CN110059456B (en) | Code protection method, code protection device, storage medium and electronic equipment | |
| US10248409B1 (en) | Limiting the effects of source code patches on corresponding native-code patches | |
| US20210312392A1 (en) | Namespace Management Techniques for Facilitating Multi-Cluster Application Development | |
| US20180081655A1 (en) | Metadata-Driven Binding of Converted Source Code to Original Source Code | |
| WO2017213846A1 (en) | Automating feature graduation | |
| US10970047B2 (en) | Lossless bi-directional formatting of program source code | |
| US9116777B1 (en) | In-flight process instance migration between business process execution language (BPEL) suites | |
| US20160224327A1 (en) | Linking a Program with a Software Library | |
| CN111414226A (en) | Method and system for establishing task sandbox | |
| US20160253157A1 (en) | Software refactoring | |
| US9798483B2 (en) | Object storage power consumption optimization | |
| US9454382B2 (en) | Verification of UML state machines | |
| KR101924747B1 (en) | Flexible metadata composition | |
| US9244706B2 (en) | Command line shell command generation based on schema | |
| CN109558121A (en) | Development approach, device, equipment and the storage medium of interface drive program | |
| KR20180131009A (en) | System and method for converting large scale application | |
| CN115098158A (en) | SDK packaging method and device, computer equipment and storage medium | |
| CN110990018B (en) | Compiling method and compiling system of embedded system | |
| US10606729B2 (en) | Estimating the number of coding styles by analyzing source code |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |