CN111078358A - Test data processing method and system based on software container - Google Patents
Test data processing method and system based on software container Download PDFInfo
- Publication number
- CN111078358A CN111078358A CN201911232506.4A CN201911232506A CN111078358A CN 111078358 A CN111078358 A CN 111078358A CN 201911232506 A CN201911232506 A CN 201911232506A CN 111078358 A CN111078358 A CN 111078358A
- Authority
- CN
- China
- Prior art keywords
- data processing
- software container
- test data
- user
- software
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45504—Abstract machines for programme code execution, e.g. Java virtual machine [JVM], interpreters, emulators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Stored Programmes (AREA)
Abstract
The invention discloses a test data processing method and a system based on a software container, wherein the test data processing method comprises the following steps: packing various types of experimental data processing software and corresponding information into a plurality of virtualized mirror images and deploying the virtualized mirror images to a core server to form a software container; establishing an independent platform management database on a core server and setting a data table about a scheduling sequence; judging whether a user logs in and requests to perform test data processing, if so, creating an individual user sequence and distributing corresponding priority, and meanwhile, distributing monitoring fields according to the characteristics of the software container; transferring user sequence information into the software container through the socket; processing test data through a software container; analyzing the monitoring field of the software container, extracting a test data processing result, transmitting the test data processing result back to the platform management database, and associating a user sequence; and reading and/or downloading the experimental data processing result. The invention has the advantages of convenient maintenance, upgrade and use, low use difficulty and low error rate.
Description
Technical Field
The invention belongs to the technical field of test data processing, and particularly relates to a test data processing method and system based on a software container.
Background
The rail transit equipment manufacturing belongs to a manufacturing industry with very high integration level, and the professional fields related to the rail transit equipment are quite many, so the detection and test data processing technology related to the rail transit equipment is quite complex. Various commercial software and self-developed test data processing software are in a variety of layers, even part of the software conflicts with an operating system and a software environment, the software cannot be installed or executed on the same computer at the same time, switching among various operation interfaces is often needed, and problems such as stability, reliability and usability exist in the aspects of maintenance, upgrading and use, so that the use cost is increased.
Software container technology is a lightweight virtual service that packages software into standardized units, i.e., container images, for software development, delivery, and deployment. The image is a lightweight, executable, stand-alone software package, comprising: code, runtime environment, system tools, system libraries, and settings. The technical characteristics comprise:
1) only necessary codes, operating environments, system tools, system libraries and settings are packaged;
2) rapid deployment (compared to traditional bare machine mode or virtual machine mode);
3) constructing once, not picking up the bottom layer, and deploying everywhere;
4) the system has strong elastic management capability and is very convenient to upgrade, maintain and troubleshoot.
Disclosure of Invention
Aiming at the problems of complexity and difficulty in maintenance, upgrading and use of the existing test data processing software, the invention provides a test data processing method and system based on a software container based on the characteristics of the software container.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows:
a test data processing method based on a software container is characterized by comprising the following steps:
and 7, reading and/or downloading an experimental data processing result through an experimental data processing front-end interface.
Further, step 8, judging whether the user needs to quit the dispatching sequence distribution service front-end interface or cancel the user, if not, the user sequence is resident in the memory; if yes, after the experimental data processing is completed by using the software container, when the user exits from the front end interface of the scheduling sequence allocation service, the scheduling sequence allocation service unit automatically deletes the user sequence.
Preferably, the lightweight virtual service unit is a docker development tool.
Further, step 1 further includes establishing a private image library including the plurality of virtualized images.
Based on the same inventive concept, the invention also provides a test data processing system based on the software container, which is characterized by comprising the following steps:
light-weight virtual service unit: the system comprises a plurality of virtual images, a plurality of test data processing software, a plurality of corresponding software operating environments, a plurality of necessary databases and a plurality of necessary operating system services, wherein the plurality of test data processing software is packaged into the plurality of virtual images; the virtual machine is used for deploying the plurality of virtual images to a core server to form a software container; the software container is used for transmitting user sequence information into the software container through the socket; performing test data processing by using the allocated software container;
a core server: used for establishing a separate platform management database;
platform management database: a data table for setting a scheduling sequence;
scheduling sequence assignment service unit: the system comprises a scheduling sequence distribution service front-end interface, a software container, a monitoring field and a control field, wherein the scheduling sequence distribution service front-end interface is used for receiving a request from a user to perform test data processing;
experiment data processing background service unit: the system comprises a software container, a platform management database, a data processing module and a data processing module, wherein the software container is used for monitoring field analysis, extracting test data processing results and transmitting the test data processing results back to the platform management database, and associating the test data processing results with corresponding user sequences;
experimental data processing front end interface: for reading and/or downloading experimental data processing results.
Further, the user sequence:
the method comprises the steps that before a user quits a scheduling sequence distribution service front-end interface or logs off the user, the user resides in a memory;
after the experimental data processing is finished by using the software container, when the user exits from the front end interface of the dispatching sequence distribution service, the dispatching sequence distribution service unit automatically deletes the experimental data.
Preferably, the lightweight virtual service unit is a docker development tool.
Further, the lightweight virtual service unit is further configured to establish a private image library including the plurality of virtualized images.
Compared with the prior art, the invention uses the unified integration entrance, so that the user-oriented interface is unified during maintenance, upgrade and use, the use difficulty and the error rate are reduced, and the invention has the following specific beneficial effects: (1) the hardware utilization rate is high, and the hardware cost is saved; (2) the mobility is good; (3) the compatibility among various test data processing software is eliminated; (4) the maintenance and use difference among various test data processing software is eliminated; (5) the problem of conflict scheduling among multiple software containers is solved.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
FIG. 2 is a block diagram of the system of the present invention.
The system comprises a core server 1, a lightweight virtual service unit 2, a private mirror library 3, a virtualization mirror 31, a virtualization mirror 32, a virtualization mirror 34, a platform management database 4, a scheduling sequence distribution service unit 5, an experimental data processing background service unit 6, a scheduling sequence distribution service front-end interface 7 and an experimental data processing front-end interface 8.
Detailed Description
The lightweight virtual service unit 2 used in this embodiment is an open-source docker development tool, and the function of the software container can be realized by applying other lightweight virtual service software, which all belong to the protection scope of the present invention.
As shown in FIG. 1, the method for processing test data based on software container of the present invention comprises the following steps:
And 2, deploying the plurality of docker virtualization images 31, 32, 33 and 34 to the core server 1 by using a built-in command of the docker development tool to form a software container.
And 3, establishing an independent platform management database 4 on the core server 1, and setting a data table related to a scheduling sequence in the platform management database 4.
And 4, judging whether a user logs in through the dispatching sequence distribution service front-end interface 7 and requests to perform test data processing, if so, establishing an independent user sequence through the dispatching sequence distribution service unit 5, distributing corresponding priority according to a user account, and automatically distributing corresponding monitoring fields according to the characteristics of the software container.
And 6, analyzing the field monitored by the software container by the experimental data processing background service unit 6, extracting the experimental data processing result, returning the experimental data processing result to the platform management database 4, and associating the experimental data processing result with the corresponding user sequence.
And 7, reading and/or downloading the experimental data processing result through the experimental data processing front-end interface 8 by the user.
As shown in fig. 2, the test data processing system based on software container of the present invention includes:
light-weight virtual service unit 2: the system is used for packaging various experimental data processing software, corresponding software running environments, necessary databases and necessary operating system services into a plurality of virtual images 31, 32, 33 and 34; for deploying the plurality of virtualized images 31, 32, 33, 34 onto the core server 1, forming a software container; the software container is used for transmitting user sequence information into the software container through the socket; performing test data processing by using the allocated software container; and also for establishing a private image library 3 comprising said plurality of virtualized images 31, 32, 33, 34.
The lightweight virtual service unit 2 is a docker development tool.
The core server 1: for building a separate platform management database 4.
Platform management database 4: for setting a data table about the scheduling sequence.
Scheduling sequence assignment service unit 5: the method is used for creating an independent user sequence when a user logs in through a scheduling sequence distribution service front-end interface 7 and requests to perform test data processing, distributing corresponding priority according to a user account, and automatically distributing corresponding monitoring fields according to the characteristics of a software container.
Experiment data processing background service unit 6: and the data processing system is used for analyzing the monitoring field of the software container, extracting the processing result of the test data, transmitting the processing result of the test data back to the platform management database 4 and associating the processing result of the test data with the corresponding user sequence.
Experimental data processing front end interface 8: for reading and/or downloading experimental data processing results.
The user sequence is as follows:
before the user quits the dispatching sequence distribution service front-end interface 7 or logs off the user, the user resides in the memory;
after the experimental data processing is completed by using the software container, when the user exits the scheduling sequence allocation service front-end interface 7, the scheduling sequence allocation service unit 5 automatically deletes the data.
While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (8)
1. A test data processing method based on a software container is characterized by comprising the following steps:
step 1, packaging various types of experimental data processing software, corresponding software running environments, necessary databases and necessary operating system services into a plurality of virtualization mirror images (31, 32, 33 and 34) by using a lightweight virtual service unit (2);
step 2, deploying the plurality of virtualization mirror images (31, 32, 33, 34) to a core server (1) by using a lightweight virtual service unit (2) to form a software container;
step 3, establishing an independent platform management database (4) on the core server (1), and setting a data table related to a scheduling sequence in the platform management database (4);
step 4, judging whether a user logs in through a dispatching sequence distribution service front-end interface (7) and requests to perform test data processing, if so, establishing an independent user sequence through a dispatching sequence distribution service unit (5), distributing corresponding priority according to a user account, and automatically distributing corresponding monitoring fields according to the characteristics of the software container;
step 5, transferring the user sequence information into a software container through a socket of the lightweight virtual service unit (2); performing test data processing by using the allocated software container;
step 6, analyzing the test data processing background service unit (6) through the monitoring field of the software container, extracting the test data processing result, returning the test data processing result to the platform management database (4), and associating the test data processing result with the corresponding user sequence;
and 7, reading and/or downloading an experimental data processing result through an experimental data processing front-end interface (8).
2. The software container-based test data processing method of claim 1, further comprising:
step 8, judging whether the user needs to quit the dispatching sequence distribution service front-end interface (7) or cancel the user, if not, the user sequence is resident in the memory; if yes, after the experimental data processing is finished by using the software container, when the user quits the scheduling sequence distribution service front-end interface (7), the scheduling sequence distribution service unit (5) automatically deletes the user sequence.
3. The software container-based test data processing method of claim 1 or 2, wherein the lightweight virtual service unit (2) is a docker development tool.
4. The software container-based experimental data processing method according to claim 1 or 2, wherein the step 1 further comprises establishing a private image library (3) including the plurality of virtualized images (31, 32, 33, 34).
5. A software container-based test data processing system, comprising:
lightweight virtual service unit (2): the system is used for packaging various experimental data processing software, corresponding software running environments, necessary databases and necessary operating system services into a plurality of virtualization mirror images (31, 32, 33 and 34); for deploying the plurality of virtualized images (31, 32, 33, 34) onto a core server (1), forming a software container; the software container is used for transmitting user sequence information into the software container through the socket; performing test data processing by using the allocated software container;
core server (1): for building a separate platform management database (4);
platform management database (4): a data table for setting a scheduling sequence;
scheduling sequence assignment service unit (5): the system is used for creating an independent user sequence when a user logs in through a scheduling sequence distribution service front-end interface (7) and requests to perform test data processing, distributing corresponding priority according to a user account, and automatically distributing corresponding monitoring fields according to the characteristics of a software container;
experiment data processing background service unit (6): the system is used for analyzing the monitoring field of the software container, extracting a test data processing result, returning the test data processing result to the platform management database (4), and associating the test data processing result with a corresponding user sequence;
experimental data processing front end interface (8): for reading and/or downloading experimental data processing results.
6. The software container-based test data processing system of claim 5,
the user sequence is as follows:
the user stays in the memory before quitting the dispatch sequence distribution service front-end interface (7) or logging off the user;
after the experimental data processing is finished by using the software container, when the user exits from the scheduling sequence distribution service front-end interface (7), the scheduling sequence distribution service unit (5) automatically deletes the experimental data.
7. The software container-based test data processing system of claim 5 or 6, wherein the lightweight virtual service unit (2) is a docker development tool.
8. The software container-based experimental data processing system according to claim 5 or 6, characterized in that said lightweight virtual service unit (2) is further adapted to establish a private image library (3) comprising said plurality of virtualized images (31, 32, 33, 34).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911232506.4A CN111078358B (en) | 2019-12-05 | 2019-12-05 | Test data processing method and system based on software container |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911232506.4A CN111078358B (en) | 2019-12-05 | 2019-12-05 | Test data processing method and system based on software container |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111078358A true CN111078358A (en) | 2020-04-28 |
| CN111078358B CN111078358B (en) | 2023-09-05 |
Family
ID=70312977
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911232506.4A Active CN111078358B (en) | 2019-12-05 | 2019-12-05 | Test data processing method and system based on software container |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111078358B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090210873A1 (en) * | 2008-02-15 | 2009-08-20 | International Business Machines Corporation | Re-tasking a managed virtual machine image in a virtualization data processing system |
| CN106790483A (en) * | 2016-12-13 | 2017-05-31 | 武汉邮电科学研究院 | Hadoop group systems and fast construction method based on container technique |
| CN107294772A (en) * | 2017-05-23 | 2017-10-24 | 甘肃万维信息技术有限责任公司 | One kind combines Docker and realizes dynamic management and monitoring service system |
| CN107733704A (en) * | 2017-09-29 | 2018-02-23 | 中国石油化工股份有限公司 | A kind of system and method for the exploration and development cloud based on virtualization and container technique |
| CN108415795A (en) * | 2018-02-12 | 2018-08-17 | 人和未来生物科技（长沙）有限公司 | A kind of container Dockerfile, container mirror image rapid generation and system |
| CN109656682A (en) * | 2018-12-03 | 2019-04-19 | 中国石油化工股份有限公司 | A kind of system and method for the exploration and development big data processing platform based on container technique |
| CN110008286A (en) * | 2019-03-26 | 2019-07-12 | 华南理工大学 | A kind of injection molding equipment big data acquisition and storage system and method |
-
2019
- 2019-12-05 CN CN201911232506.4A patent/CN111078358B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090210873A1 (en) * | 2008-02-15 | 2009-08-20 | International Business Machines Corporation | Re-tasking a managed virtual machine image in a virtualization data processing system |
| CN106790483A (en) * | 2016-12-13 | 2017-05-31 | 武汉邮电科学研究院 | Hadoop group systems and fast construction method based on container technique |
| CN107294772A (en) * | 2017-05-23 | 2017-10-24 | 甘肃万维信息技术有限责任公司 | One kind combines Docker and realizes dynamic management and monitoring service system |
| CN107733704A (en) * | 2017-09-29 | 2018-02-23 | 中国石油化工股份有限公司 | A kind of system and method for the exploration and development cloud based on virtualization and container technique |
| CN108415795A (en) * | 2018-02-12 | 2018-08-17 | 人和未来生物科技（长沙）有限公司 | A kind of container Dockerfile, container mirror image rapid generation and system |
| CN109656682A (en) * | 2018-12-03 | 2019-04-19 | 中国石油化工股份有限公司 | A kind of system and method for the exploration and development big data processing platform based on container technique |
| CN110008286A (en) * | 2019-03-26 | 2019-07-12 | 华南理工大学 | A kind of injection molding equipment big data acquisition and storage system and method |
Non-Patent Citations (2)
| Title |
|---|
| 谭宏楠;石京燕;邹佳恒;杜然;姜晓巍;孙震宇;: "应用于JUNO实验的容器技术研究" * |
| 赵乐乐;黄刚;马越;: "基于Docker的Hadoop平台架构研究" * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111078358B (en) | 2023-09-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7802248B2 (en) | Managing a service having a plurality of applications using virtual machines | |
| CN105893083B (en) | Mobile code unloading support system and its discharging method under cloud environment based on container | |
| US20190250946A1 (en) | Migrating a software container taking into account resource constraints | |
| CN109032806A (en) | The service scheduling method and device of container | |
| US20180189109A1 (en) | Management system and management method for computer system | |
| CN110098946B (en) | Method and device for deploying virtualized network element equipment | |
| CN103051710A (en) | Virtual cloud platform management system and method | |
| CN110417613A (en) | Distributed performance test method, device, equipment and storage medium based on Jmeter | |
| CN111399970B (en) | Reserved resource management method, device and storage medium | |
| CN113641457A (en) | Container creation method, device, apparatus, medium, and program product | |
| CN111221618B (en) | Deployment method and device for containerized virtual network function | |
| US10025630B2 (en) | Operating programs on a computer cluster | |
| CN109697159A (en) | A kind of development of automation operational system based on mixing cloud platform | |
| US20220179711A1 (en) | Method For Platform-Based Scheduling Of Job Flow | |
| CN111857951A (en) | Containerized deployment platform and deployment method | |
| CN107463357A (en) | Task scheduling system, dispatching method, Simulation of Brake system and emulation mode | |
| CN115242752B (en) | Address allocation method, device, equipment and medium of battery management system | |
| CN112235373A (en) | Cloud desktop scheduling system based on application AI and scheduling method thereof | |
| JP4500090B2 (en) | Information management system and information management method | |
| US20220206836A1 (en) | Method and Apparatus for Processing Virtual Machine Migration, Method and Apparatus for Generating Virtual Machine Migration Strategy, Device and Storage Medium | |
| CN113438295A (en) | Container group address allocation method, device, equipment and storage medium | |
| US11194629B2 (en) | Handling expiration of resources allocated by a resource manager running a data integration job | |
| CN111078358A (en) | Test data processing method and system based on software container | |
| CN107885593B (en) | User authentication method and device | |
| US11360756B2 (en) | Processing unit subtype configuration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |