CN103220363A - Distributed network training resource management system based on cloud computing and scheduling method - Google Patents
Distributed network training resource management system based on cloud computing and scheduling method Download PDFInfo
- Publication number
- CN103220363A CN103220363A CN2013101523079A CN201310152307A CN103220363A CN 103220363 A CN103220363 A CN 103220363A CN 2013101523079 A CN2013101523079 A CN 2013101523079A CN 201310152307 A CN201310152307 A CN 201310152307A CN 103220363 A CN103220363 A CN 103220363A
- Authority
- CN
- China
- Prior art keywords
- module
- scheduling
- courseware
- server
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Computer And Data Communications (AREA)
Abstract
The invention discloses a distributed network training resource management system based on cloud computing and a scheduling method. The system comprises a server management module, a courseware access scheduling module, a report module, a system monitoring module, an interface management module and a system management module, wherein the server management module is used for managing the hardware server information of the management system; the courseware access scheduling module is used for performing distributed scheduling of system load balance, generating a dispatching log and providing basis for the report module; the report module is used for counting the information in distributed scheduling in the courseware access scheduling module; the system monitoring module is used for acquiring the server information and providing the scheduling basis for the courseware access scheduling module; the interface management module is used for providing the server monitoring information for the system monitoring module and providing courseware resource information for other modules; and the system management module is used for carrying out unified maintenance on parameter required by the operation of the system and configuration information required by service operation. According to the system, intensive management of network training resources can be realized, and the response speed of user access is improved under the condition that the system load balance requirement is met.
Description
Technical field
The present invention relates to field of computer technology, be specifically related to a kind of distributed network training resource management system and dispatching method based on cloud computing.
Background technology
In the prior art, can improve access efficiency and the fail safe that strengthens storage system by the distributed storage mode, but simultaneously with new problem---scheduling problem.
Therefore, the research to the load balancing strategy of distributed storage in recent years becomes focus, goes so far as the relative better dynamic load balancing of effect strategy from the load balancing strategy of static state.The static load balance policy needn't be considered the actual loading situation on each node server, as long as come allocating task according to pre-set scheme, the advantage applies of this algorithm is on the little and simplicity of its expense, but because the shared server resource of each task is not quite similar and is difficult to prediction, therefore can only realize load balancing to a certain extent, the result often can not be satisfactory.The dynamic load leveling strategy is considered the real-time load and the response condition of each node server, with the request priority allocation of newly coming in to the highest server of a certain evaluation of estimate, with this throughput and reduction response time of improving whole system, its effect is better than the static load balance policy relatively.
Existing online training resource management system causes the network resource consumption height because the dispatching algorithm performance is not good, and network resource utilization is low, haves much room for improvement.
Summary of the invention
The invention provides a kind of distributed network training resource management system and dispatching method based on cloud computing, the response speed that improves user capture under the prerequisite of system load balancing can satisfied, and the overall the best of realization is visited nearby under the prerequisite of proof load equilibrium.
A kind of distributed network training resource management system based on cloud computing of the present invention comprises:
Server management module is used for the hardware server information of management system, for courseware access scheduling module provides server info;
Courseware access scheduling module is used for the distributed scheduling of system load balancing, generates dispatching log, for Reports module statistical server operating position and resource operating position provide foundation;
Reports module is used for the information of described statistics courseware access scheduling module distribution formula scheduling, and the data of optimization are provided for described courseware access scheduling module;
The system monitoring module is used to obtain the session number and the memory information of server memory, storage, cpu busy percentage and application server, for described courseware access scheduling module provides the scheduling foundation;
The interface management module is used to described system monitoring module that server monitoring information is provided and provides the courseware resource information for other module;
System management module is used to provide system's operation required parameter and the required configuration information of work flow is unified to safeguard, for other module provides the authority configuration.
Preferably, described courseware access scheduling module comprises: policy configurations module, distributed scheduling module, dispatching log module, wherein,
The policy configurations module is used for the direction of the overweight time scheduling migration of configuration load;
The distributed scheduling module is used for configuration schedules value computing formula and scheduling decision algorithm, and utilizes described policy configurations module to start scheduling, and provides data for the dispatching log module;
The dispatching log module is used for writing down the information of scheduling time that the distributed scheduling module produces, URL, access time, resource name.
Preferably, described distributed scheduling module comprises: scheduler module, load balancing policy data generation module; Wherein,
Scheduler module is used for directly visit being dispatched, and is responsible for the request first of user capture is responded, and described response comprises tabling look-up and visit redirect operation IP storehouse and load balancing policy data;
Load balancing policy data generation module is used to generate the load balancing policy data, is used to described scheduler module that data dispatching is provided.
Preferably, when described load balancing policy data generation module generates data, with in the setting-up time from the visit quantity statistics of each QueryUnit as the foundation that generates data, described statistics is to be responsible for collection by described Reports module.
The present invention also provides a kind of distributed network training resource management system dispatching method based on cloud computing, adopts said system, and described system call flow process comprises the steps:
501: the IP address that obtains visit;
502: according to described IP address lookup IP storehouse;
503: query load balance policy data;
504: carry out redirection process.
Preferably, described IP library storage describedly comprises according to described IP address lookup IP storehouse in Hash table: input IP address, output districts and cities id.
Preferably, described load balancing policy data is stored on the one-dimension array, the corresponding districts and cities id of the subscript of this array.
Technique scheme as can be seen, because the embodiment of the invention adopts distributed network training resource management system and dispatching method based on cloud computing, can satisfy the response speed that improves user capture under the prerequisite of system load balancing, the overall the best of realization is visited nearby under the prerequisite of proof load equilibrium.
Description of drawings
In order to be illustrated more clearly in the embodiment of the invention or technical scheme of the prior art, to do to introduce simply to the accompanying drawing of required use in embodiment or the description of the Prior Art below, apparently, accompanying drawing in describing below only is some embodiments of the present invention, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain other accompanying drawing according to these accompanying drawings.
Fig. 1 is the general schematic view of the distributed network training resource management system of cloud computing of the present invention;
Fig. 2 is the concrete structure schematic diagram of the distributed network training resource management system of cloud computing of the present invention;
Fig. 3 is the distributed scheduling modular structure schematic diagram in the system of the present invention;
Fig. 4 is the handling process schematic diagram of the distributed scheduling module in the system of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the invention, the technical scheme in the embodiment of the invention is clearly and completely described, obviously, described embodiment only is the present invention's part embodiment, rather than whole embodiment.Based on the embodiment among the present invention, those of ordinary skills belong to the scope of protection of the invention not making all other embodiment that obtained under the creative work prerequisite.
The embodiment of the invention provides a kind of distributed network training resource management system and dispatching method based on cloud computing, can satisfy the response speed that improves user capture under the prerequisite of system load balancing, the overall the best of realization is visited nearby under the prerequisite of proof load equilibrium.Below be elaborated respectively.
Fig. 1 and Fig. 2 have described the structure of system of the present invention.Wherein Fig. 1 is the general schematic view of system, and Fig. 2 is the concrete schematic diagram of the functional structure of system.
Distributed network training resource management system of the present invention is made up of server management module, courseware access scheduling module, system monitoring module, interface management module, Reports module and system management module.Wherein:
Server management module is used for the hardware server information of management system, for courseware access scheduling module provides server info.
Courseware access scheduling module is used for the distributed scheduling of system load balancing, generates dispatching log, for Reports module statistical server operating position and resource operating position provide foundation.
Courseware access scheduling module is made up of policy configurations module, distributed debugging module, dispatching log module.
The policy configurations module is used for the direction of the overweight time scheduling migration of configuration load;
The distributed scheduling module is used for configuration schedules value computing formula and scheduling decision algorithm, and utilizes the policy configurations module to start scheduling, and provides data for the dispatching log module;
The dispatching log module is used for writing down information such as scheduling time that the distributed scheduling module produces, URL, access time, resource name.
Reports module is used for adding up the relevant information of courseware access scheduling module distribution formula scheduling, and the data of optimization are provided for course access scheduling module.
The system monitoring module is used to obtain the information such as session number, JVM internal memory of server memory, storage, cpu busy percentage and application server, for courseware access scheduling module provides the scheduling foundation.
The interface management module, being mainly used in to the system monitoring module provides relevant traffic device monitor message and provides the courseware resource related information for other module.
System management module is mainly used in to provide system's operation required parameter and the required relevant configuration information of work flow is unified to safeguard, for other module provides the authority configuration.
As shown in Figure 3, be distributed scheduling modular structure schematic diagram in the courseware access scheduling module.
The distributed scheduling module is made up of scheduler module and load balancing data generation module.
Wherein, scheduler module is used for directly visit being dispatched, and is responsible for the request first of user capture is responded.This response mainly is meant tabling look-up and visit redirect operation IP storehouse and load balancing policy data.
Load balancing strategy generation module generates the load balancing policy data, is used to scheduler module that data dispatching is provided.When load balancing policy data generation module generates data, need with in a period of time from the visit quantity statistics of each QueryUnit (value defined of " districts and cities' name ") as the foundation that generates data.This statistics is responsible for collecting by Reports module.Courseware visit situation statistical module in the Reports module must write down the visit situation to system at any time, is used as adding up foundation with the QueryUnit under this visit.Statistics must be stored in the database, and provides real time data for load balancing module.
As shown in Figure 4, be the handling process schematic diagram of distributed scheduling module in the system of the present invention, this handling process is divided into four-stage, is respectively to obtain client IP address, inquiry IP storehouse, query load balance policy data and redirection process.
(1) obtains client IP address;
Obtain client IP address and be the system function that simple calling platform system provides, can utilize existing techniques in realizing, in this superfluous words no longer.Below three processes that selective analysis is remaining how to realize.
(2) inquiry IP storehouse;
In order to make inquiry more efficient, the IP library storage in the Hash table.The type definition of Hash table key is the unsigned int of 4 bytes, and value be defined as pair＜char＞, the element among the pair is represented districts and cities id respectively.In pair, why adopt char and do not adopt int to be, use the char of 1 byte can satisfy the demands fully because districts and cities' quantity is fewer.System realizes adopting ready-made C++ STL type hash_map＜unsigned int, pair＜char＞＞, adopt this data structure can make the efficient of single query manipulation reach O (1) in theory.The input and output of inquiry IP storehouse process are as follows:
Input: IP address.
Output: districts and cities id is QueryUnit.
Member function find () or operator overloading function [] () that in fact this process calls a hash_map just can solve.
Because each scheduler module that starts promptly is to start a cgi script, all will use above-mentioned Hash table in this program at every turn.It is obviously unrealistic to be written into this Hash table in each program, this Hash table can be written in one section shared drive in advance, makes it need be used its cgi script to be shared by all.
(3) query load balance policy data;
Finish the process in above-mentioned inquiry IP storehouse, just can enter query load balance policy data phase.The process in the process of query load balance policy data and above-mentioned inquiry IP storehouse is similar a bit.Inquire about these data for convenience, with this storage on the one-dimension array of a vector, the corresponding districts and cities id of the subscript of this array.Therefore, behind the QueryUnit that has obtained under the visit, just can obtain a vector according to it, what store among this vector is several Resource Server IP address lists.Remaining work is exactly to try to achieve to choose a specific Resource Server IP address in this vector.
Wherein, the target of problem of load balancing may be defined as follows:
(1) all requests of each QueryUnit all must obtain distributing.
(2) summation that is assigned to the request quantity of each server all can not surpass its heap(ed) capacity.
(3) make the average retardation minimum of all requests.
For above problem, use greedy algorithm or linear programming algorithm to solve.The algorithm of computing module must could move under several data necessary.Mainly contain following several data.The first, each QueryUnit access number quantitative statistics data that the service access server reports.The second, the Resource Server status data.The 3rd, the network delay data between the districts and cities.
(4) redirection process;
Similar with the IP database data, the load balancing policy data neither load once when each modular program starts, but is stored in the shared drive.These data need be loaded once when whole system starts, and also need to reload in Data Update one time.
From technique scheme as can be seen, technical solution of the present invention can realize the intensive management of online training resource, save network bandwidth resources, improve user concurrent quantity, satisfying the response speed that improves user capture under the prerequisite of system load balancing, the overall the best of realization is visited nearby under the prerequisite of proof load equilibrium
One of ordinary skill in the art will appreciate that all or part of step in the whole bag of tricks of the foregoing description is to instruct relevant hardware to finish by program, this program can be stored in the computer-readable recording medium, storage medium can comprise: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), disk or CD etc.
More than to technical scheme that the embodiment of the invention provided, be described in detail, used specific case herein principle of the present invention and execution mode are set forth, the explanation of above embodiment just is used for helping to understand method of the present invention and core concept thereof; Simultaneously, for one of ordinary skill in the art, according to thought of the present invention, the part that all can change in specific embodiments and applications, in sum, this description should not be construed as limitation of the present invention.
Claims (7)
1. the distributed network training resource management system based on cloud computing is characterized in that, comprising:
Server management module is used for the hardware server information of management system, for courseware access scheduling module provides server info;
Courseware access scheduling module is used for the distributed scheduling of system load balancing, generates dispatching log, for Reports module statistical server operating position and resource operating position provide foundation;
Reports module is used for adding up the information of described courseware access scheduling module distribution formula scheduling, and the data of optimization are provided for described courseware access scheduling module;
The system monitoring module is used to obtain the session number and the memory information of server memory, storage, cpu busy percentage and application server, for described courseware access scheduling module provides the scheduling foundation;
The interface management module is used to described system monitoring module that server monitoring information is provided and provides the courseware resource information for other module.
System management module is used to provide system's operation required parameter and the required configuration information of work flow is unified to safeguard, for other module provides the authority configuration.
2. the distributed network training resource management system based on cloud computing according to claim 1 is characterized in that:
Described courseware access scheduling module comprises: policy configurations module, distributed scheduling module, dispatching log module, wherein,
The policy configurations module is used for the direction of the overweight time scheduling migration of configuration load;
The distributed scheduling module is used for configuration schedules value computing formula and scheduling decision algorithm, and utilizes described policy configurations module to start scheduling, and provides data for the dispatching log module;
The dispatching log module is used for writing down the information of scheduling time that the distributed scheduling module produces, URL, access time, resource name.
3. the distributed network training resource management system based on cloud computing according to claim 2 is characterized in that:
Described distributed scheduling module comprises: scheduler module, load balancing policy data generation module; Wherein,
Scheduler module is used for directly visit being dispatched, and is responsible for the request first of user capture is responded, and described response comprises tabling look-up and visit redirect operation IP storehouse and load balancing policy data;
Load balancing policy data generation module is used to generate the load balancing policy data, is used to described scheduler module that data dispatching is provided.
4. the distributed network training resource management system based on cloud computing according to claim 3 is characterized in that:
When described load balancing policy data generation module generates data, with in the setting-up time from the visit quantity statistics of each QueryUnit as the foundation that generates data, described statistics is to be responsible for collection by described Reports module.
5. distributed network training resource management system dispatching method based on cloud computing is characterized in that:
Adopt the described system of claim 1, described system call flow process comprises the steps:
501: the IP address that obtains visit;
502: according to described IP address lookup IP storehouse;
503: query load balance policy data;
504: carry out redirection process.
6. the distributed network training resource management system dispatching method based on cloud computing according to claim 5 is characterized in that:
Described IP library storage describedly comprises according to described IP address lookup IP storehouse in Hash table: input IP address, output districts and cities id.
7. the distributed network training resource management system dispatching method based on cloud computing according to claim 5 is characterized in that:
Described load balancing policy data is stored on the one-dimension array, the corresponding districts and cities id of the subscript of this array.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2013101523079A CN103220363A (en) | 2013-04-26 | 2013-04-26 | Distributed network training resource management system based on cloud computing and scheduling method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2013101523079A CN103220363A (en) | 2013-04-26 | 2013-04-26 | Distributed network training resource management system based on cloud computing and scheduling method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103220363A true CN103220363A (en) | 2013-07-24 |
Family
ID=48817814
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2013101523079A Pending CN103220363A (en) | 2013-04-26 | 2013-04-26 | Distributed network training resource management system based on cloud computing and scheduling method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103220363A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104468852A (en) * | 2013-09-18 | 2015-03-25 | 腾讯科技（北京）有限公司 | Method, device and system for client to select IP link address |
| CN111147325A (en) * | 2019-12-18 | 2020-05-12 | 深圳市任子行科技开发有限公司 | Distributed network information acquisition system, method and computer readable storage medium |
| CN112804164A (en) * | 2021-04-01 | 2021-05-14 | 北京每日优鲜电子商务有限公司 | Flow information generation method and device, electronic equipment and computer readable medium |
| CN113837908A (en) * | 2021-09-26 | 2021-12-24 | 北京永信至诚科技股份有限公司 | Course-based network training system and method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060228689A1 (en) * | 2005-04-12 | 2006-10-12 | Rajaram Kishore K | Interactive tutorial system and method |
| CN102063818A (en) * | 2010-08-12 | 2011-05-18 | 华东交通大学 | Experimental cloud platform system for serving computer-and-software-based education in schools of higher education |
-
2013
- 2013-04-26 CN CN2013101523079A patent/CN103220363A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060228689A1 (en) * | 2005-04-12 | 2006-10-12 | Rajaram Kishore K | Interactive tutorial system and method |
| CN102063818A (en) * | 2010-08-12 | 2011-05-18 | 华东交通大学 | Experimental cloud platform system for serving computer-and-software-based education in schools of higher education |
Non-Patent Citations (2)
| Title |
|---|
| 徐盈: "即时通信教育平台中负载均衡技术的研究与实现", 《信息科技辑》, 15 September 2007 (2007-09-15) * |
| 李保新: "市级会计培训中心集群系统的构建", 《信息科技辑》, 15 March 2013 (2013-03-15) * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104468852A (en) * | 2013-09-18 | 2015-03-25 | 腾讯科技（北京）有限公司 | Method, device and system for client to select IP link address |
| CN111147325A (en) * | 2019-12-18 | 2020-05-12 | 深圳市任子行科技开发有限公司 | Distributed network information acquisition system, method and computer readable storage medium |
| CN112804164A (en) * | 2021-04-01 | 2021-05-14 | 北京每日优鲜电子商务有限公司 | Flow information generation method and device, electronic equipment and computer readable medium |
| CN113837908A (en) * | 2021-09-26 | 2021-12-24 | 北京永信至诚科技股份有限公司 | Course-based network training system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109218355B (en) | Load balancing engine, client, distributed computing system and load balancing method | |
| Desai et al. | A survey of various load balancing techniques and challenges in cloud computing | |
| US9201686B2 (en) | Task execution by idle resources in grid computing system | |
| US9736043B2 (en) | Optimization of resource utilization in a collection of devices | |
| CN102917077A (en) | Resource allocation method in cloud computing system | |
| CN105159775A (en) | Load balancer based management system and management method for cloud computing data center | |
| Nazir et al. | Cuckoo optimization algorithm based job scheduling using cloud and fog computing in smart grid | |
| Ma et al. | Dynamic task scheduling in cloud computing based on greedy strategy | |
| CN111666131A (en) | Load balancing distribution method and device, computer equipment and storage medium | |
| CN103220363A (en) | Distributed network training resource management system based on cloud computing and scheduling method | |
| CN104112049A (en) | P2P (peer-to-peer) architecture based cross-data-center MapReduce task scheduling system and P2P architecture based cross-data-center MapReduce task scheduling method | |
| CN104144202A (en) | Hadoop distributed file system access method, system and device | |
| Remesh Babu et al. | Service‐level agreement–aware scheduling and load balancing of tasks in cloud | |
| Liu et al. | Cost-effective service provisioning for hybrid cloud applications | |
| US8032636B2 (en) | Dynamically provisioning clusters of middleware appliances | |
| Ashraf et al. | Smart grid management using cloud and fog computing | |
| Lin et al. | Novel resource allocation model and algorithms for cloud computing | |
| El-Zoghdy | A hierarchical load balancing policy for grid computing environment | |
| Lu et al. | A hybrid dynamic load balancing approach for cloud storage | |
| Vashisht et al. | Efficient dynamic replication algorithm using agent for data grid | |
| CN104683480A (en) | Distribution type calculation method based on applications | |
| Tudoran et al. | Sage: geo-distributed streaming data analysis in clouds | |
| Zhang et al. | Resilient workload manager: taming bursty workload of scaling internet applications | |
| Priyanka et al. | Comparative analysis on Virtual Machine assignment algorithms | |
| Hsu et al. | Effective memory reusability based on user distributions in a cloud architecture to support manufacturing ubiquitous computing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| DD01 | Delivery of document by public notice |
Addressee: GPGC Training and Evaluation Center Document name: Notification that Application Deemed to be Withdrawn |
|
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20130724 |