CN102436399A - Load-balanced acquisition method - Google Patents
Load-balanced acquisition method Download PDFInfo
- Publication number
- CN102436399A CN102436399A CN2011102157973A CN201110215797A CN102436399A CN 102436399 A CN102436399 A CN 102436399A CN 2011102157973 A CN2011102157973 A CN 2011102157973A CN 201110215797 A CN201110215797 A CN 201110215797A CN 102436399 A CN102436399 A CN 102436399A
- Authority
- CN
- China
- Prior art keywords
- end processor
- acquisition
- passage
- real
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Multi Processors (AREA)
Abstract
The invention discloses a load-balanced acquisition method. According to the load-balanced acquisition method, a real-time database and a plurality of front end processors are arranged; and acquisition points are connected with the front end processors through channels. The acquisition method for the load balance comprises the following steps of: firstly, establishing a network connection between each front end processor and the real-time database; secondly, configuring an acquisition system and allocating the channels for different acquisition front end processors; thirdly, when the front end processors are started, loading all channel information and information of the acquisition points in a local machine and simultaneously establishing main channels and slave channels; fourthly, starting acquisition tasks according to the configuration file information and establishing the connection of main channels by the front end processors; and fifthly, receiving acquired information transmitted by the acquisition points through the main channels and transmitting the acquired information to corresponding data points of the real-time database by the front end processors. According to the load-balanced acquisition method disclosed by the invention, by dynamically allocating acquisitioning tasks, the utilization rate of a computer can be increased; meanwhile, the acquisition efficiency and the response speed of the real-time database issuing a command can be increased.
Description
Technical field
The invention belongs to automatically-monitored technical field, especially carry out the large-scale synthesis supervisory system of many monitoring of toolss, relate to a kind of acquisition method of load balancing for needs.
Background technology
The data acquisition modes of large-scale synthesis supervisory system is the front end processor of two of employings or many redundancies, and each front end processor is from subsequent use each other state, and every front end processor is handled all communication tasks.The scope of monitoring along with supervisory system constantly enlarges, and the kind and the quantity of the equipment of monitoring constantly increase, and quantity of information is increasing, and the performance requirement of every front end processor is just increasingly high.Many front end processors just are in subsequent use each other state now, can not be fully utilized, and therefore, cause processing speed also slow; Two, the configuration file between the data point is kept in the real-time data base in collection point and the real-time data base, takies the storage space of real-time data base, causes response speed slack-off; Three, when the collection circuit goes wrong, can't acquisition tasks be shifted, so acquisition tasks is compelled to stop the influence use.
Summary of the invention
The present invention is in order to solve the front end processor wasting of resources in the existing supervisory system, and the problem that processing speed is slow provides a kind of acquisition method of load balancing, makes full use of existing front end processor device, improves processing speed.
In order to solve the problems of the technologies described above, the present invention adopts following technical scheme to be achieved:
A kind of acquisition method of load balancing comprises real-time data base and a plurality of front end processor, and the collection point is connected with front end processor through passage, and the acquisition method of described load balancing may further comprise the steps:
(1), the network of setting up each front end processor and real-time data base is connected;
(2), the configuration acquisition system, set up the association between the data point in collection point and the real-time data base, and the form of related information with configuration file be saved in all front end processors; And according to the collection of each passage count, frequency acquisition, the type of collecting device and the type of tunneling traffic carry out the calculating of weighted value, according to result of calculation with channel allocation to the different acquisition front end processor;
(3), front end processor loads the channel informations all in this machine and the information of collection point when starting, and be saved in the internal memory, the passage that will be assigned to this machine simultaneously is made as the main channel, remaining passage is made as from passage;
(4), gather to start the back front end processor according to profile information, start the acquisition tasks of main channel, set up being connected of main channel and collection point;
(5), front end processor receives the information of the collection of sending the collection point through the main channel, and converts Information Monitoring the form of real-time data base data point into according to profile information, be sent to then in the corresponding data point of real-time data base.
Further, also comprise step (6), in the front end processor operational process passage is dynamically adjusted.
Further again; The dynamic adjustment of passage described in the step (6) comprises: according to the type of the load information and the passage of front end processor, calculate each passage task CPU, memory usage and tunneling traffic type weighted index together; Carry out readjusting of passage task; Close some the passage tasks on the high front end processor of load, transfer on the low front end processor of load, on the low front end processor of load, start corresponding passage acquisition tasks and carry out data acquisition.The weighted index of tunneling traffic type is that the type (serial communication, TCP client, TCPserver etc.) according to passage is set.
Further again; Real-time data base can also be controlled the collection point; In step (2), also comprise during the configuration acquisition system: be registered to the information of the data point that is associated with controlled collection point in the real-time data base, described controlled collection point is for accepting the collection point of real-time data base control.
Further; In step (5); Real-time data base sends the steering order of the controlled collection point of control to all front end processors, and with the front end processor processing controls instruction that this controlled collection point connects, remaining front end processor returns controlled collection point not in this machine information processed.
In order to prevent when an acquisition tasks goes wrong; Can its front end processor of transferring to other from current front end processor be handled; Also comprise step (7), between front end processor, set up redundant, the main channel the data sync of gathering to being from the computing machine of passage.
In order to prevent when an acquisition tasks goes wrong; Do not influence other acquisition tasks processing; Said step also comprises in (4): for the acquisition tasks of each passage, start one independently thread handle, when an acquisition tasks goes wrong; This acquisition tasks is transferred on other the collection front end processor and handled, close the acquisition tasks that is gone wrong simultaneously.
Further again, the acquisition method of this load balancing also comprises constantly communication, the step of the status information of front end processor synchronously between the front end processor.Through synchronous information, can gather the judgement and the processing of the state between the front end processor.
Wherein, described status information comprises the load condition of front end processor and the state between each passage task.
Compared with prior art; Advantage of the present invention and good effect are: one, the unity of form of the related information between the data point of collection point and real-time data base with configuration file is kept in all front end processors; When gathering operation; Real-time data base need not to know the collection point information of front end processor, no matter is to receive the information of collection point collection or to the collection point sending controling instruction, pass through the processing of front end processor and transmit getting final product; Two, all front end processors are distributed in the acquisition tasks equilibrium, made full use of existing front end processor resource, improved processing speed; Three, dynamic load regulatory function in time with the adjustment of passage task, has improved the computing machine utilization factor; Four, set up redundancy between the front end processor, when acquisition tasks goes wrong, in time launch other front end processor processing collected tasks, guaranteed the continuity of acquisition tasks.
After the detailed description in conjunction with the advantages embodiment of the present invention, other characteristics of the present invention and advantage will become clearer.
Description of drawings
Fig. 1 is a kind of embodiment block diagram of the acquisition method of a kind of load balancing proposed by the invention.
Embodiment
Do explanation in further detail below in conjunction with the accompanying drawing specific embodiments of the invention.
Embodiment one, referring to shown in Figure 1, and the acquisition method of the load balancing of present embodiment; Comprise real-time data base 1 and a plurality of front end processors 2; Said front end processor 2 is connected collection point A, B through real time data bank interface 3 with real-time data base 1; C ... Be connected with front end processor 2 through passage, the acquisition method of the load balancing of present embodiment may further comprise the steps:
S1, the network of setting up each front end processor and real-time data base are connected;
Wherein, Comprise a plurality of front end processors 2 and a real-time data base in the system; Said front end processor receives the information of gathering from the collection point, and is sent to the real-time data base preservation after doing handled, and it can be that the cable network connection also can be that wireless network connects that described network connects.
S2, configuration acquisition system are set up the association between the data point in collection point and the real-time data base, and the form of related information with configuration file is saved in all front end processors; And according to the collection of each passage count, frequency acquisition, the type of collecting device and the type of tunneling traffic carry out the calculating of weighted value; According to result calculated with being assigned on the different acquisition front end processor of channel-equalization, thereby when front end processor starts, can reach initial equilibrium.The calculating of channel weighting value be exactly through the collection of passage count, resource occupation number during the type estimating channel task run of the type of frequency acquisition, collecting device and tunneling traffic.Described channel type comprises serial communication, TCP client, TCPserver etc., and the weighted index of said tunneling traffic type is to set according to the type of passage.
Wherein, The Information Monitoring of each collection point all has corresponding storage space in real-time data base; For the data that can make collection are effectively preserved, therefore, when configuration, at first set up the association between the data point in collection point and the real-time data base; Related information saving becomes configuration file, and then this configuration file is saved in all front end processors.The configuration information of collection point and real-time data base point is kept in the configuration file of collection, and real-time data base need not known the collection point of actual correspondence, and the data based profile information that the collection front end processor directly will be gathered when operation is sent in the real-time data base.
In addition; Owing to a plurality of collection points are arranged, and in the present supervisory system many front end processors are arranged generally, in order to make full use of existing resources; With the data acquisition Task Distribution to different front end processors; Therefore, present embodiment distributes acquisition tasks according to above-mentioned result calculated, also is about to acquisition tasks and distributes.
S3, front end processor load the channel informations all in this machine and the information of collection point when starting, and are saved in the internal memory, and the passage that will be assigned to this machine simultaneously is made as the main channel, and remaining passage is made as from passage;
This step is the operation that front end processor is carried out when starting, and clear and definite acquisition tasks, and the passage of this machine of being assigned to is made as the main channel, and remaining passage is made as from passage.
S4, gather to start the back front end processor, start the acquisition tasks of main channel, set up being connected of main channel and collection point according to profile information;
S5, front end processor receive the information of the collection of sending the collection point through the main channel, and convert Information Monitoring the form of real-time data base data point into according to profile information, are sent to then in the corresponding data point of real-time data base.
Because front end processor is when initial setting up, go out weighted value according to the empirical Calculation of history, the passage task reasonably has been assigned on the different front end processors; At that time in operational process and since on-the-spot operation can not the property estimated, when the load of the passage task on a front end processor reaches a limit value; Will influence the operation of collection; This is the dynamic assignment that just need carry out acquisition tasks, reaches the purpose that makes full use of front end processor through dynamic assignment, has improved stability simultaneously.
Therefore, present embodiment also comprises step S6, in the front end processor operational process passage is dynamically adjusted.
As a concrete embodiment; The dynamic adjustment of passage described in the step S6 comprises: according to the type of the load information and the passage of front end processor, calculate each passage task CPU usage and channel type weighted index together; Carry out readjusting of passage task; Close some the passage tasks on the high front end processor of load, transfer on the low front end processor of load, on the low front end processor of load, start corresponding passage acquisition tasks and carry out data acquisition.Present embodiment has improved the computing machine utilization factor through carrying out load regulation dynamically.
Real-time data base can also be controlled the collection point; In step S2; Also comprise during the configuration acquisition system: be registered to the information of the data point that is associated with controlled collection point in the real-time data base, described controlled collection point is for accepting the collection point of real-time data base control.Therefore, in step S5, real-time data base sends the steering order of the controlled collection point of control to all front end processors, and with the front end processor processing controls instruction that this controlled collection point connects, remaining front end processor returns controlled collection point not in this machine information processed.
In order to prevent when an acquisition tasks goes wrong; Can its front end processor of transferring to other from current front end processor be handled; Also comprise step S7, between front end processor, set up redundant, the main channel the data sync of gathering to being from the computing machine of passage.
In order to prevent when an acquisition tasks goes wrong; Do not influence other acquisition tasks processing; Also comprise among the said step S4: for the acquisition tasks of each passage, start one independently thread handle, when an acquisition tasks goes wrong; This acquisition tasks is transferred on other the collection front end processor and handled, close the acquisition tasks that is gone wrong simultaneously.
The acquisition method of this load balancing also comprises constantly communication, the step of the status information of front end processor synchronously between the front end processor.Wherein, described status information comprises the load condition of front end processor and the state between each passage task.Through synchronous information, can gather the judgement and the processing of the state between the front end processor.
Present embodiment is through moving the passage Task Distribution on different front end processors, each is gathered front end processor and all handles the order of oneself, through the equally loaded of passage task, can improve the execution efficient of order, improves the execution speed of order.
Certainly; Above-mentioned explanation is not to be limitation of the present invention; The present invention also is not limited in above-mentioned giving an example, and variation, remodeling, interpolation or replacement that those skilled in the art are made in essential scope of the present invention also should belong to protection scope of the present invention.
Claims (9)
1. the acquisition method of a load balancing is characterized in that, comprises real-time data base and a plurality of front end processor, and the collection point is connected with front end processor through passage, and the acquisition method of described load balancing may further comprise the steps:
(1), the network of setting up each front end processor and real-time data base is connected;
(2), the configuration acquisition system, set up the association between the data point in collection point and the real-time data base, and the form of related information with configuration file be saved in all front end processors; And according to the collection of each passage count, frequency acquisition, the type of collecting device and the type of tunneling traffic carry out the calculating of weighted value, according to result of calculation with channel allocation to the different acquisition front end processor;
(3), front end processor loads the channel informations all in this machine and the information of collection point when starting, and be saved in the internal memory, the passage that will be assigned to this machine simultaneously is made as the main channel, remaining passage is made as from passage;
(4), gather to start the back front end processor according to profile information, start the acquisition tasks of main channel, set up being connected of main channel and collection point;
(5), front end processor receives the information of the collection of sending the collection point through the main channel, and converts Information Monitoring the form of real-time data base data point into according to profile information, be sent to then in the corresponding data point of real-time data base.
2. the acquisition method of a kind of load balancing according to claim 1 is characterized in that, also comprises step (6), in the front end processor operational process passage is dynamically adjusted.
3. the acquisition method of a kind of load balancing according to claim 2; It is characterized in that; The dynamic adjustment of passage described in the step (6) comprises: according to the type of the load information and the passage of front end processor, calculate each passage task CPU, memory usage and tunneling traffic type weighted index together; Carry out readjusting of passage task; Close some the passage tasks on the high front end processor of load, transfer on the low front end processor of load, on the low front end processor of load, start corresponding passage acquisition tasks and carry out data acquisition.
4. the acquisition method of a kind of load balancing according to claim 1; It is characterized in that; In step (2); Also comprise during the configuration acquisition system: be registered to the information of the data point that is associated with controlled collection point in the real-time data base, described controlled collection point is for accepting the collection point of real-time data base control.
5. the acquisition method of a kind of load balancing according to claim 4; It is characterized in that; In step (5); Real-time data base sends the steering order of the controlled collection point of control to all front end processors, and with the front end processor processing controls instruction that this controlled collection point connects, remaining front end processor returns controlled collection point not in this machine information processed.
6. the acquisition method of a kind of load balancing according to claim 1 is characterized in that, also comprises step (7), between front end processor, set up redundant, the main channel the data sync of gathering to being from the computing machine of passage.
7. the acquisition method of a kind of load balancing according to claim 6; It is characterized in that; Said step also comprises in (4): for the acquisition tasks of each passage, start one independently thread handle, when an acquisition tasks goes wrong; This acquisition tasks is transferred on other the collection front end processor and handled, close the acquisition tasks that is gone wrong simultaneously.
8. the acquisition method of a kind of load balancing according to claim 1 is characterized in that, also comprises constantly communication, the step of the status information of front end processor synchronously between the front end processor.
9. the acquisition method of a kind of load balancing according to claim 8 is characterized in that, described status information comprises the load condition of front end processor and the state between each passage task.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011102157973A CN102436399A (en) | 2011-07-29 | 2011-07-29 | Load-balanced acquisition method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011102157973A CN102436399A (en) | 2011-07-29 | 2011-07-29 | Load-balanced acquisition method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102436399A true CN102436399A (en) | 2012-05-02 |
Family
ID=45984478
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011102157973A Pending CN102436399A (en) | 2011-07-29 | 2011-07-29 | Load-balanced acquisition method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102436399A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104158878A (en) * | 2014-08-18 | 2014-11-19 | 浪潮(北京)电子信息产业有限公司 | Adaptive scheduling distributive monitoring data acquisition method and system |
| CN105227677A (en) * | 2015-10-20 | 2016-01-06 | 上海许继电气有限公司 | Based on the electric network fault information processing method of load balancing |
| CN105450713A (en) * | 2014-09-02 | 2016-03-30 | 阿里巴巴集团控股有限公司 | Front-end processor cluster deployment method, device and system |
| CN105527948A (en) * | 2015-12-11 | 2016-04-27 | 东北大学 | Large scale distributed data acquisition system and method based on industrial process |
| CN105868021A (en) * | 2016-03-23 | 2016-08-17 | 浪潮通信信息系统有限公司 | Task allocation method and apparatus |
| CN106528713A (en) * | 2016-10-26 | 2017-03-22 | 珠海格力电器股份有限公司 | Data acquisition method and intelligent communication gateway |
| CN107450980A (en) * | 2016-05-31 | 2017-12-08 | 西门子公司 | Run the method and automation equipment of automation equipment |
| CN109614228A (en) * | 2018-11-27 | 2019-04-12 | 南京轨道交通系统工程有限公司 | Comprehensively monitoring front-end system and working method based on dynamic load leveling mode |
| CN109981731A (en) * | 2019-02-15 | 2019-07-05 | 联想(北京)有限公司 | A kind of data processing method and equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101188599A (en) * | 2007-12-04 | 2008-05-28 | 四方电气(集团)有限公司 | Implementation method for load balance design of electric monitoring front system of power plant |
| CN201616710U (en) * | 2010-01-13 | 2010-10-27 | 山东电力集团公司 | Telescopic terminal concurrency access and load balancing system framework |
| CN201774310U (en) * | 2010-01-12 | 2011-03-23 | 广东雅达电子股份有限公司 | Remote electricity quantity metering device |
-
2011
- 2011-07-29 CN CN2011102157973A patent/CN102436399A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101188599A (en) * | 2007-12-04 | 2008-05-28 | 四方电气(集团)有限公司 | Implementation method for load balance design of electric monitoring front system of power plant |
| CN201774310U (en) * | 2010-01-12 | 2011-03-23 | 广东雅达电子股份有限公司 | Remote electricity quantity metering device |
| CN201616710U (en) * | 2010-01-13 | 2010-10-27 | 山东电力集团公司 | Telescopic terminal concurrency access and load balancing system framework |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104158878A (en) * | 2014-08-18 | 2014-11-19 | 浪潮(北京)电子信息产业有限公司 | Adaptive scheduling distributive monitoring data acquisition method and system |
| CN104158878B (en) * | 2014-08-18 | 2019-02-15 | 浪潮(北京)电子信息产业有限公司 | A kind of the distributed monitoring collecting method and system of adaptive scheduling |
| CN105450713B (en) * | 2014-09-02 | 2019-02-12 | 阿里巴巴集团控股有限公司 | The methods, devices and systems of clustered deploy(ment) front end processor |
| CN105450713A (en) * | 2014-09-02 | 2016-03-30 | 阿里巴巴集团控股有限公司 | Front-end processor cluster deployment method, device and system |
| CN105227677A (en) * | 2015-10-20 | 2016-01-06 | 上海许继电气有限公司 | Based on the electric network fault information processing method of load balancing |
| CN105527948A (en) * | 2015-12-11 | 2016-04-27 | 东北大学 | Large scale distributed data acquisition system and method based on industrial process |
| CN105527948B (en) * | 2015-12-11 | 2018-04-24 | 东北大学 | A kind of large-scale distributed data collecting system and method based on industrial process |
| CN105868021A (en) * | 2016-03-23 | 2016-08-17 | 浪潮通信信息系统有限公司 | Task allocation method and apparatus |
| CN107450980A (en) * | 2016-05-31 | 2017-12-08 | 西门子公司 | Run the method and automation equipment of automation equipment |
| CN106528713A (en) * | 2016-10-26 | 2017-03-22 | 珠海格力电器股份有限公司 | Data acquisition method and intelligent communication gateway |
| CN109614228A (en) * | 2018-11-27 | 2019-04-12 | 南京轨道交通系统工程有限公司 | Comprehensively monitoring front-end system and working method based on dynamic load leveling mode |
| CN109614228B (en) * | 2018-11-27 | 2023-04-18 | 南京轨道交通系统工程有限公司 | Comprehensive monitoring front-end system based on dynamic load balancing mode and working method |
| CN109981731A (en) * | 2019-02-15 | 2019-07-05 | 联想(北京)有限公司 | A kind of data processing method and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102436399A (en) | Load-balanced acquisition method | |
| CN105528330B (en) | The method, apparatus of load balancing is gathered together and many-core processor | |
| CN107534583B (en) | Method implemented in a management node and related apparatus | |
| CN103152393B (en) | A kind of charging method of cloud computing and charge system | |
| CN108845878A (en) | The big data processing method and processing device calculated based on serverless backup | |
| WO2016082370A1 (en) | Distributed node intra-group task scheduling method and system | |
| CN103401947A (en) | Method and device for allocating tasks to multiple servers | |
| CN103516744A (en) | A data processing method, an application server and an application server cluster | |
| CN103944997B (en) | In conjunction with the load-balancing method of random sampling and Intel Virtualization Technology | |
| CN104038540A (en) | Method and system for automatically selecting application proxy server | |
| CN104142860A (en) | Resource adjusting method and device of application service system | |
| CN104615498A (en) | Dynamic load balancing method of trunking system based on task immigration | |
| CN103699433A (en) | Method and system for performing dynamic adjustment on number of tasks in Hadoop platform | |
| CN105491138A (en) | Load rate based graded triggering distributed load scheduling method | |
| CN115334084A (en) | Cloud platform based on cloud computing and internet | |
| CN112860974A (en) | Computing resource scheduling method and device, electronic equipment and storage medium | |
| CN102510403B (en) | Receive and the cluster distributed system and method for real-time analysis for vehicle data | |
| CN106059940A (en) | Flow control method and device | |
| CN109614228B (en) | Comprehensive monitoring front-end system based on dynamic load balancing mode and working method | |
| CN105207856A (en) | Load balancing system and method based on SDN virtual switch | |
| US9363331B2 (en) | Data allocation method and data allocation system | |
| CN105260245A (en) | Resource scheduling method and device | |
| CN112866985A (en) | Flow control method, resource downloading method, device, equipment and storage medium | |
| US20170235288A1 (en) | Process control program, process control device, and process control method | |
| CN105359489A (en) | Method, device and system for scheduling resources |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20120502 |