CN109672731A - A kind of distributed node information monitoring method, system and application - Google Patents
A kind of distributed node information monitoring method, system and application Download PDFInfo
- Publication number
- CN109672731A CN109672731A CN201811545084.1A CN201811545084A CN109672731A CN 109672731 A CN109672731 A CN 109672731A CN 201811545084 A CN201811545084 A CN 201811545084A CN 109672731 A CN109672731 A CN 109672731A
- Authority
- CN
- China
- Prior art keywords
- node
- resource information
- child node
- information
- resource
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B7/00—Radio transmission systems, i.e. using radiation field
- H04B7/14—Relay systems
- H04B7/15—Active relay systems
- H04B7/185—Space-based or airborne stations; Stations for satellite systems
- H04B7/1851—Systems using a satellite or space-based relay
- H04B7/18519—Operations control, administration or maintenance
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04J—MULTIPLEX COMMUNICATION
- H04J3/00—Time-division multiplex systems
- H04J3/02—Details
- H04J3/06—Synchronising arrangements
- H04J3/0635—Clock or time synchronisation in a network
- H04J3/0638—Clock or time synchronisation among nodes; Internode synchronisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0805—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
- H04L43/0811—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking connectivity
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0805—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
- H04L43/0817—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking functioning
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
Abstract
The present invention realizes a kind of distributed node information monitoring method, system and application, belongs to technical field of data transmission.The described method includes: by the time synchronization of host node and each child node；Each child node acquires respective resource information according to predetermined period, and the collected resource information is sent to the host node in a manner of message-oriented middleware；The resource information that each child node is sent is stored in resource database by the host node, and periodically monitors each child node and its corresponding resource information in the resource database.By means of the invention it is possible to substantially shorten the time for monitoring large-scale distributed clustered node information, the task schedule response time is improved, promotes dispatching efficiency.
Description
Technical field
The present invention relates to a kind of large-scale distributed clustered node status information monitoring methods, belong to data transmission technology neck
Domain.
Background technique
With the development of information technology and the expansion of network size, remote sensing satellite data processing is handled by single node to be changed into
Distributed type assemblies processing.It is slow because obtaining clustered node information in Remote Sensing Ground Station data handling procedure, lead to task tune
Certain modules for requiring real-time high are made to lose data during degree system job, to directly affect entire scheduling system
Function and performance.There are mainly two types of mode, host node active poll and and heartbeat detections for node resources information acquisition at present.
1, host node active polling mode
This mode acquires child node resource information by host node cycle active polling mode in cluster.Which is maximum
The shortcomings that be exactly to need the continuous each child node of poll cluster of timing, overhead is larger；If if interval time is too long, collection
Group node susceptibility is lower, and the node status information of acquisition is likely to out-of-date information；It is needed if interval time is too short
A large amount of socket connections are created repeatedly, when clustered node quantity is larger, need to create each node corresponding socket company
It connects, continuous creation, deletion socket connection impact larger, occupancy host node system resource to whole system.
2, heartbeat detection
Heartbeat detection is as the result of poll but implementation has difference, and heartbeat needs host node to issue communication interface,
The connection of host node and child node is realized in a manner of webservice or socket, although this mode is compared to poll side
Formula alleviates the pressure of host node, but webservice and socket need communication handshake, if wherein a side is because of certain
Reason delay machine, data will need to attempt reconnection, just will disconnect connection after repeatedly having no result, this process in tcp level time-out
It needs to expend the regular hour, if how this mode of delay machine node can also have inefficiency, obtains nodal information time phase
To longer.
Summary of the invention
Aiming at the problems existing in the prior art, the present invention propose a kind of distributed node information monitoring method, system and
Using this method is without constantly creating, disconnecting, and also there is no ask because the acquisition nodal information time caused by delay machine is extended
Topic, can substantially shorten the time for obtaining large-scale distributed clustered node information, improve the task schedule response time, promoted and adjusted
Spend efficiency.
The technical scheme is that
A kind of distributed node information monitoring method, comprising:
By the time synchronization of host node and each child node；
Each child node acquires respective resource information according to predetermined period, and by the collected resource information in message
Between the mode of part be sent to the host node；
The resource information that each child node is sent is stored in resource database by the host node, and periodically
Monitor each child node and its corresponding resource information in the resource database.
In an alternative embodiment, the resource information that each child node is sent is stored in resource by the host node
In database, comprising:
The resource information that each child node is sent is stored in the resource database by the host node, and is deposited
The resource information of Chu Shiyong current period covered the resource information in a upper period.
In an alternative embodiment, each child node and its corresponding of periodically monitoring in the resource database
After resource information, further includes:
According to the resource information monitored, the connected state of each child node is determined, and by the connected state
It is stored together with the corresponding resource information, obtains updated resource information.
In an alternative embodiment, the resource information that the basis monitors determines the connection of each child node
State, comprising:
Judge whether each resource information detected sends within a preset time；
If then determining that it is available for sending the child status of the resource information；
The CPU and memory usage of the available child node are analyzed according to the resource information of the available child node；
According to the CPU and memory usage, the connected state of the available child node is determined.
In an alternative embodiment, the distributed node information monitoring method, further includes:
The updated resource information is inquired, schedulable child node is determined according to load balancing；
Task schedule operation is carried out according to determining schedulable child node.
A kind of distributed node information monitoring system, comprising:
Time synchronization unit, for by the time synchronization of host node and each child node；
Each child node, for acquiring respective resource information according to predetermined period, and by the collected resource information with
The mode of message-oriented middleware is sent to the host node；And
The host node, the resource information for sending each child node are stored in resource database, and
Periodically monitor each child node and its corresponding resource information in the resource database.
In an alternative embodiment, the host node is used for:
With current when the resource information that each child node is sent being stored in the resource database, and being stored
The resource information in period covered the resource information in a upper period.
In an alternative embodiment, the host node is also used to:
According to the resource information monitored, the connected state of each child node is determined, and by the connected state
It is stored together with the corresponding resource information, obtains updated resource information.
In an alternative embodiment, the resource information that the basis monitors determines the connection of each child node
State, comprising:
Judge whether each resource information detected sends within a preset time；
If then determining that it is available for sending the child status of the resource information；
The CPU and memory usage of the available child node are analyzed according to the resource information of the available child node；
According to the CPU and memory usage, the connected state of the available child node is determined.
In an alternative embodiment, the host node is also used to:
The updated resource information is inquired, schedulable child node is determined according to load balancing；
Task schedule operation is carried out according to determining schedulable child node.
Application of the above method in remote sensing satellite Data processing.
The invention has the following beneficial effects:
Distributed node information monitoring method provided in an embodiment of the present invention, by synchronization master and each child node when
Between, respective resource information is periodically sent to host node in a manner of message intermediate child node, number is stored in by host node
It according in library, and is periodically monitored, this method avoid constantly creating, disconnecting, also there is no caused by delay machine because obtaining
The problem of taking the nodal information time to extend, can substantially shorten the time for obtaining large-scale distributed clustered node information, improve
The task schedule response time promotes dispatching efficiency.
Detailed description of the invention
Fig. 1 is a kind of distributed node information monitoring method flow chart provided in an embodiment of the present invention.
Specific embodiment
A specific embodiment of the invention is described in further details below with reference to the drawings and specific embodiments.
As shown in Figure 1, the embodiment of the invention provides a kind of distributed node information monitoring methods, comprising:
Step 101: by the time synchronization of host node and each child node；
Specifically, in the embodiment of the present invention, the host node and child node can be server, work station, ordinary PC
Deng can realize the time synchronization of host node and each child node by modes such as software ntp, automachron；
Step 102: each child node acquires respective resource information according to predetermined period, and by the collected resource information
The host node is sent in a manner of message-oriented middleware；
Specifically, the resource information may include the child nodes systems such as CPU usage, memory usage, hard disk utilization rate
System resource information, can be acquired according to unified resource template；Collection period can according to need setting, and the present invention does not limit
It is fixed；
Step 103: the resource information that each child node is sent is stored in resource database by the host node,
And periodically monitor each child node and its corresponding resource information in the resource database.
Specifically, in the embodiment of the present invention, host node monitoring cycle can identical may be used with the child node information collection period
With difference, the present invention is not construed as limiting；
Distributed node information monitoring method provided in an embodiment of the present invention, by synchronization master and each child node when
Between, respective resource information is periodically sent to host node in a manner of message intermediate child node, number is stored in by host node
It according in library, and is periodically monitored, this method avoid constantly creating, disconnecting, also there is no caused by delay machine because obtaining
The problem of taking the nodal information time to extend, can substantially shorten the time for obtaining large-scale distributed clustered node information, improve
The task schedule response time promotes dispatching efficiency；It solves the high module of certain dependence real-times and loses data problem.
In an alternative embodiment, the resource information that each child node is sent is stored in resource by the host node
In database, comprising:
The resource information that each child node is sent is stored in the resource database by the host node, and is deposited
The resource information of Chu Shiyong current period covered the resource information in a upper period.Pass through the money between the covering of new resources information
Source information, invalidation history in the resource database that can effectively disappear.
In an alternative embodiment, each child node and its corresponding of periodically monitoring in the resource database
After resource information, further includes:
According to the resource information monitored, the connected state of each child node is determined, and by the connected state
It is stored together with the corresponding resource information, obtains updated resource information.It can be light by updated resource information
Obtain the child node connected state and resource utilization.
In an alternative embodiment, the resource information that the basis monitors determines the connection of each child node
State, comprising:
Judge whether each resource information detected sends within a preset time；
If then determining that it is available for sending the child status of the resource information；
The CPU and memory usage of the available child node are analyzed according to the resource information of the available child node；
According to the CPU and memory usage, the connected state of the available child node is determined.Pass through the child node
Connected state, CPU and memory usage provide scheduling strategy reference data and support.
In an alternative embodiment, the distributed node information monitoring method, further includes:
The updated resource information is inquired, schedulable child node is determined according to load balancing；
Task schedule operation is carried out according to determining schedulable child node.It is calculated by schedulable child status by scheduling
Method realizes production scheduling.
The embodiment of the invention also provides a kind of distributed node information monitoring systems, comprising:
The present invention also provides a kind of distributed node information monitoring systems, comprising:
Time synchronization unit, for by the time synchronization of host node and each child node；
Each child node, for acquiring respective resource information according to predetermined period, and by the collected resource information with
The mode of message-oriented middleware is sent to the host node；And
The host node, the resource information for sending each child node are stored in resource database, and
Periodically monitor each child node and its corresponding resource information in the resource database.
In an alternative embodiment, the host node is used for:
With current when the resource information that each child node is sent being stored in the resource database, and being stored
The resource information in period covered the resource information in a upper period.
In an alternative embodiment, the host node is also used to:
According to the resource information monitored, the connected state of each child node is determined, and by the connected state
It is stored together with the corresponding resource information, obtains updated resource information.
In an alternative embodiment, the resource information that the basis monitors determines the connection of each child node
State, comprising:
Judge whether each resource information detected sends within a preset time；
If then determining that it is available for sending the child status of the resource information；
The CPU and memory usage of the available child node are analyzed according to the resource information of the available child node；
According to the CPU and memory usage, the connected state of the available child node is determined.
In an alternative embodiment, the host node is also used to:
The updated resource information is inquired, schedulable child node is determined according to load balancing；
Task schedule operation is carried out according to determining schedulable child node.
Present system embodiment and embodiment of the method correspond, and specifically describe and effect is referring to embodiment of the method,
This is repeated no more.
Application of the above method in remote sensing satellite Data processing.
The following are a specific embodiments of the invention:
Fig. 1 is the distributed node information prison in the remote sensing satellite ground image treatment process of embodiment according to the present invention
Method flow diagram is surveyed, as shown in Figure 1, specific implementation step is as follows:
1. cluster time synchronization: synchronous more by node time used in remote sensing satellite ground image processing cluster using software
Newly, guarantee that each process time is synchronous；
2. periodically sending resource information: the customized needs acquisition section of remote sensing satellite ground image processing system to host node
Point resource template, periodical acquisition node resource information send collected resource to host node in a manner of message-oriented middleware
Information；
3. analysis node is connected to situation and resource information: the section that remote sensing satellite ground image processing system host node will receive
In point resource information and data transmission time deposit database, the data sended over next time will cover previous knot
Fruit.It, will section if the last data transmission time is less than threshold value according to preset node connected state Threshold Analysis
Dotted state be set to it is available, conversely, node state be it is unavailable.Analyze the utilization rates such as enabled node CPU, memory.Analysis is obtained
Node connection situation and resource related information update into memory；
4. analyzing result in case scheduling uses: in remote sensing satellite ground image job-parallel processor operation process, scheduling
Node resources information analysis result and combination load balancing in program inquiring memory carry out image processing tasks scheduling and make
Industry.
The above, optimal specific embodiment only of the invention, but scope of protection of the present invention is not limited thereto,
In the technical scope disclosed by the present invention, any changes or substitutions that can be easily thought of by anyone skilled in the art,
It should be covered by the protection scope of the present invention.
The content that description in the present invention is not described in detail belongs to the well-known technique of professional and technical personnel in the field.
Claims (11)
1. a kind of distributed node information monitoring method characterized by comprising
By the time synchronization of host node and each child node；
Each child node acquires respective resource information according to predetermined period, and by the collected resource information with message-oriented middleware
Mode be sent to the host node；
The resource information that each child node is sent is stored in resource database by the host node, and is periodically monitored
Each child node and its corresponding resource information in the resource database.
2. distributed node information monitoring method according to claim 1, which is characterized in that the host node will be described each
The resource information that child node is sent is stored in resource database, comprising:
The resource information that each child node is sent is stored in the resource database by the host node, and when storage
The resource information in a upper period was covered with the resource information of current period.
3. distributed node information monitoring method according to claim 2, which is characterized in that described in the periodically monitoring
In resource database after each child node and its corresponding resource information, further includes:
According to the resource information monitored, the connected state of each child node is determined, and by the connected state and right
The resource information answered stores together, obtains updated resource information.
4. distributed node information monitoring method according to claim 3, which is characterized in that the institute that the basis monitors
Resource information is stated, determines the connected state of each child node, comprising:
Judge whether each resource information detected sends within a preset time；
If then determining that it is available for sending the child status of the resource information；
The CPU and memory usage of the available child node are analyzed according to the resource information of the available child node；
According to the CPU and memory usage, the connected state of the available child node is determined.
5. distributed node information monitoring method according to claim 3, which is characterized in that further include:
The updated resource information is inquired, schedulable child node is determined according to load balancing；
Task schedule operation is carried out according to determining schedulable child node.
6. a kind of distributed node information monitoring system characterized by comprising
Time synchronization unit, for by the time synchronization of host node and each child node；
Each child node, for acquiring respective resource information according to predetermined period, and by the collected resource information with message
The mode of middleware is sent to the host node；And
The host node, the resource information for sending each child node is stored in resource database, and the period
Property the monitoring resource database in each child node and its corresponding resource information.
7. distributed node information monitoring device according to claim 6, which is characterized in that the host node is used for:
The current period when resource information that each child node is sent being stored in the resource database, and being stored
The resource information covered the resource information in upper period.
8. distributed node information monitoring device according to claim 7, which is characterized in that the host node is also used to:
According to the resource information monitored, the connected state of each child node is determined, and by the connected state and right
The resource information answered stores together, obtains updated resource information.
9. distributed node information monitoring device according to claim 8, which is characterized in that the institute that the basis monitors
Resource information is stated, determines the connected state of each child node, comprising:
Judge whether each resource information detected sends within a preset time；
If then determining that it is available for sending the child status of the resource information；
The CPU and memory usage of the available child node are analyzed according to the resource information of the available child node；
According to the CPU and memory usage, the connected state of the available child node is determined.
10. distributed node information monitoring device according to claim 8, which is characterized in that the host node is also used
In:
The updated resource information is inquired, schedulable child node is determined according to load balancing；
Task schedule operation is carried out according to determining schedulable child node.
11. any one of Claims 1 to 5 the method is in the application of remote sensing satellite Data processing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811545084.1A CN109672731A (en) | 2018-12-17 | 2018-12-17 | A kind of distributed node information monitoring method, system and application |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811545084.1A CN109672731A (en) | 2018-12-17 | 2018-12-17 | A kind of distributed node information monitoring method, system and application |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109672731A true CN109672731A (en) | 2019-04-23 |
Family
ID=66144904
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811545084.1A Pending CN109672731A (en) | 2018-12-17 | 2018-12-17 | A kind of distributed node information monitoring method, system and application |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109672731A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110457176A (en) * | 2019-07-12 | 2019-11-15 | 平安普惠企业管理有限公司 | For the monitoring method of distributed system, device, storage medium and electronic equipment |
| CN110912977A (en) * | 2019-11-15 | 2020-03-24 | 北京浪潮数据技术有限公司 | Configuration file updating method, device, equipment and storage medium |
| WO2022151083A1 (en) * | 2021-01-13 | 2022-07-21 | Oppo广东移动通信有限公司 | Node determination method and apparatus for distributed task, and device and medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104507156A (en) * | 2014-12-17 | 2015-04-08 | 西南大学 | Improved method for time synchronization based on IEEE 1588 PTP mechanism for wireless network |
| CN104935482A (en) * | 2015-06-26 | 2015-09-23 | 曙光信息产业(北京)有限公司 | Distributed monitoring system and method |
-
2018
- 2018-12-17 CN CN201811545084.1A patent/CN109672731A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104507156A (en) * | 2014-12-17 | 2015-04-08 | 西南大学 | Improved method for time synchronization based on IEEE 1588 PTP mechanism for wireless network |
| CN104935482A (en) * | 2015-06-26 | 2015-09-23 | 曙光信息产业(北京)有限公司 | Distributed monitoring system and method |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110457176A (en) * | 2019-07-12 | 2019-11-15 | 平安普惠企业管理有限公司 | For the monitoring method of distributed system, device, storage medium and electronic equipment |
| CN110912977A (en) * | 2019-11-15 | 2020-03-24 | 北京浪潮数据技术有限公司 | Configuration file updating method, device, equipment and storage medium |
| WO2022151083A1 (en) * | 2021-01-13 | 2022-07-21 | Oppo广东移动通信有限公司 | Node determination method and apparatus for distributed task, and device and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104184819B (en) | Multi-layer load balancing cloud resource monitoring method | |
| CN109672731A (en) | A kind of distributed node information monitoring method, system and application | |
| CN108846076A (en) | The massive multi-source ETL process method and system of supporting interface adaptation | |
| CN105635279A (en) | Distributed monitor system and data acquisition method thereof | |
| CN112565415B (en) | Cross-region resource management system and method based on cloud edge cooperation | |
| CN101345652B (en) | Data acquisition method and data acquisition equipment | |
| CN105049268A (en) | Distributed computing resource allocation system and task processing method | |
| CN110794800A (en) | Monitoring system for wisdom mill information management | |
| CN104765641B (en) | A kind of job scheduling method and system | |
| CN104506373A (en) | Device and method for collecting and processing network information | |
| CN103517405B (en) | A kind of method and system of network positions, mobile terminal and network side equipment | |
| CN113568743A (en) | Management method, device and medium of Internet of things equipment and electronic equipment | |
| CN107968797B (en) | Video transmission method, device and system | |
| CN105580007A (en) | Monitoring a computing network | |
| CN103957280A (en) | Connection allocation and scheduling method of sensor network in Internet of things | |
| CN104503894A (en) | System and method for monitoring state of distributed server in real time | |
| CN102026228B (en) | Statistical method and equipment for communication network performance data | |
| CN113054743A (en) | Internet of things terminal data access system and method suitable for power distribution cloud master station | |
| CN113937894A (en) | Cloud edge cooperation-based electric intelligent terminal management system and method | |
| CN105959363A (en) | Big data cluster deployment method capable of adapting to hardware configuration | |
| CN109672728A (en) | A kind of large-scale distributed clustered node information fast acquiring method | |
| CN106713512B (en) | Efficient data transmission method and system for dynamically balancing network bandwidth of Internet of things | |
| CN105471107A (en) | Time-sharing task collection method for power grid electric energy measuring system | |
| CN104714875A (en) | Distributed automatic collecting method | |
| Ou et al. | Research on network performance optimization technology based on cloud-edge collaborative architecture |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190423 |