CN109672728A - A kind of large-scale distributed clustered node information fast acquiring method - Google Patents
A kind of large-scale distributed clustered node information fast acquiring method Download PDFInfo
- Publication number
- CN109672728A CN109672728A CN201811485311.6A CN201811485311A CN109672728A CN 109672728 A CN109672728 A CN 109672728A CN 201811485311 A CN201811485311 A CN 201811485311A CN 109672728 A CN109672728 A CN 109672728A
- Authority
- CN
- China
- Prior art keywords
- node
- information
- database
- cluster
- scale distributed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B7/00—Radio transmission systems, i.e. using radiation field
- H04B7/14—Relay systems
- H04B7/15—Active relay systems
- H04B7/185—Space-based or airborne stations; Stations for satellite systems
- H04B7/1851—Systems using a satellite or space-based relay
- H04B7/18519—Operations control, administration or maintenance
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0805—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
- H04L43/0811—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking connectivity
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0805—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
- H04L43/0817—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking functioning
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/16—Threshold monitoring
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Environmental & Geological Engineering (AREA)
- Physics & Mathematics (AREA)
- Astronomy & Astrophysics (AREA)
- Aviation & Aerospace Engineering (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
A kind of large-scale distributed clustered node information fast acquiring method of the present invention, comprising: to node time synchronized updates all in cluster；Acquisition node resource information, and collected node resources information is sent to host node periodically in a manner of message-oriented middleware；Collected node resources information and data transmission time are stored in database, and cluster resource state in periodicity analysis database by host node, comprising: the connected state of node and host node, node resources information；And the cluster resource state that analysis obtains is updated to database；Cluster resource state in the database of query master node carries out task schedule in conjunction with load-balancing algorithm.The acquisition time of large-scale distributed clustered node information is greatly saved in the present invention, so that the shortening of analysis cluster state for time, promotes the task schedule response time, improves scheduling whole efficiency.
Description
Technical field
The invention belongs to remote sensing satellite field of communication technology, it is related to a kind of large-scale distributed clustered node information and quickly obtains
Take method.
Background technique
With the development of information technology and the expansion of network size, remote sensing satellite data processing is handled by single node to be changed into
Distributed type assemblies processing.It is slow because obtaining clustered node information in Remote Sensing Ground Station data handling procedure, lead to task tune
Certain modules for requiring real-time high are made to lose data during degree system job, to directly affect entire scheduling system
Function and performance.There are mainly two types of mode, host node active poll and and heartbeat detections for node resources information acquisition at present.
1, host node active polling mode
This mode acquires child node resource information by host node cycle active polling mode in cluster.Which is maximum
The shortcomings that be exactly to need the continuous each child node of poll cluster of timing, overhead is larger；If if interval time is too long, collection
Group node susceptibility is lower, and the node status information of acquisition is likely to out-of-date information；It is needed if interval time is too short
A large amount of socket connections are created repeatedly, when clustered node quantity is larger, need to create each node corresponding socket company
It connects, continuous creation, deletion socket connection impact larger, occupancy host node system resource to whole system.
2, heartbeat detection
Heartbeat detection is as the result of poll but implementation has difference, and heartbeat needs host node to issue communication interface,
The connection of host node and child node is realized in a manner of webservice or socket, although this mode is compared to poll side
Formula alleviates the pressure of host node, but webservice and socket need communication handshake, if wherein a side is because of certain
Reason delay machine, data will need to attempt reconnection, just will disconnect connection after repeatedly having no result, this process in tcp level time-out
It needs to expend the regular hour, if how this mode of delay machine node can also have inefficiency, obtains nodal information time phase
To longer.
Summary of the invention
Problems to be solved by the invention are that scheduling whole efficiency is low in solution remote sensing earth station satellite data treatment process
Under, and certain modules for requiring real-time high lose data problem.
Technological means for solving project is that the present invention proposes that a kind of large-scale distributed clustered node information quickly obtains
Method is taken, is included the following steps:
Step 1, to node time synchronized updates all in cluster；
Step 2, acquisition node resource information, and it is collected to host node transmission periodically in a manner of message-oriented middleware
Node resources information；
Collected node resources information and data transmission time are stored in database by step 3, host node, and are periodically divided
Analyse cluster resource state in database, comprising: the connected state of node and host node, node resources information；And analysis is obtained
Cluster resource state be updated to database；
Step 4, query master node database in cluster resource state, in conjunction with load-balancing algorithm carry out task schedule.
Further, as a preferred technical solution of the present invention: automatic according to resource information template in the step 2
Acquisition node resource information.
Further, as a preferred technical solution of the present invention: host node is by collected node in the step 3
Information is stored in database, and the node resources information being once stored in front of received node resources information covering next time.
Further, as a preferred technical solution of the present invention: the company of analysis node and host node in the step 3
Logical state, comprising:
Further, as a preferred technical solution of the present invention: according to preset node connected state threshold value
Node connected state is judged as available, otherwise connects node by judgement if the last data transmission time is less than threshold value
Logical state is judged as unavailable.
Further, as a preferred technical solution of the present invention: analysis node resource information includes in the step 3
Analysis node CPU, memory and utilization rate.
Invention effect are as follows:
Large-scale distributed clustered node information fast acquiring method of the invention, cluster interior nodes resource information it is automatic
It obtains, the acquisition time of large-scale distributed clustered node information is greatly saved, it can be by analyzing the entire cluster of quick obtaining
State improves the response time of task schedule.So that the shortening of analysis cluster state for time, when promoting task schedule response
Between, improve scheduling whole efficiency.The quick obtaining for realizing clustered node information is improved in the data processing of remote sensing earth station
Whole efficiency is dispatched in journey, has been satisfactorily resolved the high module of certain dependence real-times and has been lost data problem.
Detailed description of the invention
Fig. 1 is the flow diagram of the large-scale distributed clustered node information fast acquiring method of the present invention.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, below in conjunction with attached drawing, to the present invention one
The large-scale distributed clustered node information of kind quickly passes by method further description, and specific implementation described herein is only
To explain the present invention, it is not intended to limit the present invention.
As shown in Figure 1, the present invention designs a kind of large-scale distributed clustered node information fast acquiring method, including as follows
Step:
Step 1, using software to node time synchronized updates all in cluster, guarantee that each process time is synchronous.
Step 2, system be customized to need acquisition node resource template, acquisition node resource information, and periodically with message
The mode of middleware sends collected node resources information to host node.
Collected node resources information and data transmission time are stored in database by step 3, host node, can will next time
The node resources information being once stored in front of received node resources information covering
Also, cluster resource state in periodicity analysis database, comprising: the connected state of analysis node and host node,
Analysis node resource information.
The connected state of analysis node and host node, comprising: according to preset node connected state threshold decision, such as
Fruit the last time data transmission time is less than threshold value, then node connected state is judged as available, otherwise by node connected state
It is judged as unavailable.
Analysis node resource information includes analyzing CPU, memory and the utilization rate etc. of enabled node.
It then, include that node shape connection situation and node resources information are updated to number by the cluster resource state that analysis obtains
According to library.
Step 4, query master node database in cluster resource state, in conjunction with load-balancing algorithm carry out task schedule.
In schedule job process, scheduler program inquires the node resources information analysis result in database and combination load is equal
Weighing apparatus method carries out task schedule operation, such as analyzes according to the connected state of node and host node as a result, by enabled node stock number
The low node of utilization rate is preferentially chosen for task schedule node, carries out task by the node and receives and executes.Task scheduling system
The automatic dispatching operation of Remote Sensing Data Processing is carried out according to above procedure, realizes task schedule.
To sum up, the present invention realizes the automatic acquisition of cluster interior nodes resource information, and the time is greatly saved, can be by dividing
The entire cluster state of quick obtaining is analysed, the response time of task schedule is improved.The shortening of cluster state for time is analyzed, is promoted and is appointed
It is engaged in the dispatching response time, improves scheduling whole efficiency.
It should be noted that described above is only the preferred embodiment of the present invention, it should be understood that for art technology
For personnel, several changes and improvements can also be made under the premise of not departing from the technology of the present invention design, these are included in
In protection scope of the present invention.
Claims (5)
1. a kind of large-scale distributed clustered node information fast acquiring method, which comprises the steps of:
Step 1, to node time synchronized updates all in cluster；
Step 2, acquisition node resource information, and collected node is sent to host node periodically in a manner of message-oriented middleware
Resource information；
Collected node resources information and data transmission time are stored in database, and periodicity analysis number by step 3, host node
According to cluster resource state in library, comprising: the connected state of node and host node, node resources information；And the collection for obtaining analysis
Group's resource status is updated to database；
Step 4, query master node database in cluster resource state, in conjunction with load-balancing algorithm carry out task schedule.
2. large-scale distributed clustered node information fast acquiring method according to claim 1, which is characterized in that described
According to resource information template automatic collection node resources information in step 2.
3. large-scale distributed clustered node information fast acquiring method according to claim 1, which is characterized in that described
Collected nodal information is stored in database by host node in step 3, and will received node resources information covering next time
The preceding node resources information being once stored in.
4. large-scale distributed clustered node information fast acquiring method according to claim 1, which is characterized in that described
The connected state of analysis node and host node in step 3, comprising:
It, will if the last data transmission time is less than threshold value according to preset node connected state threshold decision
Node connected state is judged as available, and node connected state is otherwise judged as unavailable.
5. large-scale distributed clustered node information fast acquiring method according to claim 1, which is characterized in that described
Analysis node resource information includes analysis node CPU, memory and utilization rate in step 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811485311.6A CN109672728A (en) | 2018-12-06 | 2018-12-06 | A kind of large-scale distributed clustered node information fast acquiring method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811485311.6A CN109672728A (en) | 2018-12-06 | 2018-12-06 | A kind of large-scale distributed clustered node information fast acquiring method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109672728A true CN109672728A (en) | 2019-04-23 |
Family
ID=66143622
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811485311.6A Pending CN109672728A (en) | 2018-12-06 | 2018-12-06 | A kind of large-scale distributed clustered node information fast acquiring method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109672728A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110649958A (en) * | 2019-09-05 | 2020-01-03 | 北京百度网讯科技有限公司 | Method, apparatus, device and medium for processing data |
| CN113872673A (en) * | 2021-09-27 | 2021-12-31 | 哈工大机器人(中山)无人装备与人工智能研究院 | Satellite constellation simulation container cloud resource scheduling system and scheduling method thereof |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102695204A (en) * | 2012-04-24 | 2012-09-26 | 北京讯风光通信技术开发有限责任公司 | Business control method used in industrial wireless network, and system thereof |
| CN103618644A (en) * | 2013-11-26 | 2014-03-05 | 曙光信息产业股份有限公司 | Distributed monitoring system based on hadoop cluster and method thereof |
| CN104283948A (en) * | 2014-09-26 | 2015-01-14 | 东软集团股份有限公司 | Server cluster system and load balancing implementation method thereof |
| CN104507156A (en) * | 2014-12-17 | 2015-04-08 | 西南大学 | Improved method for time synchronization based on IEEE 1588 PTP mechanism for wireless network |
| CN104935482A (en) * | 2015-06-26 | 2015-09-23 | 曙光信息产业(北京)有限公司 | Distributed monitoring system and method |
-
2018
- 2018-12-06 CN CN201811485311.6A patent/CN109672728A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102695204A (en) * | 2012-04-24 | 2012-09-26 | 北京讯风光通信技术开发有限责任公司 | Business control method used in industrial wireless network, and system thereof |
| CN103618644A (en) * | 2013-11-26 | 2014-03-05 | 曙光信息产业股份有限公司 | Distributed monitoring system based on hadoop cluster and method thereof |
| CN104283948A (en) * | 2014-09-26 | 2015-01-14 | 东软集团股份有限公司 | Server cluster system and load balancing implementation method thereof |
| CN104507156A (en) * | 2014-12-17 | 2015-04-08 | 西南大学 | Improved method for time synchronization based on IEEE 1588 PTP mechanism for wireless network |
| CN104935482A (en) * | 2015-06-26 | 2015-09-23 | 曙光信息产业(北京)有限公司 | Distributed monitoring system and method |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110649958A (en) * | 2019-09-05 | 2020-01-03 | 北京百度网讯科技有限公司 | Method, apparatus, device and medium for processing data |
| CN110649958B (en) * | 2019-09-05 | 2022-07-26 | 北京百度网讯科技有限公司 | Method, apparatus, device and medium for processing satellite data |
| CN113872673A (en) * | 2021-09-27 | 2021-12-31 | 哈工大机器人(中山)无人装备与人工智能研究院 | Satellite constellation simulation container cloud resource scheduling system and scheduling method thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101958804B (en) | Method for promoting warning processing efficiency, server and system | |
| CN106033476B (en) | A kind of increment type figure calculation method under distributed computation mode in cloud computing environment | |
| CN112162865A (en) | Server scheduling method and device and server | |
| CN111459665A (en) | Distributed edge computing system and distributed edge computing method | |
| CN102110022B (en) | Sensor network embedded operation system based on priority scheduling | |
| CN105677469A (en) | Timing task executing method and device | |
| CN105430030A (en) | OSG-based parallel extendable application server | |
| CN103235835A (en) | Inquiry implementation method for database cluster and device | |
| CN109672728A (en) | A kind of large-scale distributed clustered node information fast acquiring method | |
| CN103150324A (en) | Chained processing-based data collecting system and method | |
| CN103699433A (en) | Method and system for performing dynamic adjustment on number of tasks in Hadoop platform | |
| CN109672731A (en) | A kind of distributed node information monitoring method, system and application | |
| CN102970104A (en) | Method for acquiring data and server | |
| CN111160873A (en) | Batch processing device and method based on distributed architecture | |
| CN115086379B (en) | Numerical control machine tool data acquisition method based on edge calculation and virtualization technology | |
| CN102026228B (en) | Statistical method and equipment for communication network performance data | |
| CN114500543B (en) | Distributed elastic edge acquisition system and application method thereof | |
| CN102510403B (en) | Receive and the cluster distributed system and method for real-time analysis for vehicle data | |
| CN107479966B (en) | Signaling acquisition method based on multi-core CPU | |
| CN115756822A (en) | Method and system for optimizing performance of high-performance computing application | |
| CN101692737B (en) | Light weight data synchronization system and method | |
| KR102246079B1 (en) | SYSTEM OF SENSORY DATA ACQUISITION AND SYNCHRONIZATION FOR CLOUD CENTRIC IoT AND METHOD PERFORMING THEREOF | |
| CN112600842A (en) | Cluster shell method and device, electronic equipment and computer readable storage medium | |
| CN107220114A (en) | Distributed resource scheduling method based on resource United Dispatching | |
| CN116166396A (en) | Training method and device of scheduling model, electronic equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190423 |