CN110502346A - Resource information management system and method under a kind of cluster environment - Google Patents
Resource information management system and method under a kind of cluster environment Download PDFInfo
- Publication number
- CN110502346A CN110502346A CN201910800797.6A CN201910800797A CN110502346A CN 110502346 A CN110502346 A CN 110502346A CN 201910800797 A CN201910800797 A CN 201910800797A CN 110502346 A CN110502346 A CN 110502346A
- Authority
- CN
- China
- Prior art keywords
- data
- resource
- module
- cluster
- acquisition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
- G06F9/5088—Techniques for rebalancing the load in a distributed system involving task migration
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention belongs to resource information management technical fields, resource information management system and method under a kind of cluster environment are disclosed, resource information management system includes: resource data acquisition module, abnormality detection module, access authentication module, central control module, resource data encrypting module, resource classification module, resource distribution module, transferring module, safety protection module, alarm module, display module under the cluster environment.It ensure that the stability of abnormality detection；Simultaneously, close operation mechanism is exempted to each server configuration that new cluster includes by transferring module, the function of realizing while Servers-all is configured, and environment configurations and Data Migration are automatically performed, in compared with the prior art in such a way that manual hand manipulation configures new cluster each server, while cluster migration working efficiency can be increased substantially using method of the invention, cost of labor is reduced.
Description
Technical field
The invention belongs to resource information management systems under resource information management technical field more particularly to a kind of cluster environment
And method.
Background technique
Currently, the immediate prior art: trunked communication system is that a kind of movement for group's dispatch control communication is logical
Letter system is mainly used in professional mobile communication field.The available channel that the system has can share for the total user of system,
With automatically selecting channel function, it is shared resource, contribution, shared channel equipment and the multipurpose of service, high-effect
Wireless dispatching communication system.The maximum feature of cluster communication is Speech Communication using PTT (PushToTalk), is pushed to talk with one
Mode connect, called to be answered without off-hook, and connect fast speed, and can support the functions such as group call, its fortune
Make mode based on either simplex, half-duplex, mainly use the dynamics of channels method of salary distribution, and user have different priority level and
Specific function, when communication, can get ready to go into action in hundreds.However, existing cluster resource management process is to cluster abnormality detection poor accuracy；
Meanwhile the migration of existing cluster needs manually to be configured and detected, entire human configuration and detection process is very cumbersome and consumption
When, all configuration-direct needs are manually entered, check, modifying, confirming, and each server in new cluster will repeat
A time above-mentioned job could complete configuration work, this makes cluster migration working efficiency low.
In conclusion problem of the existing technology is: existing cluster resource management process is accurate to cluster abnormality detection
It is poor to spend；Meanwhile existing cluster migration needs manually to be configured and detected, entire human configuration and detection process are very cumbersome
And it is time-consuming, all configuration-direct needs are manually entered, check, modifying, confirming, and each server in new cluster will
Configuration work could be completed by coming again above-mentioned work, this makes cluster migration working efficiency low.
Summary of the invention
In view of the problems of the existing technology, the present invention provides resource information management system and sides under a kind of cluster environment
Method.
The invention is realized in this way resource information management system under a kind of cluster environment, resource under the cluster environment
Information management system includes:
Resource data acquisition module, connect with central control module, for acquiring Internet resources by the network equipment；
Abnormality detection module, connect with central control module, for being detected by detecting program to the exception of cluster；
Authentication module is accessed, is connect with central control module, for being tested by proving program access cluster resource
Card；
Central control module adds with resource data acquisition module, abnormality detection module, access authentication module, resource data
Close module, resource classification module, resource distribution module, transferring module, safety protection module 9, alarm module, display module connect
It connects, is worked normally for controlling modules by host；
Resource data encrypting module, connect with central control module, for by encipheror to cluster resource data into
Row encryption；
Resource classification module, connect with central control module, for being classified by sort program to cluster resource；
Resource distribution module is connect with central control module, for being allocated by distribution program to cluster resource；
Transferring module is connect with central control module, for carrying out migration operation to cluster；
Safety protection module is connect with central control module, for being pacified by protection capacity of safety protection software to cluster resource
Full protection；
Alarm module is connect with central control module, for being warned by warning program to detection abnormal data；
Display module is connect with central control module, for being believed by the Internet resources and warning of display display acquisition
Breath.
Further, the resource data acquisition module includes:
Acquisition tasks are submitted to task pool for generating acquisition tasks according to acquisition strategies by acquisition tasks generation unit；
The concurrent execution unit of acquisition tasks executes acquisition tasks for distributing thread from the concurrent execution thread pond of acquisition,
And it is acquired overtime monitoring；
Time-out control unit is acquired, if for acquisition time-out, acquisition tasks normal termination, returned data acquisition not to occur
As a result, thread is released back into the concurrent execution thread pond of acquisition, acquisition tasks are added and complete list；If acquisition time-out occurs, in
Disconnected acquisition tasks, discharge resource, and return time-out acquires concurrent execution thread pond as a result, thread is released back into, acquisition tasks are added
Overtime task list；
Resource base information maintenance unit, for being used to saving and modifying resource base information table, by the number of resources of input
Collection of resources information table is automatically generated according to table name；The resource allocation information maintenance module is used to acquire resource information and advise
It is then configured, modifies resource base information table；The resource regular testing module is used to carry out the collection rule of configuration
Test；Module is issued/cancelled to the resource information for issuing and cancelling resource information, automatically generates after issuing resource information
Monitoring resource information table；
Task basic information maintenance unit, for for saving and modifying task basic information table；The task opens
It moves, stop and restarts module and be used, respectively, to initiate task, stopping task and restarting task, automatically generate and appoint after starting task
Business monitoring information table；
Data Management Unit includes browsing data and unified search；The browsing data is used to acquire the unification of data
It shows, shows the acquisition information of some resource, include resource name, acquisition data volume and data browsing function；The unification
It searches for and is retrieved for the keyword to acquisition content, according to keywords search acquisition content results collection information；
Recognition unit, for acquisition terminal by the identification information collection of the sequence memory response of identification information collection；
Acquisition unit carries out channel radio by the identification information collection of storage and other measuring terminals for acquisition terminal
Letter acquires data；
Storage unit is transmitted, carries out transimission and storage for the data completed will to be acquired.
Further, the resource data encrypting module includes:
Key expansion unit, for being extended according to predetermined Hadamard orthomorphic matrices to key；
Data distribution time processing unit, for being taken according to data after data server receives data encryption request
The data encryption equipment information list stored in business device obtains the last assigned data encryption equipment, obtains the data
The corresponding distribution start time information of all data encryption equipments in facility information list is encrypted, by all data encryption equipments
The size of corresponding distribution initial time is compared, and determines maximum distribution initial time, when the maximum distribution originates
When time is not sky, using the maximum corresponding data encryption equipment of initial time that distributes as the last assigned number
According to encryption equipment；
Data encryption request unit, the location information of the data encryption equipment assigned for the last time and point of setting
With rule, it is that some idle data add that data encryption, which is requested assignment to the state in the data encryption equipment information list,
Close equipment, and be busy by status modifier of some data encryption equipment in data encryption equipment information list；
Hashed value presets unit, for calculating the corresponding length hash value of length of new data by presetting hashing algorithm,
And length hash value and the default redundant data are spliced into target data, and target data is calculated by default hashing algorithm
Hashed value；
Data transmission unit is used for encrypted Deta bearer in data frame, and passes through communication network for the data frame
It is sent to opposite end communication equipment.
Another object of the present invention is to provide a kind of clusters for running resource information management system under the cluster environment
Resource information management method under environment, resource information management method under the cluster environment the following steps are included:
Step 1 acquires Internet resources using the network equipment by resource data acquisition module；Pass through abnormality detection module
The exception of cluster is detected using detection program；By access authentication module using proving program to access cluster resource into
Row verifying；
Step 2, central control module carry out cluster resource data using encipheror by resource data encrypting module
Encryption；
Step 3 classifies to cluster resource using sort program by resource classification module；Pass through resource allocation mould
Block is allocated cluster resource using distribution program；
Step 4 carries out migration operation to cluster by transferring module；
Step 5 carries out security protection to cluster resource using protection capacity of safety protection software by safety protection module；Pass through police
Show that module warns detection abnormal data using warning program；
Step 6 utilizes the Internet resources and information warning of display display acquisition by display module.
Further, described as follows using abnormal progress detection method of the detection program to cluster by abnormality detection module:
(1) a performance number strong point to be detected for indicating the clustering performance, the performance number are obtained by detection program
Strong point includes normalized multidimensional performance indicator；
(2) it from the existing performance data class by the polymerize generation in performance number strong point acquired before, determines and is somebody's turn to do
The highest performance data class of performance number strong point to be detected similarity；Judge the performance number strong point to be detected and identified property
Whether the similarity of energy data class is more than the current similarity threshold of the performance data class；
(3) when being more than current similarity threshold, which is aggregated to identified performance
In data class, and calculate after polymerization that data point sum accounts for the data point sum of current all properties data class in the performance data class
Ratio whether be more than exception class threshold value；The performance data after performance number strong point to be detected is added is updated according to following formula
The center particle and similarity threshold of class:
Cr=(pt+cr*np)/(np+1)
Wherein, cr is center particle, and np is data point sum in class, and pt is the performance number strong point that is added, sim be pt and
The similarity of performance data class, th are exception class threshold value, and lr is the learning rate threshold value for adjusting th；
(4) when being less than exception class threshold value, to the centroplasm at the performance number strong point and the performance data class to be detected
The sum of distance maximum distance for being ranked up, and calculating predetermined ratio of each dimension performance indicator of point and all dimensions away from
From with the ratio between whether be greater than range distribution threshold value；
(5) when being greater than range distribution threshold value, determine that the performance number strong point to be detected is an abnormal point.
Further, described as follows to cluster progress migration operation method by transferring module:
1) whether the new collection swarm parameter of configuration, environment and the function for detecting each server for including in new cluster are normal；Its
In, the new cluster is the target object of cluster migration；The each server for including in the new cluster is configured to exempt from close behaviour
Make mechanism；
2) when the function of detecting each server for including in the new cluster is normal, include to the new cluster
Each server carry out environment configurations, and start environment configurations in each server that the new cluster includes and service accordingly
Function；
3) by the Data Migration in old cluster into the new cluster；Wherein, the old cluster be to the new cluster into
The object to be migrated of row cluster migration.
Another object of the present invention is to provide a kind of information for carrying resource information management system under the cluster environment
Data processing terminal.
Advantages of the present invention and good effect are as follows: the present invention can be to similarity height and phase in class by abnormality detection module
It is preferably distinguished like the point for spending low；In this way, abnormality detection scheme can reduce rate of false alarm；In addition, abnormal inspection of the invention
Survey scheme judges the ratio of abnormal point in the window and based on sliding window, may further improve the standard of abnormality alarming
Exactness；Abnormality detection scheme of the invention can also control the classification sum of Clustering Model, and change in data dimension, and
When re-create Clustering Model, to ensure that the stability of abnormality detection；Meanwhile including to new cluster by transferring module
Close operation mechanism is exempted from the configuration of each server, is realized while to the function that Servers-all is configured, and environment configurations
It is automatically performed with Data Migration, it is middle compared with the prior art to use manual hand manipulation to each server of new cluster
The mode configured reduces people while capable of increasing substantially cluster migration working efficiency using method of the invention
Work cost.
Detailed description of the invention
Fig. 1 is resource information management method flow diagram under cluster environment provided in an embodiment of the present invention.
Fig. 2 is resource information management system structural schematic diagram under cluster environment provided in an embodiment of the present invention；
Fig. 3 is the structural schematic diagram of resource data acquisition module provided in an embodiment of the present invention；
Fig. 4 is the structural schematic diagram of resource data encrypting module provided in an embodiment of the present invention；
In figure: 1, resource data acquisition module；2, abnormality detection module；3, authentication module is accessed；4, central control module；
5, resource data encrypting module；6, resource classification module；7, resource distribution module；8, transferring module；9, safety protection module；
10, alarm module；11, display module.
Specific embodiment
In order to further understand the content, features and effects of the present invention, the following examples are hereby given, and cooperate attached drawing
Detailed description are as follows.
In view of the problems of the existing technology, the present invention provides resource information management system and sides under a kind of cluster environment
Method is with reference to the accompanying drawing explained in detail the present invention.
As shown in Figure 1, under cluster environment provided by the invention resource information management method the following steps are included:
Step S101 acquires Internet resources using the network equipment by resource data acquisition module；Pass through abnormality detection mould
Block detects the exception of cluster using detection program；By access authentication module using proving program to access cluster resource
It is verified；
Step S102, central control module by resource data encrypting module using encipheror to cluster resource data into
Row encryption；
Step S103 classifies to cluster resource using sort program by resource classification module；Pass through resource allocation
Module is allocated cluster resource using distribution program；
Step S104 carries out migration operation to cluster by transferring module；
Step S105 carries out security protection to cluster resource using protection capacity of safety protection software by safety protection module；Pass through
Alarm module warns detection abnormal data using warning program；
Step S106 utilizes the Internet resources and information warning of display display acquisition by display module.
As shown in Fig. 2, resource information management system includes: that resource data is adopted under cluster environment provided in an embodiment of the present invention
Collect module 1, abnormality detection module 2, access authentication module 3, central control module 4, resource data encrypting module 5, resource classification
Module 6, resource distribution module 7, transferring module 8, safety protection module 9, alarm module 10, display module 11.
Resource data acquisition module 1 is connect with central control module 4, for acquiring Internet resources by the network equipment；
Abnormality detection module 2 is connect with central control module 4, for being examined by detecting program to the exception of cluster
It surveys；
Authentication module 3 is accessed, is connect with central control module 4, for being carried out by proving program to access cluster resource
Verifying；
Central control module 4, with resource data acquisition module 1, abnormality detection module 2, access authentication module 3, number of resources
According to encrypting module 5, resource classification module 6, resource distribution module 7, transferring module 8, safety protection module 9, alarm module 10, show
Show that module 11 connects, is worked normally for controlling modules by host；
Resource data encrypting module 5 is connect with central control module 4, for passing through encipheror to cluster resource data
It is encrypted；
Resource classification module 6 is connect with central control module 4, for being classified by sort program to cluster resource；
Resource distribution module 7 is connect with central control module 4, for being allocated by distribution program to cluster resource；
Transferring module 8 is connect with central control module 4, for carrying out migration operation to cluster；
Safety protection module 9 is connect with central control module 4, for being carried out by protection capacity of safety protection software to cluster resource
Security protection；
Alarm module 10 is connect with central control module 4, for being warned by warning program to detection abnormal data
Show；
Display module 11 is connect with central control module 4, Internet resources and warning for being acquired by display display
Information.
As shown in figure 3, resource data acquisition module 1 includes:
Acquisition tasks are submitted to task pool for generating acquisition tasks according to acquisition strategies by acquisition tasks generation unit；
The concurrent execution unit of acquisition tasks executes acquisition tasks for distributing thread from the concurrent execution thread pond of acquisition,
And it is acquired overtime monitoring；
Time-out control unit is acquired, if for acquisition time-out, acquisition tasks normal termination, returned data acquisition not to occur
As a result, thread is released back into the concurrent execution thread pond of acquisition, acquisition tasks are added and complete list；If acquisition time-out occurs, in
Disconnected acquisition tasks, discharge resource, and return time-out acquires concurrent execution thread pond as a result, thread is released back into, acquisition tasks are added
Overtime task list；
Resource base information maintenance unit, for being used to saving and modifying resource base information table, by the number of resources of input
Collection of resources information table is automatically generated according to table name；The resource allocation information maintenance module is used to acquire resource information and advise
It is then configured, modifies resource base information table；The resource regular testing module is used to carry out the collection rule of configuration
Test；Module is issued/cancelled to the resource information for issuing and cancelling resource information, automatically generates after issuing resource information
Monitoring resource information table；
Task basic information maintenance unit, for for saving and modifying task basic information table；The task opens
It moves, stop and restarts module and be used, respectively, to initiate task, stopping task and restarting task, automatically generate and appoint after starting task
Business monitoring information table；
Data Management Unit includes browsing data and unified search；The browsing data is used to acquire the unification of data
It shows, shows the acquisition information of some resource, include resource name, acquisition data volume and data browsing function；The unification
It searches for and is retrieved for the keyword to acquisition content, according to keywords search acquisition content results collection information；
Recognition unit, for acquisition terminal by the identification information collection of the sequence memory response of identification information collection；
Acquisition unit carries out channel radio by the identification information collection of storage and other measuring terminals for acquisition terminal
Letter acquires data；
Storage unit is transmitted, carries out transimission and storage for the data completed will to be acquired.
As shown in figure 4, resource data encrypting module 5 includes:
Key expansion unit, for being extended according to predetermined Hadamard orthomorphic matrices to key；
Data distribution time processing unit, for being taken according to data after data server receives data encryption request
The data encryption equipment information list stored in business device obtains the last assigned data encryption equipment, obtains the data
The corresponding distribution start time information of all data encryption equipments in facility information list is encrypted, by all data encryption equipments
The size of corresponding distribution initial time is compared, and determines maximum distribution initial time, when the maximum distribution originates
When time is not sky, using the maximum corresponding data encryption equipment of initial time that distributes as the last assigned number
According to encryption equipment；
Data encryption request unit, the location information of the data encryption equipment assigned for the last time and point of setting
With rule, it is that some idle data add that data encryption, which is requested assignment to the state in the data encryption equipment information list,
Close equipment, and be busy by status modifier of some data encryption equipment in data encryption equipment information list；
Hashed value presets unit, for calculating the corresponding length hash value of length of new data by presetting hashing algorithm,
And length hash value and the default redundant data are spliced into target data, and target data is calculated by default hashing algorithm
Hashed value；
Data transmission unit is used for encrypted Deta bearer in data frame, and passes through communication network for the data frame
It is sent to opposite end communication equipment.
2 detection method of abnormality detection module provided by the invention is as follows:
(1) a performance number strong point to be detected for indicating the clustering performance, the performance number are obtained by detection program
Strong point includes normalized multidimensional performance indicator；
(2) it from the existing performance data class by the polymerize generation in performance number strong point acquired before, determines and is somebody's turn to do
The highest performance data class of performance number strong point to be detected similarity；Judge the performance number strong point to be detected and identified property
Whether the similarity of energy data class is more than the current similarity threshold of the performance data class；
(3) when being more than current similarity threshold, which is aggregated to identified performance
In data class, and calculate after polymerization that data point sum accounts for the data point sum of current all properties data class in the performance data class
Ratio whether be more than exception class threshold value；The performance data after performance number strong point to be detected is added is updated according to following formula
The center particle and similarity threshold of class:
Cr=(pt+cr*np)/(np+1)
Wherein, cr is center particle, and np is data point sum in class, and pt is the performance number strong point that is added, sim be pt and
The similarity of performance data class, th are exception class threshold value, and lr is the learning rate threshold value for adjusting th；
(4) when being less than exception class threshold value, to the centroplasm at the performance number strong point and the performance data class to be detected
The sum of distance maximum distance for being ranked up, and calculating predetermined ratio of each dimension performance indicator of point and all dimensions away from
From with the ratio between whether be greater than range distribution threshold value；
(5) when being greater than range distribution threshold value, determine that the performance number strong point to be detected is an abnormal point.
The step of obtaining a performance number strong point to be detected for indicating the clustering performance provided by the invention includes:
Receive a performance data group from performance collection device instruction clustering performance collected, the performance data group packet
Include multidimensional performance indicator；
Normalizing the performance data group is the performance number strong point.
The step of determination performance number strong point to be detected similarity provided by the invention highest performance data class, wraps
It includes:
Performance number strong point to be detected is calculated at a distance from the center particle of existing each performance data class；
According at a distance from the center particle of each performance data class, performance number strong point to be detected and this performance are calculated
The similarity of data class；
The determining and highest performance data class of performance number strong point to be detected similarity.
The center particle provided by the invention for calculating performance number strong point to be detected and existing each performance data class
Apart from the step of include:
Calculate the Euclidean distance of the center particle of performance number strong point to be detected and each performance data class；
It calculates performance number strong point to be detected and the operation of the similarity of this performance data class includes:
Wherein, d is the performance number strong point to be detected that is calculated at a distance from the center particle of this performance data class,
Sim is the similarity with this performance data class.
Each dimension provided by the invention to the performance number strong point to be detected and the center particle of the performance data class
The distance of performance indicator is ranked up, and calculate the maximum distance of predetermined ratio and at a distance from all dimensions and the ratio between, whether
Include: greater than the step of range distribution threshold value
Pt={ n1,...,niCr={ c1,...,ci}di=| ni-ci|niFor i-th dimension in performance number strong point pt to be detected
Performance indicator, ciFor the i-th dimension numerical value of center particle cr, diIt is pt i-th dimension at a distance from i-th dimension in cr, to the di of all dimensions
It is ranked up, and calculatesWherein, N is all dimension sums, and M is the number of dimensions of predetermined ratio in N,
For N-dimensional distance in it is maximum M be worth sum,For the sum of N number of distance；
Judge whether pr is greater than range distribution threshold value.
8 moving method of transferring module provided by the invention is as follows:
1) whether the new collection swarm parameter of configuration, environment and the function for detecting each server for including in new cluster are normal；Its
In, the new cluster is the target object of cluster migration；The each server for including in the new cluster is configured to exempt from close behaviour
Make mechanism；
2) when the function of detecting each server for including in the new cluster is normal, include to the new cluster
Each server carry out environment configurations, and start environment configurations in each server that the new cluster includes and service accordingly
Function；
3) by the Data Migration in old cluster into the new cluster；Wherein, the old cluster be to the new cluster into
The object to be migrated of row cluster migration.
Before whether the environment and function for each server for including in the new cluster of detection provided by the invention are normal,
Further include:
Release the operation data size limitation of the new cluster.
The each server provided by the invention for including to the new cluster carries out environment configurations, and starts the new cluster
Including each server in the corresponding service function of environment configurations, comprising:
Zookeeper service, Hadoop service, the Hbase service of each server that the new cluster includes are carried out
Configuration, and be written in the Host file of the management server in the new cluster the old cluster IP address and
The configuration information of HostName；
Start the service function of the Zookeeper service, Hadoop service, Hbase service.
It is provided by the invention before the Data Migration by old cluster is into the new cluster, further includes:
Detect whether the corresponding service function of environment configurations in each server for including in the new cluster starts normally；
When detecting that the corresponding service function of environment configurations starts just in each server for including in the new cluster
Chang Shi, by the Data Migration in the old cluster into the new cluster.
Data Migration provided by the invention by old cluster is into the new cluster, comprising:
Instruction is migrated using the distcp for being added to skipcrccheck parameter to arrive the Data Migration in the old cluster
In the new cluster.
The above is only the preferred embodiments of the present invention, and is not intended to limit the present invention in any form,
Any simple modification made to the above embodiment according to the technical essence of the invention, equivalent variations and modification, belong to
In the range of technical solution of the present invention.
Claims (7)
1. resource information management system under a kind of cluster environment, which is characterized in that resource information management system under the cluster environment
System includes:
Resource data acquisition module, connect with central control module, for acquiring Internet resources by the network equipment；
Abnormality detection module, connect with central control module, for being detected by detecting program to the exception of cluster；
Authentication module is accessed, is connect with central control module, for being verified by proving program to access cluster resource；
Central control module encrypts mould with resource data acquisition module, abnormality detection module, access authentication module, resource data
Block, resource classification module, resource distribution module, transferring module, safety protection module 9, alarm module, display module connection, are used
It is worked normally in controlling modules by host；
Resource data encrypting module, connect with central control module, for being added by encipheror to cluster resource data
Close processing；
Resource classification module, connect with central control module, for being classified by sort program to cluster resource；
Resource distribution module is connect with central control module, for being allocated by distribution program to cluster resource；
Transferring module is connect with central control module, for carrying out migration operation to cluster；
Safety protection module is connect with central control module, anti-for carrying out safety to cluster resource by protection capacity of safety protection software
Shield；
Alarm module is connect with central control module, for being warned by warning program to detection abnormal data；
Display module is connect with central control module, for the Internet resources and information warning by display display acquisition.
2. resource information management system under cluster environment as described in claim 1, which is characterized in that the resource data acquisition
Module includes:
Acquisition tasks are submitted to task pool for generating acquisition tasks according to acquisition strategies by acquisition tasks generation unit；
The concurrent execution unit of acquisition tasks, for going forward side by side from distribution thread execution acquisition tasks in concurrent execution thread pond are acquired
Row acquisition time-out monitoring；
Time-out control unit is acquired, if for acquisition time-out, acquisition tasks normal termination, returned data acquisition knot not to occur
Fruit, thread are released back into the concurrent execution thread pond of acquisition, acquisition tasks are added and complete list；If acquisition time-out occurs, interrupt
Acquisition tasks discharge resource, and return time-out acquires concurrent execution thread pond as a result, thread is released back into, and acquisition tasks are added super
When task list；
Resource base information maintenance unit, for being used to saving and modifying resource base information table, by the resource data table of input
Title automatically generates collection of resources information table；The resource allocation information maintenance module be used for resource information collection rule into
Row configuration, modifies resource base information table；The resource regular testing module is for testing the collection rule of configuration；
Module is issued/cancelled to the resource information for issuing and cancelling resource information, automatically generates resource after issuing resource information
Monitoring information table；
Task basic information maintenance unit, for for saving and modifying task basic information table；The starting of the task stops
Only and restart module and be used, respectively, to initiate task, stopping task and restarting task, task prison is automatically generated after starting task
Control information table；
Data Management Unit includes browsing data and unified search；The browsing data is used to acquire the unified of data and shows,
It shows the acquisition information of some resource, includes resource name, acquisition data volume and data browsing function；The unified search is used
It is retrieved in the keyword to acquisition content, according to keywords search acquisition content results collection information；
Recognition unit, for acquisition terminal by the identification information collection of the sequence memory response of identification information collection；
Acquisition unit is carried out wireless communication with other measuring terminals by the identification information collection of storage for acquisition terminal, is adopted
Collect data；
Storage unit is transmitted, carries out transimission and storage for the data completed will to be acquired.
3. resource information management system under cluster environment as described in claim 1, which is characterized in that the resource data encryption
Module includes:
Key expansion unit, for being extended according to predetermined Hadamard orthomorphic matrices to key；
Data distribution time processing unit, for data server receive data encryption request after, according to data server
The data encryption equipment information list of middle storage obtains the last assigned data encryption equipment, obtains the data encryption
The corresponding distribution start time information of all data encryption equipments in facility information list, all data encryption equipments are corresponding
The size of distribution initial time be compared, maximum distribution initial time is determined, when the maximum distribution initial time
When being not empty, the maximum corresponding data encryption equipment of the initial time data assigned as the last time of distributing are added
Close equipment；
Data encryption request unit, the location information of the data encryption equipment assigned for the last time and the distribution rule of setting
Then, data encryption is requested assignment to the state in the data encryption equipment information list is that some idle data encryption is set
It is standby, and be busy by status modifier of some data encryption equipment in data encryption equipment information list；
Hashed value presets unit, for calculating the corresponding length hash value of length of new data by presetting hashing algorithm, and will
Length hash value and the default redundant data are spliced into target data, and calculate dissipating for target data by presetting hashing algorithm
Train value；
Data transmission unit, for the data frame to be sent to encrypted Deta bearer in data frame, and by communication network
Give opposite end communication equipment.
4. resource information management under the cluster environment of resource information management system under cluster environment described in a kind of operation claim 1
Method, which is characterized in that resource information management method under the cluster environment the following steps are included:
Step 1 acquires Internet resources using the network equipment by resource data acquisition module；It is utilized by abnormality detection module
Detection program detects the exception of cluster；Access cluster resource is tested using proving program by access authentication module
Card；
Step 2, central control module encrypt cluster resource data using encipheror by resource data encrypting module
Processing；
Step 3 classifies to cluster resource using sort program by resource classification module；Pass through resource distribution module benefit
Cluster resource is allocated with distribution program；
Step 4 carries out migration operation to cluster by transferring module；
Step 5 carries out security protection to cluster resource using protection capacity of safety protection software by safety protection module；By warning mould
Block warns detection abnormal data using warning program；
Step 6 utilizes the Internet resources and information warning of display display acquisition by display module.
5. resource information management method under cluster environment as claimed in claim 4, which is characterized in that described to pass through abnormality detection
Module is as follows using abnormal progress detection method of the detection program to cluster:
(1) a performance number strong point to be detected for indicating the clustering performance, the performance number strong point are obtained by detection program
Including normalized multidimensional performance indicator；
(2) from the existing performance data class by the polymerize generation in performance number strong point acquired before, determination is to be checked with this
The highest performance data class of performance number strong point similarity of survey；Judge the performance number strong point to be detected and identified performance number
It whether is more than the current similarity threshold of the performance data class according to the similarity of class；
(3) when being more than current similarity threshold, which is aggregated to identified performance data
In class, and calculate after polymerization that data point sum accounts for the ratio of the data point sum of current all properties data class in the performance data class
Whether example is more than exception class threshold value；The performance data class after performance number strong point to be detected is added is updated according to following formula
Center particle and similarity threshold:
Cr=(pt+cr*np)/(np+1)
Wherein, cr is center particle, and np is data point sum in class, and pt is the performance number strong point being added, and sim is pt and performance
The similarity of data class, th are exception class threshold value, and lr is the learning rate threshold value for adjusting th；
(4) when being less than exception class threshold value, to the performance number strong point to be detected and the center particle of the performance data class
The sum of maximum distance that the distance of each dimension performance indicator is ranked up, and calculates predetermined ratio at a distance from all dimensions and
The ratio between whether be greater than range distribution threshold value；
(5) when being greater than range distribution threshold value, determine that the performance number strong point to be detected is an abnormal point.
6. resource information management method under cluster environment as claimed in claim 4, which is characterized in that described to pass through transferring module
It is as follows that migration operation method is carried out to cluster:
1) whether the new collection swarm parameter of configuration, environment and the function for detecting each server for including in new cluster are normal；Wherein, institute
State the target object that new cluster is cluster migration；The each server for including in the new cluster is configured to exempt from close operation machine
System；
2) when the function of detecting each server for including in the new cluster is normal, to the new cluster include it is each
A server carries out environment configurations, and starts environment configurations in each server that the new cluster includes and service function accordingly
Energy；
3) by the Data Migration in old cluster into the new cluster；Wherein, the old cluster is to be collected to the new cluster
The object to be migrated of group's migration.
7. a kind of carry under cluster environment described in claims 1 to 3 any one at the information data of resource information management system
Manage terminal.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910800797.6A CN110502346A (en) | 2019-08-28 | 2019-08-28 | Resource information management system and method under a kind of cluster environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910800797.6A CN110502346A (en) | 2019-08-28 | 2019-08-28 | Resource information management system and method under a kind of cluster environment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110502346A true CN110502346A (en) | 2019-11-26 |
Family
ID=68588689
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910800797.6A Pending CN110502346A (en) | 2019-08-28 | 2019-08-28 | Resource information management system and method under a kind of cluster environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110502346A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113660556A (en) * | 2021-07-10 | 2021-11-16 | 北京邮电大学 | Method, device, equipment and storage medium for allocating resources of optical network |
| CN116956293A (en) * | 2023-09-19 | 2023-10-27 | 天津华来科技股份有限公司 | API security vulnerability detection system and method |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103888424A (en) * | 2012-12-20 | 2014-06-25 | 航天信息股份有限公司 | Cluster-type data encryption system and data processing method thereof |
| CN105871634A (en) * | 2016-06-01 | 2016-08-17 | 北京蓝海讯通科技股份有限公司 | Method and application for detecting cluster anomalies and cluster managing system |
| CN106603292A (en) * | 2016-12-19 | 2017-04-26 | 国云科技股份有限公司 | Network resource data acquisition system and realization method thereof |
| CN108418868A (en) * | 2018-02-08 | 2018-08-17 | 佛山市三盟新能源有限公司 | A kind of educational resource collecting method |
| CN109150512A (en) * | 2018-08-22 | 2019-01-04 | 网宿科技股份有限公司 | A kind of data encryption, decryption method, system and data encryption, decryption device |
| CN109426564A (en) * | 2017-09-01 | 2019-03-05 | 上海宝信软件股份有限公司 | A kind of IT resource data acquisition unifies manufacture timeout control method and system |
| CN109818794A (en) * | 2019-01-31 | 2019-05-28 | 北京搜狐互联网信息服务有限公司 | Cluster moving method and tool |
| CN110163000A (en) * | 2019-05-15 | 2019-08-23 | 青岛黄海学院 | A kind of intelligent data acqusition system and method based on cloud computing system |
-
2019
- 2019-08-28 CN CN201910800797.6A patent/CN110502346A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103888424A (en) * | 2012-12-20 | 2014-06-25 | 航天信息股份有限公司 | Cluster-type data encryption system and data processing method thereof |
| CN105871634A (en) * | 2016-06-01 | 2016-08-17 | 北京蓝海讯通科技股份有限公司 | Method and application for detecting cluster anomalies and cluster managing system |
| CN106603292A (en) * | 2016-12-19 | 2017-04-26 | 国云科技股份有限公司 | Network resource data acquisition system and realization method thereof |
| CN109426564A (en) * | 2017-09-01 | 2019-03-05 | 上海宝信软件股份有限公司 | A kind of IT resource data acquisition unifies manufacture timeout control method and system |
| CN108418868A (en) * | 2018-02-08 | 2018-08-17 | 佛山市三盟新能源有限公司 | A kind of educational resource collecting method |
| CN109150512A (en) * | 2018-08-22 | 2019-01-04 | 网宿科技股份有限公司 | A kind of data encryption, decryption method, system and data encryption, decryption device |
| CN109818794A (en) * | 2019-01-31 | 2019-05-28 | 北京搜狐互联网信息服务有限公司 | Cluster moving method and tool |
| CN110163000A (en) * | 2019-05-15 | 2019-08-23 | 青岛黄海学院 | A kind of intelligent data acqusition system and method based on cloud computing system |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113660556A (en) * | 2021-07-10 | 2021-11-16 | 北京邮电大学 | Method, device, equipment and storage medium for allocating resources of optical network |
| CN113660556B (en) * | 2021-07-10 | 2022-04-22 | 北京邮电大学 | Method, device, equipment and storage medium for allocating resources of optical network |
| CN116956293A (en) * | 2023-09-19 | 2023-10-27 | 天津华来科技股份有限公司 | API security vulnerability detection system and method |
| CN116956293B (en) * | 2023-09-19 | 2024-01-30 | 天津华来科技股份有限公司 | API security vulnerability detection system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6637620B2 (en) | Agent allocation method, apparatus, server and recording medium | |
| CN105653398B (en) | A kind of smart allocation operating system mirror method | |
| CN109656973B (en) | Target object association analysis method and device | |
| CN104378262A (en) | Intelligent monitoring analyzing method and system under cloud computing | |
| CN110708256A (en) | CDN scheduling method, device, network equipment and storage medium | |
| CN110502346A (en) | Resource information management system and method under a kind of cluster environment | |
| CN112256682B (en) | Data quality detection method and device for multi-dimensional heterogeneous data | |
| CN111338888B (en) | Data statistics method and device, electronic equipment and storage medium | |
| CN109670932B (en) | Credit data accounting method, apparatus, system and computer storage medium | |
| CN111626896B (en) | Automatic acquisition and information management system for building engineering quality detection data | |
| CN109241154A (en) | A kind of distributed data base monitoring analysis method and system | |
| CN109167673B (en) | Novel cloud service screening method integrating abnormal Qos data detection | |
| CN117236559A (en) | Intelligent charging pile data analysis method and device based on Internet of things | |
| CN117170969A (en) | Operation and maintenance method, partition equilibrium recovery device, equipment and medium of server cluster | |
| WO2023273529A1 (en) | Service log monitoring method and apparatus, and storage medium and electronic device | |
| KR102464688B1 (en) | Method and apparatus for detrmining event level of monitoring result | |
| CN109947615A (en) | The monitoring method and device of distributed system | |
| KR101288535B1 (en) | Method for monitoring communication system and apparatus therefor | |
| CN112202850B (en) | Intelligent cabinet after-sale maintenance work order system and processing method | |
| CN113467892A (en) | Distributed cluster resource configuration method and corresponding device, equipment and medium | |
| CN112817987A (en) | Method, device, equipment and storage medium for accessing distributed storage cluster | |
| CN113298115A (en) | User grouping method, device, equipment and storage medium based on clustering | |
| CN110009420A (en) | The ticket system and ticket drawing method of value-added service | |
| CN104469302A (en) | Video monitoring optimization method and device | |
| CN114546759B (en) | Database access error monitoring and analyzing method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191126 |
|
| RJ01 | Rejection of invention patent application after publication |