CN107277151B - Timing method of job scheduling system - Google Patents
Timing method of job scheduling system Download PDFInfo
- Publication number
- CN107277151B CN107277151B CN201710511733.5A CN201710511733A CN107277151B CN 107277151 B CN107277151 B CN 107277151B CN 201710511733 A CN201710511733 A CN 201710511733A CN 107277151 B CN107277151 B CN 107277151B
- Authority
- CN
- China
- Prior art keywords
- time
- consumed
- user node
- user nodes
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1461—Backup scheduling policy
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/06—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols the encryption apparatus using shift registers or memories for block-wise or stream coding, e.g. DES systems or RC4; Hash functions; Pseudorandom sequence generators
- H04L9/0643—Hash functions, e.g. MD5, SHA, HMAC or f9 MAC
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Power Engineering (AREA)
- Computer Security & Cryptography (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses a timing method of an operation scheduling system, which comprises the following steps: the server monitors the operation of a certain user node to obtain the start time and the end time of the operation, and broadcasts the start time and the end time to all other user nodes; each user node acquires the consumed time of the operation according to the starting time and the ending time, and feeds the acquired consumed time back to all other user nodes; each user node counts a plurality of consumed time received by the user node, local backup is carried out on the consumed time according to a counting result, and meanwhile, the server stores the consumed time into a database. According to the invention, the information of the database is synchronized, timely and symmetrical when all user nodes maintain the machine, so that the safety of system data can be ensured, the risk of data loss and damage caused by centralized management is avoided, and once the block database of the central server side fails, other user nodes can timely restore the system database.
Description
Technical Field
The invention relates to the field of servers, in particular to a timing method of a job scheduling system.
Background
The block chain technology is a bottom layer technology of the bit money, which is a decentralized technology and also has information non-tamper property and anonymity, information is permanently stored once being confirmed and written into a database, unless more than 51% of nodes in the system can be controlled simultaneously, modification of the database on a single node is invalid, the whole network has no centralized hardware or management mechanism, rights and obligations between any nodes are equal, and damage or loss of any single node does not affect the operation of the whole system, so that the data stability and reliability are extremely high.
In addition, in the high-performance computing field, the machine hour charging (counting the time consumed by running the job by each node) of the client is a centralized operation and management mode, when the user node submits the job, the record and the storage of the machine hour are operated by the server, and the server is separately responsible for counting and recording the machine hour use condition of the user node. However, the centralized management mode often has data opacity, the data of the server (or the management) and the user node are asymmetric, and once the central server fails, the problems of data loss, damage and the like are caused, and in the centralized management mode, the server has the right to maliciously tamper with the use time of the user, and the rights and interests of the user are damaged.
An effective solution to the problems in the related art has not been proposed yet.
Disclosure of Invention
Aiming at the problems in the related art, the invention provides a timing method of an operation scheduling system, which is characterized in that data of a recorder is managed and coordinated through a server and a user node, and the data symmetry of the server and the user node is also ensured, so that the risk of data loss and damage caused by centralized management can be avoided.
The technical scheme of the invention is realized as follows:
according to one aspect of the invention, a timing method for a job scheduling system is provided.
The timing method of the job scheduling system comprises the following steps: the server monitors the operation of a certain user node to obtain the start time and the end time of the operation, and broadcasts the start time and the end time to all other user nodes; each user node acquires the consumed time of the operation according to the starting time and the ending time, and feeds the acquired consumed time back to all other user nodes; each user node counts a plurality of consumed time received by the user node, local backup is carried out on the consumed time according to a counting result, and meanwhile, the server stores the consumed time into a database.
According to an embodiment of the present invention, the client monitoring the job of a certain user node, before obtaining the start time and the end time of the job, includes: creating a block chain network, wherein a server and all user nodes are in the block chain network; and a certain user node submits the operation through the blockchain network.
According to one embodiment of the invention, the start time and the end time are broadcast to all remaining user nodes by a hashing algorithm.
According to an embodiment of the present invention, performing local backup in a time consuming manner according to the statistical result includes: in the case where more than 51% of the user node feedbacks take time to be consistent, it will take time to make a local backup.
The invention has the beneficial technical effects that:
the invention monitors the operation of a certain user node through the service end to obtain the starting time and the ending time of the operation, broadcasts the starting time and the ending time to all other user nodes, then each user node obtains the consumed time of the operation according to the starting time and the ending time, feeds the obtained consumed time back to all other user nodes, finally each user node counts a plurality of consumed times received by itself, locally backups the consumed time according to the counting result, and simultaneously stores the consumed time into the database by the service end, so that all user nodes commonly maintain the machine-hour database, the information is synchronous, timely and symmetrical, the safety of system data can be ensured, the data loss and damage risk caused by centralized management are avoided, and once the block database of the central service end fails, and other user nodes can restore the system database in time.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a flow diagram of a timing method of a job scheduling system according to an embodiment of the present invention;
fig. 2 is a specific flowchart of a timing method of a job scheduling system according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments that can be derived by one of ordinary skill in the art from the embodiments given herein are intended to be within the scope of the present invention.
For a better understanding of the present invention, the following description refers to words commonly used in the art.
The Hash algorithm is a one-way cryptosystem, namely, the Hash algorithm is irreversible mapping from a plaintext to a ciphertext, only has an encryption process and does not have a decryption process;
bitcoin is a digital currency in the form of P2P (peer-to-peer network);
the block chain is a technical solution similar to NOSQL (Not Only SQL, non-relational database), is Not a specific technology, can realize the block chain technology through a plurality of programming languages and architectures, and has larger distributed scale and no central node;
high performance computing refers to computing systems and environments that typically use many processors (as part of a single machine) or several computers organized in a cluster (operating as a single computing resource);
the operation refers to a group of associated processing procedures managed as a whole, basic execution objects of a batch processing subsystem and shell scripts running in a Linux session;
the job scheduling System is a software System responsible for monitoring and managing resources and jobs in a cluster, and there are a PBS (Portable Batch System), an SGE (Sun Grid, Sun Grid engine), an LSF (Load Sharing Facility), and the like in common;
the machine-time means time taken to run the job.
According to an embodiment of the present invention, a timing method of a job scheduling system is provided.
As shown in fig. 1, a timing method of a job scheduling system according to an embodiment of the present invention includes:
step S101, a server monitors the operation of a certain user node to obtain the start time and the end time of the operation, and broadcasts the start time and the end time to all other user nodes;
step S103, each user node acquires the consumed time of the operation according to the starting time and the ending time, and feeds the acquired consumed time back to all other user nodes;
and step S105, each user node counts a plurality of consumed time received by the user node, performs local backup on the consumed time according to the counting result, and simultaneously stores the consumed time into a database by the server.
By means of the technical scheme, the operation of a certain user node is monitored through the service end to obtain the starting time and the ending time of the operation, the starting time and the ending time are broadcasted to all other user nodes, then each user node obtains the consumed time of the operation according to the starting time and the ending time and feeds the obtained consumed time back to all other user nodes, finally each user node counts a plurality of consumed times received by the user node, local backup is carried out on the consumed time according to the counting result, and the consumed time is stored in the database by the service end, so that all user nodes commonly maintain the machine-hour database, information is synchronized and symmetrical in time, the safety of system data can be ensured, the data loss and damage risks caused by centralized management are avoided, and once the block database of the central service end fails, and other user nodes can restore the system database in time.
According to an embodiment of the present invention, the client monitoring the job of a certain user node, before obtaining the start time and the end time of the job, includes: creating a block chain network, wherein a server and all user nodes are in the block chain network; and a certain user node submits the operation through the blockchain network.
According to one embodiment of the invention, the start time and the end time are broadcast to all remaining user nodes by a hashing algorithm.
According to an embodiment of the present invention, performing local backup in a time consuming manner according to the statistical result includes: in the case where more than 51% of the user node feedbacks take time to be consistent, it will take time to make a local backup.
In order to better describe the present invention, the following detailed description is given by way of specific examples.
With continued reference to fig. 2, the specific process of the technical solution of the present invention is as follows:
step S21, establishing an on-machine billing block chain network, and uniformly adding a server and all user nodes into the block chain network;
step S22, a certain user node submits operation through the blockchain network, and the system broadcasts the operation start time to each node in the network through the Hash encryption algorithm;
step S23, the operation is finished, the system broadcasts the operation end time to each node in the network through the Hash encryption algorithm;
step S24, each user node counts the consumed time of the operation and feeds back the consumed time to all other nodes in the network;
step S25, each user node locally counts the received feedback information, if the return information of more than 51% of the nodes is consistent, the service end writes the operation running time of the operation into a database;
step S26, other user nodes locally make data backup;
in step S27, the time counting is ended.
In summary, with the above technical solution of the present invention, the server monitors the operation of a certain user node to obtain the start time and the end time of the operation, and broadcasts the start time and the end time to all other user nodes, and then each user node obtains the consumed time of the operation according to the start time and the end time, and feeds back the obtained consumed time to all other user nodes, and finally each user node counts a plurality of consumed times received by itself, and performs local backup on the consumed time according to the statistical result, and the server stores the consumed time into the database, so that the server and the user node manage and cooperate with the data of the recorder, and all user nodes maintain the database of the recorder together, and the information is synchronized in time and symmetrical, thereby ensuring the security of the system data, in addition, local backup data of a user side and data recorded by a server side are subjected to collaborative verification and collaborative management, so that once the block database of the central server side fails, other user nodes can restore the system database in time.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (4)
1. A timing method of a job scheduling system, comprising:
establishing an on-machine charging block chain network, wherein a server side and all user nodes of the job scheduling system are positioned in the on-machine charging block chain network;
the server monitors the operation of a certain user node to obtain the start time and the end time of the operation, and broadcasts the start time and the end time to all other user nodes;
each user node acquires the consumed time of the operation according to the starting time and the ending time, and feeds the acquired consumed time back to all other user nodes;
the user nodes count a plurality of consumed time received by the user nodes, carry out local backup on the consumed time according to the counting result, and simultaneously store the consumed time into a database by the server.
2. A timing method according to claim 1, wherein said creating an on-machine billing blockchain network comprises:
and the certain user node submits the operation through the machine-hour charging block chain network.
3. Timing method according to claim 1, characterized in that said start time and said end time are broadcasted to all remaining user nodes by means of a hashing algorithm.
4. A method of timekeeping according to claim 1, wherein the locally backing up the elapsed time according to the statistical result comprises:
and in the case that the consumed time of more than 51% of the user node feedbacks is consistent, performing local backup on the consumed time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710511733.5A CN107277151B (en) | 2017-06-27 | 2017-06-27 | Timing method of job scheduling system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710511733.5A CN107277151B (en) | 2017-06-27 | 2017-06-27 | Timing method of job scheduling system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107277151A CN107277151A (en) | 2017-10-20 |
| CN107277151B true CN107277151B (en) | 2020-11-03 |
Family
ID=60071223
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710511733.5A Active CN107277151B (en) | 2017-06-27 | 2017-06-27 | Timing method of job scheduling system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107277151B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1787000A (en) * | 2005-11-15 | 2006-06-14 | 中国工商银行股份有限公司 | System and method for centralized managing and dispatching of batch work |
| CN106209947A (en) * | 2015-05-07 | 2016-12-07 | 中兴通讯股份有限公司 | The data processing method of a kind of decentration autonomy organization and system |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8082546B2 (en) * | 2006-09-29 | 2011-12-20 | International Business Machines Corporation | Job scheduling to maximize use of reusable resources and minimize resource deallocation |
| CN101674194B (en) * | 2009-09-28 | 2011-11-09 | 北京航空航天大学 | Cluster load model based on log feature analysis and modeling method thereof |
| CN105354111A (en) * | 2015-10-29 | 2016-02-24 | 国电南瑞科技股份有限公司 | Redundancy backup method suitable for wide-area distributed real-time database |
| CN105630609B (en) * | 2016-02-24 | 2021-05-11 | 杭州复杂美科技有限公司 | Block chain packing storage method |
| CN105976231A (en) * | 2016-06-24 | 2016-09-28 | 深圳前海微众银行股份有限公司 | Asset management method based on intelligent block chain contracts and nodes |
| CN106815728A (en) * | 2017-01-03 | 2017-06-09 | 北京供销科技有限公司 | A kind of big data based on block chain technology really weighs method and system |
-
2017
- 2017-06-27 CN CN201710511733.5A patent/CN107277151B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1787000A (en) * | 2005-11-15 | 2006-06-14 | 中国工商银行股份有限公司 | System and method for centralized managing and dispatching of batch work |
| CN106209947A (en) * | 2015-05-07 | 2016-12-07 | 中兴通讯股份有限公司 | The data processing method of a kind of decentration autonomy organization and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107277151A (en) | 2017-10-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6653358B2 (en) | Blockchain logging of data from multiple systems | |
| JP6775086B2 (en) | Blockchain monitoring and management | |
| US20200143372A1 (en) | Methods for decentralized digital asset transfer and smart contract state transition | |
| US10586210B2 (en) | Blockchain checkpoints and certified checkpoints | |
| CN110019240B (en) | Service data interaction method, device and system | |
| US7885412B2 (en) | Pre-generation of generic session keys for use in communicating within communications environments | |
| TWI733514B (en) | A storage system, a network node of a blockchain network, and a blockchain-based log-structured storage system | |
| CN111837115A (en) | Shared blockchain data storage | |
| JP7050955B2 (en) | Prioritize storage of shared blockchain data | |
| CN107451147B (en) | Method and device for dynamically switching kafka clusters | |
| CN108667717B (en) | Block chain processing method, medium, device and computing equipment based on instant messaging message record | |
| CN111930851A (en) | Control data processing method, device, medium and electronic equipment of block chain network | |
| US11689616B2 (en) | Optimization of delivery of blocks | |
| CN112749969B (en) | Data processing method and device, computer equipment and storage medium | |
| CN103581332B (en) | HDFS framework and pressure decomposition method for NameNodes in HDFS framework | |
| CN109919691B (en) | Data processing system, method and device | |
| CN111046109A (en) | Cross-chain task processing method, device and equipment and readable storage medium | |
| EP3483749A1 (en) | Method for parallel maintenance of data consistency | |
| CN103986748A (en) | Method and device for achieving servitization | |
| US11816069B2 (en) | Data deduplication in blockchain platforms | |
| US20220019490A1 (en) | Blockchain event processing method and apparatus | |
| JP2022500730A (en) | Methods, devices, and systems for real-time data consistency checks in distributed heterogeneous storage systems | |
| WO2020216077A1 (en) | Blockchain event deduplication method and apparatus, and computer device and storage medium | |
| CN111813868B (en) | Data synchronization method and device | |
| CN116896564B (en) | Block generation method and device based on block chain network and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20211011 Address after: 100089 building 36, courtyard 8, Dongbeiwang West Road, Haidian District, Beijing Patentee after: Dawning Information Industry (Beijing) Co.,Ltd. Patentee after: ZHONGKE SUGON INFORMATION INDUSTRY CHENGDU Co.,Ltd. Address before: 100193 No. 36 Building, No. 8 Hospital, Wangxi Road, Haidian District, Beijing Patentee before: Dawning Information Industry (Beijing) Co.,Ltd. |
|
| TR01 | Transfer of patent right |