CN113630446A - File distribution method and system based on path diffusion and readable storage medium - Google Patents
File distribution method and system based on path diffusion and readable storage medium Download PDFInfo
- Publication number
- CN113630446A CN113630446A CN202110820202.0A CN202110820202A CN113630446A CN 113630446 A CN113630446 A CN 113630446A CN 202110820202 A CN202110820202 A CN 202110820202A CN 113630446 A CN113630446 A CN 113630446A
- Authority
- CN
- China
- Prior art keywords
- node
- path
- nodes
- file
- diffusion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/20—Education
- G06Q50/205—Education administration or guidance
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/02—Topology update or discovery
Abstract
The invention discloses a file distribution method, a system and a readable storage medium based on path diffusion, wherein the method comprises the following steps: after detecting that each main node in the path is started, determining a first path value between an initial node and each main node in the path, and determining a plurality of path diffusion nodes according to the first path value; distributing the file to be distributed to a plurality of path diffusion nodes from the initial node, taking the path diffusion nodes which receive the file to be distributed as target diffusion nodes, and determining second path values between the target diffusion nodes and other main nodes; determining a node to be diffused according to the second path value, and distributing the file to be distributed to the node to be diffused by the target diffusion node; and setting the nodes to be diffused as new target diffused nodes, and executing the step of determining second path values between the target diffused nodes and other main nodes until all the main nodes in the paths receive the files to be distributed. The invention realizes the quick and effective transmission of various files in a path diffusion mode.
Description
Technical Field
The invention relates to the technical field of internet, in particular to a file distribution method and system based on path diffusion and a readable storage medium.
Background
With the popularization and rapid development of the internet, more and more fields rely on the internet to realize various functions, for example, the traditional field teaching is gradually developed into the online teaching through the internet. For online teaching, teaching materials are generally transmitted from a teacher to students in a manner of transmitting the teaching materials to terminals of the students in a divergent manner centering on a terminal used by the teacher, as shown in fig. 1.
In the teaching material transmission mode, the network bandwidth of the teacher terminal is the bottleneck of the whole transmission. All student terminals wait for the data transmitted from the teacher terminal, and once the transmitted data is large, the transmission speed is very slow. In addition, there is a problem of a single point of failure of the teacher terminal. If the teacher's terminal fails, the entire transfer is disabled.
Therefore, how to realize the rapid and effective transmission of teaching materials is a technical problem to be solved urgently at present.
Disclosure of Invention
The invention mainly aims to provide a file distribution method, a file distribution system and a readable storage medium based on path diffusion, and aims to solve the technical problem of how to realize quick and effective transmission of teaching materials in the prior art.
In order to achieve the above object, the present invention provides a file distribution method based on path diffusion, including:
after detecting that all main nodes in a path are started, determining first path values between an initial node in the path and all the main nodes respectively, and determining a plurality of path diffusion nodes from all the main nodes according to the first path values;
distributing a file to be distributed to a plurality of path diffusion nodes from the initial node, taking the path diffusion node which receives the file to be distributed as a target diffusion node, and determining a second path value between the target diffusion node and other main nodes which do not start to receive the file to be distributed in each main node;
determining a node to be diffused corresponding to the target diffusion node in each other main node according to the second path value, and distributing the file to be distributed to the node to be diffused by the target diffusion node;
after the to-be-distributed file is received by the to-be-distributed node, setting the to-be-distributed node as a new target diffusion node, and executing a step of determining a second path value between the target diffusion node and other host nodes which do not start to receive the to-be-distributed file in each host node until each host node in the path receives the to-be-distributed file.
Optionally, the step of distributing the file to be distributed from the initial node to a plurality of path diffusion nodes includes:
determining the sequence of sending time among the path diffusion nodes according to the magnitude relation among the first path values respectively corresponding to the path diffusion nodes;
and distributing the files to be distributed from the initial node to a plurality of path diffusion nodes in sequence according to the sequence of the sending time.
Optionally, the step of determining, according to the second path value, a node to be diffused, which corresponds to the target diffusion node, in each of the other master nodes includes:
comparing the second path values to determine the minimum value of the second path values;
and searching other main nodes corresponding to the minimum value in the other main nodes, and taking the searched other main nodes as the nodes to be diffused corresponding to the target diffusion node.
Optionally, the step of distributing the file to be distributed to the node to be diffused by the target diffusion node includes:
judging whether the node to be diffused is in a file receiving state, if so, searching a second path value with a numerical value in the order above the minimum value according to the ascending order of the second path values;
taking other main nodes corresponding to the searched second path value in the other main nodes as new nodes to be diffused, and executing the step of judging whether the nodes to be diffused are in a file receiving state;
and if the node to be diffused is not in the state of receiving the file, distributing the file to be distributed to the node to be diffused by the target diffusion node.
Optionally, the step of determining a plurality of path diffusion nodes from each of the master nodes according to each of the first path values includes:
sequencing the first path values in a descending order to generate a numerical sequence;
screening a plurality of first path values arranged in the front from the numerical sequence, and taking the main nodes respectively corresponding to the screened first path values as a plurality of path diffusion nodes;
wherein the number of the first path values is the same as the concurrent number of the initial nodes.
Optionally, the step of determining a first path value between each initial node in the path and each master node includes:
based on the initial node running preset command, the preset command transmits test data to each main node respectively;
and determining each first path value according to the time for each main node to respectively return feedback data to the initial node based on the test data.
Optionally, after detecting that all the master nodes in the path are started, the step of determining a first path value between each initial node in the path and each master node includes:
detecting whether each main node in the path returns a heartbeat, if so, verifying the effectiveness of each main node in the path according to the heartbeat returned by each main node, and judging that each main node in the path is started after verifying that each main node is effective;
if it is detected that at least one main node in the path does not return to the heartbeat, it is determined that an un-started main node exists in the path, and the un-started main node is searched for restarting.
Optionally, the step of detecting whether each master node in the path returns a heartbeat includes:
dividing each distribution node in the path into a master node and a slave node, wherein each distribution node comprises a master node and at least one slave node, and at least one slave node is subordinate to the master node so as to perform auxiliary processing on the file to be distributed received by the master node;
and controlling the starting of each main node and receiving the heartbeat returned after the starting of each main node.
Further, to achieve the above object, the present invention further provides a file distribution system based on path diffusion, including: a memory, a processor and a control program stored on the memory and executable on the processor, the control program when executed by the processor implementing the steps of the path diffusion based file distribution method as described above.
Further, to achieve the above object, the present invention also provides a readable storage medium having stored thereon a control program which, when executed by a processor, implements the steps of the path diffusion-based file distribution method as described above.
The invention discloses a file distribution method, a system and a readable storage medium based on path diffusion, which transmit various files including teaching materials in a path diffusion mode, determine first path values between an initial node and each main node in a path after detecting that each main node in the path is started, and determine a plurality of path diffusion nodes from each main node according to each first path value; the path diffusion nodes are nodes for directly sending files by the initial nodes; then, taking the file to be sent as a culture to be distributed, distributing the file to a plurality of path diffusion nodes by the initial node, taking any path diffusion node as a target diffusion node after receiving the file to be distributed, and determining a second path value between the path diffusion node and other main nodes which do not start to receive the file to be distributed; and then according to the second path value, determining a node to be diffused corresponding to the target diffusion node, distributing the file to be distributed to the node to be diffused from the target diffusion node, setting the node to be diffused, which receives the file to be distributed, as a new target diffusion node, determining the corresponding node to be diffused to distribute the file, and repeating the steps until the file to be distributed is distributed to each main node in the path to be distributed. Therefore, a single node is prevented from being used as a distribution center to distribute files to other nodes, and after the initial node distributes the files to each path diffusion node, the files are continuously distributed to other nodes by each path diffusion node, so that the parallel transmission of the files is realized, and the improvement of the transmission speed is facilitated; meanwhile, even if one node in the path diffusion nodes fails, the file distribution of other path diffusion nodes cannot be influenced, so that the rapid and effective transmission of various files including teaching materials is realized.
Drawings
FIG. 1 is a schematic diagram of a prior art teaching material being sent in a divergent fashion to various student terminals;
FIG. 2 is a schematic structural diagram of a hardware operating environment according to an embodiment of the file distribution system based on path diffusion;
FIG. 3 is a flowchart illustrating a first embodiment of a method for distributing files based on path diffusion according to the present invention;
FIG. 4 is a flowchart illustrating a second embodiment of a method for distributing files based on path diffusion according to the present invention;
FIG. 5 is a flowchart illustrating a third embodiment of a method for distributing files based on path diffusion according to the present invention;
FIG. 6 is a flowchart illustrating a fourth embodiment of a method for distributing files based on path diffusion according to the present invention;
fig. 7 is a schematic diagram of a distribution path of a file to be distributed in the file distribution method based on path diffusion according to the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention provides a file distribution system based on path diffusion.
Referring to fig. 2, fig. 2 is a schematic structural diagram of a hardware operating environment according to an embodiment of the file distribution system based on path diffusion.
As shown in fig. 2, the path flooding based file distribution system may include a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a memory device separate from the processor 1001 described above.
Those skilled in the art will appreciate that the hardware architecture of the path-flooding based file distribution system shown in FIG. 2 does not constitute a limitation of the path-flooding based file distribution system, and may include more or fewer components than those shown, or some components in combination, or a different arrangement of components.
As shown in fig. 2, the memory 1005, which is a readable storage medium, may include therein an operating system, a network communication module, a user interface module, and a control program. The operating system is a program for managing and controlling the file distribution system and the software resources based on the path diffusion, and supports the operation of a network communication module, a user interface module, a control program and other programs or software; the network communication module is used to manage and control the network interface 1004; the user interface module is used to manage and control the user interface 1003.
In the hardware structure of the file distribution system based on path diffusion shown in fig. 2, the network interface 1004 is mainly used for connecting a background server and performing data communication with the background server; the user interface 1003 is mainly used for connecting a client (user side) and performing data communication with the client; the processor 1001 may call the control program stored in the memory 1005 and perform the following operations:
after detecting that all main nodes in a path are started, determining first path values between an initial node in the path and all the main nodes respectively, and determining a plurality of path diffusion nodes from all the main nodes according to the first path values;
distributing a file to be distributed to a plurality of path diffusion nodes from the initial node, taking the path diffusion node which receives the file to be distributed as a target diffusion node, and determining a second path value between the target diffusion node and other main nodes which do not start to receive the file to be distributed in each main node;
determining a node to be diffused corresponding to the target diffusion node in each other main node according to the second path value, and distributing the file to be distributed to the node to be diffused by the target diffusion node;
after the to-be-distributed file is received by the to-be-distributed node, setting the to-be-distributed node as a new target diffusion node, and executing a step of determining a second path value between the target diffusion node and other host nodes which do not start to receive the to-be-distributed file in each host node until each host node in the path receives the to-be-distributed file.
Further, the step of distributing the file to be distributed from the initial node to a plurality of path diffusion nodes comprises:
determining the sequence of sending time among the path diffusion nodes according to the magnitude relation among the first path values respectively corresponding to the path diffusion nodes;
and distributing the files to be distributed from the initial node to a plurality of path diffusion nodes in sequence according to the sequence of the sending time.
Further, the step of determining, according to the second path value, a node to be diffused, which corresponds to the target diffusion node, in each of the other master nodes includes:
comparing the second path values to determine the minimum value of the second path values;
and searching other main nodes corresponding to the minimum value in the other main nodes, and taking the searched other main nodes as the nodes to be diffused corresponding to the target diffusion node.
Further, the step of distributing the file to be distributed to the node to be diffused by the target diffusion node comprises:
judging whether the node to be diffused is in a file receiving state, if so, searching a second path value with a numerical value in the order above the minimum value according to the ascending order of the second path values;
taking other main nodes corresponding to the searched second path value in the other main nodes as new nodes to be diffused, and executing the step of judging whether the nodes to be diffused are in a file receiving state;
and if the node to be diffused is not in the state of receiving the file, distributing the file to be distributed to the node to be diffused by the target diffusion node.
Further, the step of determining a plurality of path diffusion nodes from each of the master nodes according to each of the first path values includes:
sequencing the first path values in a descending order to generate a numerical sequence;
screening a plurality of first path values arranged in the front from the numerical sequence, and taking the main nodes respectively corresponding to the screened first path values as a plurality of path diffusion nodes;
wherein the number of the first path values is the same as the concurrent number of the initial nodes.
Further, the step of determining a first path value between each initial node in the path and each master node includes:
based on the initial node running preset command, the preset command transmits test data to each main node respectively;
and determining each first path value according to the time for each main node to respectively return feedback data to the initial node based on the test data.
Further, before the step of determining the first path values between the initial node and each of the master nodes in the path after detecting that each of the master nodes in the path is started; the processor 1001 may call the control program stored in the memory 1005 and perform the following operations:
detecting whether each main node in the path returns a heartbeat, if so, verifying the effectiveness of each main node in the path according to the heartbeat returned by each main node, and judging that each main node in the path is started after verifying that each main node is effective;
if it is detected that at least one main node in the path does not return to the heartbeat, it is determined that an un-started main node exists in the path, and the un-started main node is searched for restarting.
Further, before the step of detecting whether each master node in the path returns a heartbeat; the processor 1001 may call the control program stored in the memory 1005 and perform the following operations:
dividing each distribution node in the path into a master node and a slave node, wherein each distribution node comprises a master node and at least one slave node, and at least one slave node is subordinate to the master node so as to perform auxiliary processing on the file to be distributed received by the master node;
and controlling the starting of each main node and receiving the heartbeat returned after the starting of each main node.
The implementation of the file distribution system based on path diffusion is basically the same as that of the file distribution method based on path diffusion, and is not described herein again.
The invention provides a file distribution method based on path diffusion, and referring to fig. 3, fig. 3 is a schematic flow diagram of a first embodiment of the file distribution method based on path diffusion.
While a logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than here. Specifically, the file distribution method based on path diffusion in the embodiment includes:
step S10, after detecting that all the main nodes in the path are started, determining first path values between the initial nodes in the path and all the main nodes respectively, and determining a plurality of path diffusion nodes from all the main nodes according to all the first path values;
the file distribution method based on path diffusion is applied to a distribution terminal needing to distribute files, and the distribution terminal and a plurality of receiving terminals needing to receive the files form the file distribution system. And a transmission channel between the distribution terminal and the plurality of receiving terminals and a transmission channel between the receiving terminals and the receiving terminals form a path for distribution, the plurality of receiving terminals form a master node in the path, and the distribution terminal distributes the file to the plurality of receiving terminals serving as the master node according to the path diffusion.
Specifically, whether all the main nodes in the path are started is detected, and if all the main nodes in the path are started, a first path value from an initial node in the path to each main node is determined. The initial node is formed by a distribution node, the first path value reflects the speed of transmission between the initial node and the main node, and the initial node can send test data to the main node to determine the speed.
Further, after each master node determines a first path value between each master node and the initial node, a plurality of path diffusion nodes may be determined from each master node. The path diffusion node is used for directly receiving the file from the initial node and then sending the file to other nodes; and the number thereof can be determined by the concurrent number set for the initial node in advance, and can also be determined by limiting the range of the path value. And comparing the preset concurrency number with each other, and selecting the main nodes with high transmission speed from the first path values, wherein the selected number is the same as the concurrency number.
And for the range of the limited path value, presetting a numerical range with the representation speed higher than a certain value, and when the first path value is in the numerical range, taking the main node generating the first path value as a path diffusion node so as to select a plurality of path diffusion nodes from the main nodes.
Specifically, as shown in fig. 7, where V1 is an initial node, V2 to V7 are main nodes, a value on an arrow indicated by V1 is a first path value, and V2, V3, and V4 are definite path diffusion nodes.
Step S20, distributing the file to be distributed from the initial node to a plurality of path diffusion nodes, taking the path diffusion node that has received the file to be distributed as a target diffusion node, and determining a second path value between the target diffusion node and another host node that has not started to receive the file to be distributed in each host node;
furthermore, files to be distributed are respectively distributed to the path diffusion nodes as files to be distributed from the initial node, and the time for the initial node to send the files to be distributed to each path diffusion node is different because the first initial values between different path diffusion nodes and the initial node are different. The smaller the first initial value is, the higher the transmission speed is, the less time is spent on sending the file to be distributed, and the earlier the path diffusion node receives the file to be distributed.
In addition, the file to be distributed may be distributed to each path diffusion node by the initial node at the same time, or may be distributed one by one, and this embodiment preferably distributes one by one. Specifically, the step of distributing the file to be distributed from the initial node to a plurality of path diffusion nodes includes:
step S21, determining the sequence of the sending time among the path diffusion nodes according to the magnitude relation among the first path values respectively corresponding to the path diffusion nodes;
and step S22, sequentially distributing the file to be distributed from the initial node to a plurality of path diffusion nodes according to the sequence of the sending time.
Further, the first path values of the path diffusion nodes are compared, a size relation is determined, the path diffusion nodes are sequenced according to the size relation, and the sequence of the sending time among the path diffusion nodes is generated. The path diffusion nodes with larger first path values are arranged in front of the node diffusion nodes, and the path diffusion nodes with smaller first path values are arranged behind the node diffusion nodes; according to the sequence of the sending time, the file to be distributed is sent to the path diffusion node with the large first path value, so that the path diffusion node receives the file to be distributed and then distributes the file to other main nodes, the non-interference parallel distribution between the initial node and the path diffusion node is achieved, and the distribution efficiency is improved.
It should be noted that the file to be distributed can be distributed in an encrypted manner, so as to ensure the security of the file to be distributed and avoid file leakage. Specifically, after acquiring a file to be distributed, an initial node judges whether the file to be distributed is an encrypted distributed file according to a privacy level identifier of the file to be distributed; and if the file is an encrypted distribution file, identifying an encryption scheme corresponding to the privacy level identification, encrypting the file to be distributed according to the corresponding encryption scheme, and distributing the encrypted file to be distributed to each path diffusion node.
Further, after a certain path diffusion node receives the file to be distributed, the path diffusion node is set as a target diffusion node, and a second path value between the target diffusion node and other main nodes which are not left to receive the file to be distributed is calculated based on the target diffusion node. As for fig. 7, if V2 has received the file to be distributed, it is taken as the target diffusion node, and the second path values between it and V5, V6 and V7, i.e. the values on the respective arrows from V2 are calculated; and, it is also detected whether V1 starts distributing the file to be distributed to V3. If the distribution of the file to be distributed to the V3 is started, the second path value with the V3 is not calculated, and if the distribution of the file to be distributed to the V3 has not started, the second path value with the V3 is also calculated.
And if the path diffusion node receives the file to be distributed, returning a file size value to the initial node through the path diffusion node for judgment. Specifically, after the initial node receives a file size value returned by any path diffusion node, comparing the file size value with a reference size value in the initial node, and judging whether the file size value is matched with the reference size value; the file size value is a data size value of a file to be distributed received by the path diffusion node, and the reference size value is a data size value of the file to be distributed from the initial node to the path diffusion node. If the file size value is matched with the reference size value, the path diffusion node receives a complete file to be distributed, a distribution completion identifier is distributed to the path diffusion node, the path diffusion node is used as a target diffusion node based on the distribution completion identifier, the file size value is formed into the reference size value of the target diffusion node, and the reference size value is used for judging whether a main node which issues the file receives the complete file to be distributed.
Further, if the file size value is determined to be not matched with the reference size value through comparison, counting the number of unmatched times, and judging whether the number of times is greater than the preset number of times; the preset times are preset numerical values used for reflecting that the number of times that the main node does not receive the complete file to be distributed is large and abnormal possibly exists. If the counted times are greater than the preset times, outputting prompt information of the abnormity of the path diffusion node and prompting the timely repair of the path diffusion node; if the counted times are less than or equal to the preset times, the initial node continues to distribute the file to be distributed to the path diffusion node again.
Step S30, according to the second path value, determining a node to be diffused corresponding to the target diffusion node in each other host node, and distributing the file to be distributed to the node to be diffused from the target diffusion node;
furthermore, after calculating the second path values between the target diffusion node and other host nodes which have not started to receive the file to be distributed, the node to be diffused corresponding to the target diffusion node can be determined from the other host nodes according to the magnitude relationship between the second path values, so that the target diffusion node can distribute the file to be distributed to the node to be diffused. Specifically, the step of determining, according to the second path value, a node to be diffused, which corresponds to the target diffusion node, in each of the other master nodes includes:
step S31, comparing the second path values, and determining a minimum value of the second path values;
step S32, finding other host nodes corresponding to the minimum value in each of the other host nodes, and using the found other host nodes as the to-be-diffused nodes corresponding to the target diffused node.
Further, the second path value represents the speed of transmission between the target diffusion node and other main nodes, and the minimum value is searched by comparing the second path values. The other main nodes generating the minimum value have the fastest transmission speed with the target transmission node, so that the other main nodes generating the minimum value are searched out from the other main nodes to serve as the nodes to be diffused corresponding to the target diffusion node, the main nodes distribute the files to be distributed to the nodes to be expanded, and the rapid transmission of the files to be distributed is achieved.
Step S40, after the to-be-diffused node receives the to-be-distributed file, setting the to-be-diffused node as a new target diffused node, and executing a step of determining a second path value between the target diffused node and another host node that has not started to receive the to-be-distributed file in each host node until each host node in the path receives the to-be-distributed file.
Further, whether the file to be distributed is received by the node to be diffused is judged according to the file size value fed back to the target diffusion node by the node to be diffused, if the file to be distributed is received, the node to be diffused is set as a new target diffusion node, a second path value between the node to be diffused and other main nodes which are not started to receive the file to be distributed is calculated on the basis of the new target diffusion node, the new node to be diffused is determined through the newly calculated second path value, and the file to be distributed is distributed to the new node to be diffused by the new target diffusion node. And circulating the steps until all the main nodes receive the files to be distributed. Specifically, as shown in fig. 7, if the V3 is determined as the to-be-diffused node by the second path value, the to-be-distributed file is distributed to the V3 by the V2, after the to-be-distributed file is received by the V3, the to-be-distributed file is set as a new target diffused node, the second path values between the new target diffused node and the V5, the second path values between the new target diffused node and the V6, and the second path values between the new target diffused node and the V7 are calculated, and the new to-be-diffused node is determined to distribute the to-be-distributed file until each main node receives the to-be-distributed file.
The invention discloses a file distribution method based on path diffusion, which transmits various files including teaching materials in a path diffusion mode, determines first path values between an initial node and each main node in a path after detecting that each main node in the path is started, and determines a plurality of path diffusion nodes from each main node according to each first path value; the path diffusion nodes are nodes for directly sending files by the initial nodes; then, taking the file to be sent as a culture to be distributed, distributing the file to a plurality of path diffusion nodes by the initial node, taking any path diffusion node as a target diffusion node after receiving the file to be distributed, and determining a second path value between the path diffusion node and other main nodes which do not start to receive the file to be distributed; and then according to the second path value, determining a node to be diffused corresponding to the target diffusion node, distributing the file to be distributed to the node to be diffused from the target diffusion node, setting the node to be diffused, which receives the file to be distributed, as a new target diffusion node, determining the corresponding node to be diffused to distribute the file, and repeating the steps until the file to be distributed is distributed to each main node in the path to be distributed. Therefore, a single node is prevented from being used as a distribution center to distribute files to other nodes, and after the initial node distributes the files to each path diffusion node, the files are continuously distributed to other nodes by each path diffusion node, so that the parallel transmission of the files is realized, and the improvement of the transmission speed is facilitated; meanwhile, even if one node in the path diffusion nodes fails, the file distribution of other path diffusion nodes cannot be influenced, so that the rapid and effective transmission of various files including teaching materials is realized.
Further, referring to fig. 4, a second embodiment of the file distribution method based on path diffusion is provided based on the first embodiment of the file distribution method based on path diffusion of the present invention.
The second embodiment of the file distribution method based on path diffusion is different from the first embodiment of the file distribution method based on path diffusion in that the step of distributing the file to be distributed to the node to be diffused by the target diffusion node comprises the following steps:
step S33, judging whether the node to be diffused is in a file receiving state, if so, searching a second path value with a numerical value in the order above the minimum value according to the ascending order of the second path values;
step S34, taking another host node corresponding to the found second path value among the other host nodes as a new node to be diffused, and executing the step of determining whether the node to be diffused is in a file receiving state;
step S35, if the node to be diffused is not in the file receiving state, the file to be distributed is distributed from the target diffusion node to the node to be diffused.
Further, there are many master nodes in the path, and more master nodes receive files to be distributed, and there may be multiple target diffusion nodes at the same time. After one target diffusion node distributes files by taking one other main node as a node to be diffused, another target diffusion node may also determine the other main node as the node to be diffused. Therefore, in order to avoid repeated distribution of the file to be distributed due to the same other master node serving as the node to be diffused, the embodiment is provided with a receiving state detection mechanism for the node to be diffused.
Specifically, whether the node to be diffused is in a file receiving state is judged according to the state identifier of the node to be diffused, and if the node to be diffused is in the file receiving state, it is indicated that other target diffusion nodes distribute the file to be distributed to the node to be diffused, and the file to be distributed does not need to be distributed to the node to be diffused. At this time, according to the ascending order of the second path values, second path values with numerical values in the order above the minimum value are searched, that is, the second path value with the numerical value larger than the minimum second path value is searched in the second path values.
Further, for the searched second path value, determining other host nodes which generate the second path value from the other host nodes as new nodes to be diffused, and judging whether the new nodes to be diffused are in a file receiving state or not. And if the node is in the file receiving state, continuously searching a next new second path value and the node to be diffused corresponding to the next new second path value for judgment. If the file is not in the file receiving state, the target diffusion node distributes the file to be distributed to the node to be diffused. And circulating the steps until all the main nodes receive the files to be received.
In this embodiment, when the target diffusion node distributes the file to be distributed to the node to be diffused, whether the node to be diffused is in the file receiving state is detected, so that the file to be distributed is prevented from being repeatedly distributed to the node to be diffused, and the file is accurately transmitted to each master node, and the transmission efficiency is improved.
Further, referring to fig. 5, a third embodiment of the file distribution method based on path diffusion is provided based on the first or second embodiment of the file distribution method based on path diffusion of the present invention.
The third embodiment of the file distribution method based on path diffusion differs from the first or second embodiment of the file distribution method based on path diffusion in that the step of determining the first path values between the initial nodes and the master nodes in the path respectively comprises:
step S11, based on the initial node running preset command, the preset command transmits test data to each main node;
step S12, determining each first path value according to the time when each master node returns feedback data to the initial node based on the test data.
In this embodiment, for the first path value between the initial node and each master node, the initial node sends test data to the master node to determine the first path value. Specifically, a preset command, which is a command preset to transmit test data to the master node, such as a "ping" command, is run in the initial node. And respectively transmitting test data to each main node through the preset command, and returning feedback data to the initial node after each main node receives the test data. And determining a first path value for transmitting data to each main node according to the time for transmitting the test data and the time for receiving the feedback data.
The first path value may be a time difference between the sending time and the receiving time, and the speed of the transmission speed is represented by the time difference, and the smaller the time difference is, the faster the transmission speed is, and otherwise, the slower the transmission speed is. It is also possible to set a coefficient corresponding to the time difference range, and use the coefficient as the first path value, for example, set the time difference range to correspond to the coefficient a between 5ms and 10ms, and to correspond to the coefficient a +2 between 11ms and 20ms, and if the time difference between sending the test data and receiving the feedback data is 8ms, the first path value may be determined to be a. Therefore, the transmission speed is represented by the coefficient, and the smaller the coefficient value is, the faster the transmission speed is, and the slower the transmission speed is. Of course, the larger the coefficient value is, the faster the transmission speed is, and the slower the transmission speed is, which is not limited.
It should be noted that the second path value between the target diffusion node and the other host node is the same as the determination method of the first path value, and is not described herein again.
Further, the step of determining a plurality of path diffusion nodes from each of the master nodes according to each of the first path values includes:
step S13, sorting the first path values in a descending order to generate a numerical sequence;
step S14 of screening a plurality of first path values arranged in the preceding order from the numerical sequence, and using master nodes corresponding to the screened plurality of first path values as a plurality of path diffusion nodes;
wherein the number of the first path values is the same as the concurrent number of the initial nodes.
Furthermore, after the first path values between the initial node and each main node are determined, the first path values are sorted from small to large to generate a numerical sequence. In this embodiment, the smaller the first path value is, the faster the transmission speed is, the more likely the corresponding master node is to be determined as a path diffusion node; therefore, for a sequence of values from small to large, the more forward the value indicates the faster the transmission speed, the more likely the corresponding master node is to be determined as a path diffusion node.
Further, in order to ensure the transmission efficiency of the file to be transmitted, the embodiment sets a concurrency mechanism for the initial node. Specifically, the concurrency number of the initial node is preset, and a plurality of first path values arranged in the front are screened from the numerical sequence, wherein the number of the screened first path values is the same as the concurrency number of the initial node. And then searching the corresponding main node for the screened first path value, and taking the searched main node as a path diffusion node. The number of the path diffusion nodes is the same as the concurrent number, so that the initial node concurrently distributes the files to be distributed to each path diffusion node, and the transmission speed of the files to be distributed is improved.
In this embodiment, by detecting the first path value between the initial node and each master node, the first path value reflects the communication transmission speed between the initial node and the master node, and then selects a plurality of first path values reflecting a high transmission speed and the master nodes corresponding to the first path values as path spreading nodes, so that the initial node concurrently transmits the file to be distributed, and efficient distribution of the file to be distributed is facilitated.
Further, referring to fig. 6, a fourth embodiment of the file distribution method based on path diffusion according to the present invention is provided based on the first, second or third embodiments of the file distribution method based on path diffusion according to the present invention.
The fourth embodiment of the file distribution method based on path diffusion is different from the first, second, or third embodiment of the file distribution method based on path diffusion in that, after detecting that each master node in a path is started, the step of determining a first path value between an initial node and each master node in the path includes:
step S50, detecting whether each main node in the path returns a heartbeat, if so, verifying the validity of each main node in the path according to the heartbeat returned by each main node, and judging that each main node in the path is started after verifying that each main node is valid;
step S60, if it is detected that at least one master node in the path does not return a heartbeat, it is determined that an unactivated master node exists in the path, and the unactivated master node is searched for restart.
In this embodiment, whether each master node in the path is started is determined by monitoring the heartbeat of each master node. Specifically, the initial node sends a detection data packet to each master node every fixed time or no more than a certain time interval, and data in the detection data packet indicates that each master node needs to reply a feedback data packet after receiving the detection data packet, and indicates that the replied feedback data packet needs to carry a master node identifier.
And further, taking the feedback data packet replied by each main node as a returned heartbeat, detecting whether each main node in the path returns the heartbeat, and if each main node returns the heartbeat, indicating that the communication between each main node and the initial node is normal. However, in order to avoid the existence of counterfeit nodes in each master node, the validity of each master node in the path is verified through the master node identifier carried in the feedback data packet serving as the heartbeat. The method comprises the steps of presetting reference marks for representing uniqueness of each main node in a path, reading the main node marks carried in the heartbeat returned by each main node and comparing the main node marks with the respective reference marks, and judging whether the main node marks are consistent with the respective reference marks.
If the main node identification of each main node is consistent with the respective reference identification, judging that each main node in the path is effective; at this time, each master node in the path is judged to be started, and the first path value can be determined to transmit the file to be distributed. If any one of the main node identifiers of the main nodes is inconsistent with the reference identifier, judging that invalid main nodes exist in the main nodes; at this time, prompt information is output to prompt the invalid master node to be processed in time.
Further, if any one of the master nodes does not return the heartbeat, it is determined that the master node which is not started exists in the path. At this time, the master node which is not started is found out from the master nodes, and the master node is started again until the master node is successfully started within the preset starting times or is not successfully started within the preset starting times, and prompt information for maintaining the master node is output.
Note that the terminal as each master node may be a virtual machine. For example, during the time of big data teaching, a KVM virtual machine is used as a virtual machine of a big data cluster for students, and for efficient big data teaching, a virtual machine for students may be set as a virtual machine group. Setting a plurality of virtual machines for each student, wherein one virtual machine is used as a main virtual machine to form a main node in a path for receiving files transmitted by a teacher; the other slave virtual machines are all used as slave virtual machines subordinate to the main virtual machine, are equivalent to slave nodes in the path, and are used for processing the received files. For this reason, in this embodiment, the step of detecting whether each master node in the path returns a heartbeat includes:
step S70, dividing each distribution node in the path into a master node and a slave node, wherein each distribution node comprises a master node and at least one slave node, and at least one slave node is subordinate to the master node to perform auxiliary processing on the file to be distributed received by the master node;
and step S80, controlling the start of each main node, and receiving the returned heartbeat after the start of each main node.
Furthermore, a plurality of distribution nodes are arranged in the path, each distribution node corresponds to one virtual machine group, and each virtual machine group comprises a plurality of virtual machines. One virtual machine in the virtual machine group is identified as a master virtual machine, and the other virtual machines are identified as slave virtual machines subordinate to the master virtual machine. For each distribution node, the primary virtual machine identified therein is divided into a primary node, and the secondary virtual machine identified therein is divided into a secondary node. The slave virtual machines are subordinate to the master virtual machine, so that the divided slave nodes are subordinate to the master node, the master node is used for receiving the files to be distributed in the path diffusion mode, and the slave nodes are used for performing auxiliary processing on the files to be distributed received by the master node.
Further, each master node divided in the control path is started and sends a detection data packet to each master node. And then, receiving a feedback data packet returned by each main node aiming at the detection data packet as a heartbeat returned by each node after starting. Whether the main nodes are effectively started or not is detected by detecting whether the main nodes return heartbeats or not.
In the embodiment, the distribution nodes of the path are divided into the master node and the slave nodes, and only the master node in the path needs to be started in the stage of receiving the file to be distributed, so that the situation that the file to be distributed is distributed only after all the nodes are started is avoided, and the distribution efficiency of the file to be distributed is improved by saving the time for starting the nodes. Meanwhile, the effective starting of each main node is ensured by judging whether each main node returns the heartbeat or not and verifying the effectiveness of each main node, and the effective distribution of subsequent files to be distributed is facilitated.
The embodiment of the invention also provides a readable storage medium. The readable storage medium has stored thereon a control program which, when executed by the processor, implements the steps of the path-flooding based file distribution method as described above.
The readable storage medium of the present invention may be a computer-readable storage medium, and the specific implementation manner of the readable storage medium of the present invention is basically the same as that of each embodiment of the file distribution method based on path diffusion, and will not be described herein again.
The present invention is described in connection with the accompanying drawings, but the present invention is not limited to the above embodiments, which are only illustrative and not restrictive, and those skilled in the art can make various changes without departing from the spirit and scope of the invention as defined by the appended claims, and all changes that come within the meaning and range of equivalency of the specification and drawings that are obvious from the description and the attached claims are intended to be embraced therein.
Claims (10)
1. A file distribution method based on path diffusion is characterized by comprising the following steps:
after detecting that all main nodes in a path are started, determining first path values between an initial node in the path and all the main nodes respectively, and determining a plurality of path diffusion nodes from all the main nodes according to the first path values;
distributing a file to be distributed to a plurality of path diffusion nodes from the initial node, taking the path diffusion node which receives the file to be distributed as a target diffusion node, and determining a second path value between the target diffusion node and other main nodes which do not start to receive the file to be distributed in each main node;
determining a node to be diffused corresponding to the target diffusion node in each other main node according to the second path value, and distributing the file to be distributed to the node to be diffused by the target diffusion node;
after the to-be-distributed file is received by the to-be-distributed node, setting the to-be-distributed node as a new target diffusion node, and executing a step of determining a second path value between the target diffusion node and other host nodes which do not start to receive the to-be-distributed file in each host node until each host node in the path receives the to-be-distributed file.
2. The path-flooding-based file distribution method according to claim 1, wherein said step of distributing the file to be distributed from the initial node to a plurality of the path-flooding nodes comprises:
determining the sequence of sending time among the path diffusion nodes according to the magnitude relation among the first path values respectively corresponding to the path diffusion nodes;
and distributing the files to be distributed from the initial node to a plurality of path diffusion nodes in sequence according to the sequence of the sending time.
3. The method for distributing files based on path diffusion according to claim 1, wherein the step of determining the node to be diffused corresponding to the target diffusion node in each of the other master nodes according to the second path value comprises:
comparing the second path values to determine the minimum value of the second path values;
and searching other main nodes corresponding to the minimum value in the other main nodes, and taking the searched other main nodes as the nodes to be diffused corresponding to the target diffusion node.
4. The path-flooding-based file distribution method of claim 3, wherein said step of distributing the file to be distributed from the target flooding node to the node to be flooded comprises:
judging whether the node to be diffused is in a file receiving state, if so, searching a second path value with a numerical value in the order above the minimum value according to the ascending order of the second path values;
taking other main nodes corresponding to the searched second path value in the other main nodes as new nodes to be diffused, and executing the step of judging whether the nodes to be diffused are in a file receiving state;
and if the node to be diffused is not in the state of receiving the file, distributing the file to be distributed to the node to be diffused by the target diffusion node.
5. The method for file distribution based on path flooding according to claim 1, wherein said step of determining a plurality of path flooding nodes from each of the master nodes according to each of the first path values comprises:
sequencing the first path values in a descending order to generate a numerical sequence;
screening a plurality of first path values arranged in the front from the numerical sequence, and taking the main nodes respectively corresponding to the screened first path values as a plurality of path diffusion nodes;
wherein the number of the first path values is the same as the concurrent number of the initial nodes.
6. The method for file distribution based on path flooding according to any one of claims 1-5, wherein the step of determining the first path value between the initial node and each of the master nodes in the path comprises:
based on the initial node running preset command, the preset command transmits test data to each main node respectively;
and determining each first path value according to the time for each main node to respectively return feedback data to the initial node based on the test data.
7. The method for file distribution based on path flooding according to any one of claims 1-5, wherein the step of determining the first path value between the initial node and each of the host nodes in the path after detecting that each of the host nodes in the path is started comprises:
detecting whether each main node in the path returns a heartbeat, if so, verifying the effectiveness of each main node in the path according to the heartbeat returned by each main node, and judging that each main node in the path is started after verifying that each main node is effective;
if it is detected that at least one main node in the path does not return to the heartbeat, it is determined that an un-started main node exists in the path, and the un-started main node is searched for restarting.
8. The method for distributing files based on path diffusion according to claim 7, wherein the step of detecting whether each master node in the path returns a heartbeat comprises:
dividing each distribution node in the path into a master node and a slave node, wherein each distribution node comprises a master node and at least one slave node, and at least one slave node is subordinate to the master node so as to perform auxiliary processing on the file to be distributed received by the master node;
and controlling the starting of each main node and receiving the heartbeat returned after the starting of each main node.
9. A path-flooding based file distribution system, characterized in that the path-flooding based file distribution system comprises a memory, a processor and a control program stored on the memory and executable on the processor, which control program, when executed by the processor, implements the steps of the path-flooding based file distribution method according to any one of claims 1-8.
10. A readable storage medium, characterized in that the readable storage medium has stored thereon a control program which, when executed by a processor, implements the steps of the path diffusion-based file distribution method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110820202.0A CN113630446A (en) | 2021-07-20 | 2021-07-20 | File distribution method and system based on path diffusion and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110820202.0A CN113630446A (en) | 2021-07-20 | 2021-07-20 | File distribution method and system based on path diffusion and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113630446A true CN113630446A (en) | 2021-11-09 |
Family
ID=78380290
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110820202.0A Pending CN113630446A (en) | 2021-07-20 | 2021-07-20 | File distribution method and system based on path diffusion and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113630446A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018076765A1 (en) * | 2016-10-31 | 2018-05-03 | 华为技术有限公司 | Content distribution method and device for cloud computing system, computing node and system |
| CN110086847A (en) * | 2019-03-12 | 2019-08-02 | 中国科学院计算技术研究所 | Data distributing method, medium and electronic equipment |
| WO2020063372A1 (en) * | 2018-09-29 | 2020-04-02 | 华为技术有限公司 | Data transmission method and related apparatus |
-
2021
- 2021-07-20 CN CN202110820202.0A patent/CN113630446A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018076765A1 (en) * | 2016-10-31 | 2018-05-03 | 华为技术有限公司 | Content distribution method and device for cloud computing system, computing node and system |
| CN108023812A (en) * | 2016-10-31 | 2018-05-11 | 华为技术有限公司 | The content distribution method and device of cloud computing system, calculate node and system |
| WO2020063372A1 (en) * | 2018-09-29 | 2020-04-02 | 华为技术有限公司 | Data transmission method and related apparatus |
| CN110086847A (en) * | 2019-03-12 | 2019-08-02 | 中国科学院计算技术研究所 | Data distributing method, medium and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7668831B2 (en) | Assigning unique identification numbers to new user accounts and groups in a computing environment with multiple registries | |
| CN108696589B (en) | Block chain data transmission method, device, equipment and storage medium | |
| CN109981279B (en) | Block chain system, communication method, device, equipment and medium | |
| CN110995513A (en) | Data sending and receiving method in Internet of things system, Internet of things equipment and platform | |
| CN114049123A (en) | Block chain consensus method and device, computer equipment and storage medium | |
| CN111242784A (en) | Block pre-packing method, block node, device and storage medium | |
| US11303532B2 (en) | Method and system for detecting service quality of CDN system | |
| CN110445678B (en) | Multicast network detection method, device and forwarding equipment | |
| JP5961471B2 (en) | Output comparison method in multiple information systems | |
| CN111092956A (en) | Resource synchronization method, device, storage medium and equipment | |
| CN113630446A (en) | File distribution method and system based on path diffusion and readable storage medium | |
| CN104717280A (en) | Data synchronism marking method and device | |
| WO2023040453A1 (en) | Transaction information processing method and apparatus | |
| CN111385613B (en) | Television system repairing method, storage medium and application server | |
| CN108494589B (en) | Management method and system of distributed Nginx server | |
| CN112306962B (en) | File copying method, device and storage medium in computer cluster system | |
| CN112181485B (en) | Script execution method and device, electronic equipment and storage medium | |
| CN115269269A (en) | Drill method, device, backup server, medium, server cluster and system | |
| CN114676140A (en) | Scene data storage method, device, equipment and medium | |
| CN108306937A (en) | Sending method, acquisition methods, server and the storage medium of short message verification code | |
| CN110765016B (en) | Game test instruction processing method, game test instruction processing system, server and storage device | |
| CN112566221A (en) | Information synchronization method, device, equipment and readable storage medium | |
| CN104657507A (en) | Distributed system-based mode detection method and device of image data | |
| JP2003228497A (en) | Fault notification system and fault notification program | |
| CN113055498B (en) | Data source access method, device, storage medium and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |