CN109660639B

CN109660639B - Data uploading method, device, system and medium

Info

Publication number: CN109660639B
Application number: CN201811512205.2A
Authority: CN
Inventors: 鲁塔
Original assignee: Shenzhen Onething Technology Co Ltd; Shenzhen Xunlei Network Technology Co Ltd
Current assignee: Shenzhen Onething Technology Co Ltd; Shenzhen Xunlei Network Technology Co Ltd
Priority date: 2018-12-11
Filing date: 2018-12-11
Publication date: 2022-02-25
Anticipated expiration: 2038-12-11
Also published as: CN109660639A

Abstract

The invention discloses a data uploading method, a device, a system and a medium, wherein the method comprises the following steps: initiating an information acquisition instruction to a scheduling server, and receiving node attribute information of a target storage node transmitted by the scheduling server; the target storage node is a node in the cloud storage service and belongs to the same ISP with the user node; establishing corresponding communication connection with each target storage node according to the node attribute information; and carrying out fragment encoding on the source file through a preset encoding function to obtain an encoded file, and uploading the encoded file to a corresponding target storage node through each communication connection. The method can relatively reduce the time overhead required for connecting the storage nodes when the user nodes access the cloud storage service, and can relatively improve the overall uploading efficiency of the file data. In addition, the invention also provides data uploading equipment, a system and a medium, and the beneficial effects are as above.

Description

Data uploading method, device, system and medium

Technical Field

The invention relates to the field of cloud storage, in particular to a data uploading method, device, system and medium.

Background

With the advent of the big data era, data itself has higher and higher value, so the requirement of a user on data security is continuously increased, and due to the fact that the number of equipment nodes of the user is single, data storage security is difficult to ensure only through the equipment nodes of the user, and under the background, the cloud storage technology is brought forward.

Cloud storage is a technology that takes a large number of server devices in a network as storage nodes and controls the server devices to cooperatively work to provide data storage and service access functions to the outside through cluster application, network technology or a distributed file system. The user uploads the file data to the cloud storage server by taking own equipment as a user node, so that the purpose of storing the file data in a distributed manner through a plurality of storage nodes in the Internet is achieved, and the safety of file data storage is further ensured. In the current situation, when file data is uploaded in a cloud storage manner, a target storage node to which the file data is to be uploaded is often selected in a random manner, but because the number of storage nodes forming a cloud storage service is large and the network conditions among the storage nodes are relatively complex, the network state between the randomly selected target storage node and a user node is difficult to guarantee, and further, the connection between the target storage node and the user node is likely to be long in time consumption, and the overall transmission efficiency of the file data is low.

Therefore, it is obvious that a data uploading method is provided to relatively reduce the time overhead required for connecting a storage node when a user node accesses a cloud storage service, and relatively improve the overall uploading efficiency of file data, and a problem to be solved by technical personnel in the field is urgently needed.

Disclosure of Invention

The invention aims to provide a data uploading method, which aims to relatively reduce the time overhead required by connecting a storage node when a user node accesses a cloud storage service and relatively improve the overall uploading efficiency of file data.

In order to solve the above technical problem, the present invention provides a data uploading method, which is applied to a user node, and the method includes:

initiating an information acquisition instruction to a scheduling server, and receiving node attribute information of a target storage node transmitted by the scheduling server; the target storage node is a node in the cloud storage service and belongs to the same ISP with the user node;

establishing corresponding communication connection with each target storage node according to the node attribute information;

and carrying out fragment encoding on the source file through a preset encoding function to obtain an encoded file, and uploading the encoded file to a corresponding target storage node through each communication connection.

Preferably, the ISP for which the target storage node and the user node belong to the same includes:

the target storage node and the user node belong to the same ISP and belong to the same region as the IP address of the user node.

Preferably, the content of the node attribute information includes:

the network type, the external network mapping state, the NAT type and the P2P historical penetration rate of the target storage node;

correspondingly, establishing corresponding communication connection with each target storage node according to the node attribute information comprises:

judging the content of the node attribute information, and establishing communication connection with a target storage node in a TCP direct connection mode when the network type of the target storage node is an external network type or the external network mapping state is a mapping success state;

when TCP direct connection fails, or the network type of the target storage node is an intranet type, and the mapping state of an extranet is a mapping failure state, judging whether the NAT type of the target storage node and the historical penetration rate of P2P meet preset conditions;

if yes, establishing communication connection with the target storage node in a P2P penetration mode;

otherwise, establishing communication connection with the target storage node in a TCP forwarding mode.

Preferably, the preset conditions include:

the NAT type of the user node and the NAT type of the target storage node are not both symmetric NAT types, and the historical penetration rate of P2P is greater than a preset value.

Preferably, before the source file is sliced and encoded by the preset encoding function to obtain the encoded file, the method further includes:

receiving standby node attribute information of the standby storage nodes transmitted by the scheduling server, and establishing corresponding standby communication connection with each standby storage node according to the standby node attribute information; the standby storage nodes are obtained in the cloud storage service in a random selection mode and are not target storage nodes.

Correspondingly, after the source file is sliced and encoded through the preset encoding function to obtain the encoded file, the method further comprises:

and uploading the encoded file to the corresponding standby storage node through the standby communication connection.

Preferably, after uploading the encoded file to the corresponding target storage node through each communication connection, the method further includes:

when the number of successfully uploaded coding files reaches a specified threshold value, waiting for a preset time length;

and if the number of the coding files is constant within the preset time length, disconnecting the communication connection with the target storage node which is not successfully uploaded.

Preferably, the value of the preset duration includes:

and the average estimated time length of the target storage node which is not successfully uploaded to reach the completion state.

Preferably, the step of encoding the source file in a slicing manner by using a preset encoding function to obtain the encoded file includes:

and carrying out fragment encoding on the source file through a fountain encoding function to obtain an encoded file.

In addition, the invention also provides user node equipment, which comprises a memory, a processor and a bus, wherein the memory is stored with a data uploading program which can run on the processor, and the data uploading program is acquired by the processor through the bus and is executed to realize the data uploading method applied to the user node.

In addition, the invention also provides a data uploading system, which comprises:

the user node is used for initiating an information acquisition instruction to the scheduling server and receiving the node attribute information of the target storage node transmitted by the scheduling server; the target storage node is a node in the cloud storage service and belongs to the same ISP with the user node; establishing corresponding communication connection with each target storage node according to the node attribute information; the method comprises the steps that a source file is subjected to fragment coding through a preset coding function to obtain a coded file, and the coded file is uploaded to a corresponding target storage node through each communication connection;

the scheduling server is used for receiving an information acquisition instruction transmitted by the user node and selecting a target storage node which belongs to the same ISP as the user node from a preset node relation table; and acquiring node attribute information of the target storage node, and returning the node attribute information to the user node.

In addition, the present invention also provides a computer-readable storage medium, on which a data uploading program is stored, where the data uploading program can be executed by one or more processors to implement the data uploading method as described above.

The data uploading method provided by the invention comprises the steps of firstly initiating an information acquisition instruction to a scheduling server through a user node, then acquiring and returning node attribute information of target storage nodes which belong to the same ISP (Internet service provider) as the user node by the scheduling server, further establishing corresponding communication connection between the user node and each target storage node according to the content of the node attribute information, further coding source files into coded files in a segmented mode, and uploading the coded files to the corresponding target storage nodes through each communication connection, so that the purposes of distributing and uploading the source files to a cloud storage service are achieved. The method comprises the steps of sending node attribute information of a target storage node which belongs to the same Internet service provider as a user node to the user node through a scheduling server, further establishing communication connection between the user node and the target storage node, and uploading data. In addition, the invention also provides data uploading equipment, a system and a medium, and the beneficial effects are as above.

Drawings

In order to illustrate the embodiments of the present invention more clearly, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings can be obtained by those skilled in the art without inventive effort.

Fig. 1 is a flowchart of a data uploading method applied to a user node according to an embodiment of the present invention;

fig. 2 is a flowchart of another data uploading method applied to a user node according to an embodiment of the present invention;

fig. 3 is a flowchart of a data uploading method applied to a scheduling server according to an embodiment of the present invention;

fig. 4 is a structural diagram of a user node device according to an embodiment of the present invention;

fig. 5 is a schematic diagram of a data uploading system according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the drawings described above, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

It should be noted that the description relating to "first", "second", etc. in the present invention is for descriptive purposes only and is not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In addition, technical solutions between various embodiments may be combined with each other, but must be realized by a person skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present invention.

Example one

The invention provides a data uploading method applied to a user node.

Referring to fig. 1, fig. 1 is a flowchart of a data uploading method applied to a user node according to an embodiment of the present invention.

In one embodiment, the method comprises:

step S10: and initiating an information acquisition instruction to the scheduling server, and receiving the node attribute information of the target storage node transmitted by the scheduling server.

The target storage node is a node in the cloud storage service and belongs to the same ISP with the user node.

It should be noted that the premise that this step can be smoothly performed is that a communication connection has been established between the scheduling server and the user node, and normal communication is possible. An execution main body of the step is a user node, namely, an initiator of data uploading, which sends an information acquisition instruction to a scheduling server to inform the scheduling server of providing a storage node for user data storage, and then after receiving the information acquisition instruction, the scheduling server provides node attribute information corresponding to a target storage node belonging to the same ISP (internet service provider) to the user node. On the basis, all target storage nodes can be uniformly controlled and managed by the cloud storage service, and therefore after the data of the user are uploaded to the target storage nodes, the data in the target storage nodes can be normally acquired through the cloud storage service in the subsequent operation. In this step, the target storage node and the user node are both the same internet service provider, and it is considered that the communication mechanism between the nodes of the same internet service provider is relatively consistent, and adaptive compatibility conversion according to the difference of the internet service provider is not needed when the nodes are connected or communicate, thereby reducing extra time overhead.

Step S11: and establishing corresponding communication connection with each target storage node according to the node attribute information.

In the step, appropriate communication connection is established between the user node and each target storage node according to the specific content of the node attribute information transmitted by the scheduling server, so that the reliability of each communication connection is ensured as much as possible according to the node attribute information.

Step S12: and carrying out fragment encoding on the source file through a preset encoding function to obtain an encoded file, and uploading the encoded file to a corresponding target storage node through each communication connection.

The preset encoding function in this step is used for performing fragment encoding processing on a source file that needs to be uploaded by a user, where the fragment encoding refers to dividing the source file into data segments, and performing encoding processing on each data segment through a preset encoding mode, and an encoded file obtained after the fragment encoding often has certain redundancy, so that a situation that the normal recovery of the source file is affected due to possible loss or damage of the encoded file is avoided, and therefore the number of the encoded files is often greater than the number of the data segments. The preset coding function can realize the function of coding the source file in a corresponding coding mode, and particularly can process the source file in an RS coding mode, wherein the coded data in the RS code generally comprises data information and correction information, the information in the RS code is not processed by bit, but is processed by sign bit in one code formed by digital bit, and compared with the traditional array code, more data can be recovered under the condition of smaller redundancy; the source file can also be processed by adopting an LDPC coding mode, which mainly adopts a sparse matrix and only uses exclusive OR operation, thereby achieving the purpose of replacing low coding and decoding time with small decoding failure rate. The preset encoding function used in the step of performing the fragment encoding may be determined according to the actual encoding requirement of the user on the source file, and the above-mentioned only some encoding modes that can be selected by the user according to the requirement are not limited to this.

After the coding file is obtained through coding, the coding file is stored in the target storage nodes in a distributed mode, and in order to balance storage pressure of each target storage node and improve overall data uploading efficiency, the coding file can be evenly distributed to each target storage node. When the total number of the target storage nodes is greater than or equal to the number of the encoded files, only one encoded file may be stored in the target storage nodes, so as to average the storage overhead of each target storage node to the maximum extent.

The data uploading method provided by the invention comprises the steps of firstly initiating an information acquisition instruction to a scheduling server through a user node, then acquiring and returning node attribute information of target storage nodes which belong to the same ISP (Internet service provider) as the user node by the scheduling server, further establishing corresponding communication connection between the user node and each target storage node according to the content of the node attribute information, further coding source files into coded files in a segmented mode, and uploading the coded files to the corresponding target storage nodes through each communication connection, so that the purposes of distributing and uploading the source files to a cloud storage service are achieved. The method comprises the steps of sending node attribute information of a target storage node which belongs to the same Internet service provider as a user node to the user node through a scheduling server, further establishing communication connection between the user node and the target storage node, and uploading data.

Example two

On the basis of the above examples, the present invention also provides a series of preferred embodiments as follows.

As a preferred embodiment, the ISP where the target storage node and the user node belong to the same includes:

It should be noted that, since it is considered that data signals transmitted by communication between nodes are often attenuated along with the distance between nodes, which may cause data loss and retransmission during transmission, the transmission efficiency is difficult to be guaranteed. The 'region' in the embodiment is a geographical region pre-divided by an internet service provider, and since the density of network base stations is higher in the same geographical region, the quality of data signals transmitted and communicated between nodes in the same region is higher.

Referring to fig. 2, fig. 2 is a flowchart of another data uploading method applied to a user node according to an embodiment of the present invention.

Step S10 and step S12 in fig. 2 are the same as those in fig. 1, and are not repeated herein.

As shown in fig. 2, as a preferred embodiment, the content of the node attribute information includes:

the network type, the extranet mapping status, the NAT type and the P2P historical penetration rate of the target storage node.

It should be noted that the network types in the node attribute information mainly include an intranet type and an extranet type, and the difference between the intranet type node and the extranet type node is that the node belonging to the intranet type cannot initiate access to the extranet node through its own IP address, and the node belonging to the extranet type can directly access the extranet node through its own IP address; the mapping state of the external network refers to that the node can map the IP address of the internal network into the IP address of the external network, that is, the node can directly access the external network node in the external network mapping mode, so the content of the mapping state of the external network is often "success" or "failure"; NAT types are mainly divided into a complete cone type NAT, an address limiting cone type NAT, a port limiting cone type NAT and a symmetrical type NAT; the P2P historical penetration rate refers to a success rate of establishing connection between a target storage node and other nodes in a P2P penetration manner, and the essence of P2P penetration is to establish a direct transmission relationship between nodes for a data packet, so that time overhead caused by forwarding the data packet through a router can be avoided.

Accordingly, step S11 includes:

step S20: and judging the content of the node attribute information, and establishing communication connection with the target storage node in a TCP direct connection mode when the network type of the target storage node is an external network type or the external network mapping state is a mapping success state.

When the network type of the target storage node is an extranet type or the extranet mapping state is a mapping success state, it is indicated that the target storage node is in an extranet or the target storage node is a node of an intranet but can directly communicate with a node of the extranet, in this case, a communication relationship between the user node and the target storage node is directly established in a TCP direct connection manner, and the operations of address conversion and router addressing forwarding can be performed across the NAT server, so that the efficiency of communication connection is relatively improved.

Step S21: when the TCP direct connection fails, or the network type of the target storage node is an intranet type, and the mapping state of the extranet is a mapping failure state, judging whether the NAT type of the target storage node and the historical penetration rate of P2P meet preset conditions, if so, executing a step S22; otherwise, step S23 is executed.

Step S22: and establishing communication connection with the target storage node by means of P2P penetration.

Step S23: and establishing communication connection with the target storage node in a TCP forwarding mode.

It should be noted that, when TCP direct connection fails, or the network type of the target storage node is an intranet type, and the extranet mapping state is a mapping failure state, it indicates that TCP direct connection between the user node and the target storage node cannot be directly established, in this case, in order to improve the communication efficiency of the established communication connection as much as possible on the basis of ensuring that the user node and the target storage node can establish the communication connection, further, the NAT type and the P2P historical penetration rate are determined according to the preset conditions, and the establishment mode of the subsequent communication connection is determined according to whether the two satisfy the preset conditions.

If the preset condition can be met, P2P penetration is performed, and the P2P penetration is also the 'TCP hole' adopted in the field. For example, to implement direct communication between the user node a and the target storage node B, the following steps are required: firstly, a user node A is connected with a server S on the Internet and sends a message, so that the S can acquire the actual terminal (the external network IP and the port number for sending the message) of the user node A on the Internet; then, the target storage node B also carries out the same steps, and S knows the terminals (namely 'holes') of the node A and the node B on the Internet; then, the S tells the A node and the B node the actual IP address and the port of the opposite side on the internet respectively, and further, after the A node and the B node both learn the actual IP address and the port of the opposite side on the internet, the TCP communication between the A node and the B node can be directly carried out, and further, the data transmission and receiving are only carried out between the A node and the B node without the forwarding operation of a router when the data is transmitted, so that the data transmission efficiency is improved. In addition, the TCP holing is selected instead of the UDP holing, so that the communication based on the TCP protocol is considered, the integrity and the reliability of data transmission are relatively high, and the method is suitable for the requirement of a user when the data is uploaded.

If the NAT type and the historical penetration rate of P2P cannot meet preset conditions, a TCP forwarding mode is adopted, and data packets are addressed and forwarded through a router to establish a communication transmission relation between the user node and the target storage node, so that the communication feasibility between the user node and the target storage node is ensured to the maximum extent.

According to the method and the device, the adaptive communication connection is established between the user node and the target storage node according to the content of the node attribute information, so that the efficiency of the communication connection is improved to the greatest extent.

On the basis of the above embodiment, as a preferred embodiment, the preset conditions include:

Considering that when the NAT type of the user node and the NAT type of the target storage node are not both symmetric NAT types, the user node may establish connection with each target storage node through a plurality of port numbers, and then the user node often has a port conflict, which makes it difficult to ensure the reliability of data communication after P2P penetration. In addition, whether P2P penetration on a target storage node can have high feasibility can be known through the P2P historical penetration rate of the target storage node. Therefore, the two items are used as the basis for judging whether to perform P2P penetration or not, so that the accuracy of the connection mode between the selected target storage node and the user node can be relatively improved.

In addition, as a preferred embodiment, before the source file is sliced and encoded by the preset encoding function to obtain the encoded file, the method further includes:

It should be noted that, in this embodiment, before the user node performs fragment coding on the source file through the preset coding function to obtain the coded file, the user node acquires the standby node attribute information of the standby storage node from the scheduling server again, and then establishes a communication connection with the standby storage node according to the standby node attribute information, where the content of the standby node attribute information may refer to the node attribute information of the target storage node, and the establishment of the standby communication connection may also refer to the establishment mode of the communication connection between the user node and the target storage node. The standby nodes and the target storage nodes are all nodes in the same cloud storage server, the standby nodes and the target storage nodes are required to be ensured not to be overlapped, and the standby nodes are selected in a random mode, so that the standby nodes are not limited to the regions of ISPs (internet service providers) and IP (internet protocol) addresses of the standby nodes, and further, after the source files are subjected to fragment coding through a preset coding function to obtain coded files, the coded files are uploaded to the corresponding standby storage nodes through standby communication connection. In the embodiment, the random selection of the backup storage node is considered that the backup storage node and the user node may not belong to the same ISP and region, and the backup storage node stores the encoded file, so that the disaster resistance of the uploaded data can be relatively improved, and the security of the data is ensured.

In addition, as a preferred embodiment, after uploading the encoded file to the corresponding target storage node through each communication connection, the method further includes:

It can be understood that when the number of successfully uploaded encoded files reaches a specified threshold, the encoded files uploaded to the cloud storage service are considered to be completely decoded into source files of users, on this basis, in order to further improve the disaster tolerance of the encoded files, the present embodiment continues waiting for a preset time duration, and if the number of encoded files is constant within the preset time duration, the communication connection with the target storage node that is not successfully uploaded is disconnected, so as to avoid the long occupation of network resources. The preset duration in this embodiment may be determined according to the overall performance of the network, and is not specifically limited herein.

On the basis of the foregoing embodiment, as a preferred embodiment, the value of the preset duration includes:

It should be noted that, in this embodiment, the target storage node that has not been successfully uploaded refers to a target file node that has not completed receiving the encoded file, and the remaining time length is a quotient calculated from the current data amount that has not been uploaded and the average uploading speed of each target storage node, and then the preset time length is an average value of the remaining time lengths of each target storage node, so that the preset time length can be set relatively accurately according to the data uploading efficiency of the entire target storage node.

Further, on the basis of the above-mentioned series of embodiments, as a preferred embodiment, the encoding the source file in a sliced manner by using a preset encoding function to obtain the encoded file includes:

It should be noted that the fountain code has a relatively simple coding logic, so that it can generate a lower decoding overhead and coding complexity, thereby improving the coding efficiency.

EXAMPLE III

The invention also provides a data uploading method applied to the scheduling server.

Referring to fig. 3, fig. 3 is a flowchart of a data uploading method applied to a scheduling server according to an embodiment of the present invention.

In a third embodiment, the method comprises:

step S30: and receiving an information acquisition instruction transmitted by the user node, and selecting a target storage node which belongs to the same ISP as the user node from a preset node relation table.

Note that a node relationship table is preset in the scheduling server, and each storage node and its corresponding ISP are recorded in the node relationship table. In the step, after the scheduling server receives the information acquisition instruction, a target storage node with the same ISP as the user node is selected according to the ISP of the user node.

Step S31: and acquiring node attribute information of the target storage node, and returning the node attribute information to the user node.

After the target storage node is selected, node attribute information corresponding to the target storage node is obtained, and the node attribute information is returned to the user node, so that communication connection with the target storage node is established through analysis of the node attribute information by the user node, and data uploading operation is completed.

In addition, as a preferred embodiment, selecting a target storage node belonging to the same ISP as the user node in the preset node relationship table includes:

and selecting a target storage node which belongs to the same ISP with the user node and has the IP address in the same region from the node relation table.

It should be noted that, since it is considered that data signals transmitted by communication between nodes are often attenuated along with the distance between nodes, which may cause data loss and retransmission during transmission, the transmission efficiency is difficult to be guaranteed. The "region" in the present embodiment is a geographical region divided in advance by the internet service provider, and since the density of network base stations is high in the same geographical region, the quality of data signals transmitted and communicated between nodes in the same geographical region is high, and therefore the node relationship table in the present embodiment records the region to which the IP address of each storage node belongs, in addition to each storage node and its corresponding ISP. The target storage node and the user node belong to the same node in one region, so that the distance between the user node and the target storage node can be relatively reduced, and the data transmission efficiency between the user node and the target storage node is improved.

Example four

In the foregoing, the embodiment of the data uploading method is described in detail, and the present invention further provides a user node device corresponding to the method.

Fig. 4 is a structural diagram of a user node device according to an embodiment of the present invention.

In this embodiment, the user node device 1 may be a PC (Personal Computer), or may be a terminal device of a smart phone, a tablet Computer, a palm Computer, a portable Computer, an intelligent router, an ore machine, or a network storage device.

The device 1 may be a node constituting a CDN network or a blockchain network.

The user node device 1 may comprise a memory 11, a processor 12 and a bus 13.

The memory 11 includes at least one type of readable storage medium, which includes a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, and the like. The memory 11 may in some embodiments be an internal storage unit of the user node device 1, e.g. a hard disk of the user node device 1. The memory 11 may also be an external storage device of the user node device 1 in other embodiments, such as a plug-in hard disk provided on the user node device 1, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and so on. Further, the memory 11 may also include both an internal storage unit of the user node apparatus 1 and an external storage device. The memory 11 may be used not only to store application software installed in the user node device 1 and various types of data, such as codes of a data upload program, etc., but also to temporarily store data that has been output or is to be output.

The processor 12 may be a Central Processing Unit (CPU), a controller, a microcontroller, a microprocessor or other data Processing chip in some embodiments, and is used for executing program codes stored in the memory 11 or Processing data, such as executing a data uploading program.

The bus 13 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 4, but this does not indicate only one bus or one type of bus.

Further, the user node device 1 may further comprise a network interface, which may optionally comprise a wired interface and/or a wireless interface (such as a WI-FI interface, a bluetooth interface, etc.), typically for establishing a communication connection between the device 1 and other electronic devices.

Optionally, the device 1 may further comprise a user interface, which may comprise a Display (Display), an input unit such as a Keyboard (Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the user node device 1 and for displaying a visualized user interface.

In addition, the present invention further provides a scheduling server device corresponding to the method, and since the embodiment of the device portion corresponds to the embodiment of the method portion, for the embodiment of the device portion, reference is made to the description of the embodiment of the method portion, which is not repeated here.

The scheduling server device comprises a memory, a processor and a bus, wherein the memory is stored with a data uploading program which can run on the processor, and the data uploading program is acquired by the processor through the bus and is executed to realize the data uploading method applied to the scheduling server.

EXAMPLE five

The invention also provides a data uploading system.

Referring to fig. 5, fig. 5 is a schematic diagram of a data uploading system according to an embodiment of the present invention.

The data uploading system comprises:

the user node 20 is configured to initiate an information acquisition instruction to the scheduling server 21, and receive node attribute information of the target storage node 23 transmitted by the scheduling server 21; the target storage node 23 is a node in the cloud storage service, and belongs to the same ISP as the user node 20; establishing corresponding communication connection with each target storage node 23 according to the node attribute information; the source file is subjected to fragment coding through a preset coding function to obtain a coded file, and the coded file is uploaded to a corresponding target storage node 23 through each communication connection;

the scheduling server 21 is configured to receive an information acquisition instruction transmitted by the user node 20, and select a target storage node 23 belonging to the same ISP as the user node 20 from a preset node relation table; node attribute information of the target storage node 23 is acquired, and the node attribute information is returned to the user node 20.

The data uploading system provided by the invention firstly initiates an information acquisition instruction to a scheduling server through a user node, then the scheduling server acquires and returns node attribute information of target storage nodes which belong to the same ISP (Internet service provider) as the user node, and then the user node establishes corresponding communication connection with each target storage node according to the content of the node attribute information, and further the source file is fragmented and encoded into an encoded file, and the encoded file is uploaded to the corresponding target storage node through each communication connection, so that the purposes of distributing and uploading the source file and storing the source file to the cloud storage service are realized. The system sends node attribute information of a target storage node which belongs to the same Internet service provider as the user node to the user node through the scheduling server, further establishes communication connection between the user node and the target storage node, and uploads data.

EXAMPLE six

The present invention also provides a computer-readable storage medium having a data uploading program stored thereon, where the data uploading program can be executed by one or more processors to implement the data uploading method applied to the user node and/or the data uploading method applied to the scheduling server as described above.

The computer-readable storage medium provided by the invention firstly initiates an information acquisition instruction to a scheduling server through a user node, then the scheduling server acquires and returns node attribute information of target storage nodes which belong to the same ISP (Internet service provider) as the user node, and then the user node establishes corresponding communication connection with each target storage node according to the content of the node attribute information, so that the source file is fragmented and encoded into an encoded file, and the encoded file is uploaded to the corresponding target storage node through each communication connection, thereby realizing the purpose of distributing and uploading the source file to the cloud storage service. The computer readable storage medium sends node attribute information of a target storage node belonging to the same Internet service provider as a user node to the user node through a scheduling server, so that communication connection between the user node and the target storage node is established, and data uploading is performed.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product.

The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can store or a data storage device, such as a server, a data center, etc., that is integrated with one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

It should be noted that the above-mentioned numbers of the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments. And the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, apparatus, article, or method that includes the element.

The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

Claims

1. A data uploading method is applied to a user node, and is characterized by comprising the following steps:

initiating an information acquisition instruction to a scheduling server, and receiving node attribute information of a target storage node transmitted by the scheduling server; the target storage node is a node in cloud storage service and belongs to the same ISP with the user node;

the source file is subjected to fragment coding through a preset coding function to obtain a coded file, and the coded file is uploaded to the corresponding target storage node through each communication connection;

wherein the content of the node attribute information includes: the network type and the external network mapping state of the target storage node;

correspondingly, the establishing of the corresponding communication connection with each target storage node according to the node attribute information includes:

and judging the content of the node attribute information, and establishing the communication connection with the target storage node in a TCP direct connection mode when the network type of the target storage node is an external network type or the external network mapping state of the target storage node is a mapping success state.

2. The method of claim 1, wherein the target storage node belonging to the same ISP as the user node comprises:

the target storage node and the user node belong to the same ISP, and belong to the same region as the IP address of the user node.

3. The method of claim 1, wherein the contents of the node attribute information further comprises: the NAT type and the P2P historical penetration rate of the target storage node;

when the TCP direct connection fails, or the network type of the target storage node is an intranet type, and the mapping state of the extranet is a mapping failure state, judging whether the NAT type of the target storage node and the historical penetration rate of P2P meet preset conditions;

if so, establishing the communication connection with the target storage node in a P2P penetration manner;

otherwise, the communication connection is established with the target storage node in a TCP forwarding mode.

4. The method of claim 3, wherein the preset conditions include:

5. The method of claim 1, wherein prior to said tile encoding the source file by the preset encoding function to obtain the encoded file, the method further comprises:

receiving standby node attribute information of standby storage nodes transmitted by the dispatching server, and establishing corresponding standby communication connection with each standby storage node according to the standby node attribute information; the standby storage nodes are obtained in the cloud storage service in a randomly selected mode and are not the target storage nodes;

correspondingly, after the source file is sliced and encoded through the preset encoding function to obtain the encoded file, the method further includes:

and uploading the coding file to the corresponding standby storage node through the standby communication connection.

6. The method of claim 1, wherein after said uploading said encoded file to a corresponding said target storage node over each said communication connection, said method further comprises:

when the number of the successfully uploaded coding files reaches a specified threshold value, waiting for a preset time length;

7. The method of claim 6, wherein the preset duration comprises:

8. The method according to any one of claims 1 to 7, wherein the encoding the source file by slicing through a preset encoding function to obtain the encoded file comprises:

and carrying out the fragment encoding on the source file through a fountain encoding function to obtain the encoded file.

9. A user node device, characterized in that the device comprises a memory, a processor and a bus, the memory having stored thereon a data upload program executable on the processor, the data upload program being retrieved by the processor via the bus and implementing the method according to any of claims 1-8 when executed.

10. A data upload system, characterized in that the system comprises:

the user node is used for initiating an information acquisition instruction to the scheduling server and receiving the node attribute information of the target storage node transmitted by the scheduling server; the target storage node is a node in cloud storage service and belongs to the same ISP with the user node; establishing corresponding communication connection with each target storage node according to the node attribute information; the source file is subjected to fragment coding through a preset coding function to obtain a coded file, and the coded file is uploaded to the corresponding target storage node through each communication connection;

the scheduling server is used for receiving an information acquisition instruction transmitted by the user node and selecting a target storage node which belongs to the same ISP as the user node from a preset node relation table; acquiring node attribute information of the target storage node, and returning the node attribute information to the user node;

correspondingly, the user node is specifically configured to:

11. A computer-readable storage medium having stored thereon a data upload program executable by one or more processors to implement the data upload method of any one of claims 1 to 8.