CN116743785A

CN116743785A - Cloud network data storage method, device, equipment and medium based on fog calculation

Info

Publication number: CN116743785A
Application number: CN202310728254.4A
Authority: CN
Inventors: 陈海锋; 李朝霞; 赵鑫
Original assignee: China United Network Communications Group Co Ltd; Unicom Digital Technology Co Ltd; Unicom Cloud Data Co Ltd
Current assignee: China United Network Communications Group Co Ltd; Unicom Digital Technology Co Ltd; Unicom Cloud Data Co Ltd
Priority date: 2023-06-19
Filing date: 2023-06-19
Publication date: 2023-09-12

Abstract

The application provides a cloud network data storage method, a cloud network data storage device, cloud network data storage equipment and a cloud network data storage medium based on fog calculation, and relates to the technical field of cloud servers. Establishing a plurality of fog computing nodes according to the density degree of the user terminal; generating a resource locator in response to a registration request of the user terminal, and transmitting the resource locator to the user terminal; responding to a file uploading request packet sent by a user terminal, determining a target cloud storage pool for the user terminal according to an Internet interconnection protocol address cloud resource pool association table, and sending a target cloud storage pool address identifier to the user terminal; responding to the data uploading request, and determining a target fog computing node list for uploading data for the user terminal according to a preset fog computing node screening algorithm; and sending the target fog computing node list to the user terminal.

Description

Cloud network data storage method, device, equipment and medium based on fog calculation

Technical Field

The present application relates to the field of cloud servers, and in particular, to a cloud network data storage method, device, equipment, and medium based on fog computing.

Background

The cloud network service architecture is mainly two fundamental basic stones, namely an original data center or service center of an operator, so that the cloud network service architecture becomes a new service and a new data center; and secondly, the network of the next generation is constructed. Both operate on the internet protocol (Internet Protocol, IP) network and are now well integrated together, thus forming a powerful and flexible unified service implementation (USD) system. The cloud computing service and the network service which are convenient and quick are provided for various clients and are collectively called as a 'cloud network integrated' service.

According to the current cloud storage mode, data are stored in different resource pools respectively according to different areas, for example, data in an area A are stored in a resource pool corresponding to the area A, data in an area B are stored in a resource pool corresponding to the area B, a user can only use the storage service of the purchased resource pool, if the user purchases the service of the area A, the user roams to the area B, firstly accesses the Internet from the area B, then routes to the area A through a link of the Internet, and then routes to the storage service of the cloud storage resource pool of the area A through an Internet link of the area A.

However, the cloud network data mode in the prior art has longer data transmission delay and poor user experience.

Disclosure of Invention

The application provides a cloud network data storage method, device, equipment and medium based on fog calculation, which are used for solving the technical problems that a cloud network data mode in the prior art has longer data transmission time delay and poor user experience.

In a first aspect, the present application provides a cloud network data storage method based on fog calculation, including:

generating an internet protocol address cloud resource pool association table according to the cloud resource pool identifier and the user terminal internet protocol address, wherein connection is realized between any two cloud resource pools through a high-speed link, and the bandwidth of the high-speed link is larger than a preset bandwidth threshold;

establishing a plurality of fog computing nodes according to the density degree of the user terminal;

generating a resource locator in response to a registration request of a user terminal, and sending the resource locator to the user terminal;

responding to a file uploading request packet sent by a user terminal, determining a target cloud storage pool for the user terminal according to the Internet interconnection protocol address cloud resource pool association table, and sending the target cloud storage pool address identification to the user terminal so that the user terminal initiates an uploading data request after receiving the target cloud storage pool address identification, wherein the file uploading request packet comprises a resource locator of the user terminal;

Responding to the uploading data request, and determining a target fog computing node list for uploading data for the user terminal according to a preset fog computing node screening algorithm, wherein the target fog computing node list comprises at least one target fog computing node;

and sending the target fog computing node list to the user terminal so as to enable the user terminal to establish connection with the target fog computing node, and uploading data to the target cloud resource pool through the target fog computing node.

The cloud computing node can provide data uploading service for users, the data can be processed and analyzed at a place closer to the source at the network edge, the data to be stored by the users are rapidly uploaded to the cloud resource pool closest to the users in the cloud resource pool association table of the internet protocol address, and more access points are established for the user terminals, so that the data transmission efficiency between the users and the cloud is improved, the business data throughput rate is improved greatly, the transmission delay is reduced, the jitter is reduced, the access of the users from the closest end to a storage system is facilitated, the data transmission time delay is reduced, the user experience is improved, and the convenience of cloud storage is improved.

Optionally, the registration request includes a user identification; accordingly, the generating a resource locator in response to the registration request of the user terminal includes: and responding to the registration request of the user terminal, and carrying out hash calculation on the user identifier and the registration date to obtain the resource locator.

The application obtains the resource locator for each unique link generated by each user through hash calculation, prevents the middleman from obtaining the user information, and improves the safety of the user data and the safety of cloud network data storage.

Optionally, the file upload request packet further includes an internet protocol address of the user terminal; correspondingly, the determining, in response to the file upload request packet sent by the user terminal, a target cloud storage pool for the user terminal according to the internet protocol address cloud resource pool association table includes: and determining the cloud resource pool closest to the user terminal as a target cloud resource pool according to the Internet protocol address of the user terminal and the Internet protocol address cloud resource pool association table.

According to the method and the device for transmitting the data, the target cloud resource pool is determined for the user according to the distance between the user terminal and the cloud resource pool, the high-speed link is established between the cloud resource pools, the data can be transmitted rapidly, the data transmission efficiency between the user and the cloud is improved, the business data throughput rate can be improved greatly, the transmission time delay is reduced, the jitter is reduced, the user can conveniently access the storage system from the nearest end, and the user experience is improved.

Optionally, the upload data request includes a chunk number and a chunk size of the data; correspondingly, the determining, according to a preset fog computing node screening algorithm, a target fog computing node list for uploading data to the user terminal includes: determining a fog computing node with the distance from the user terminal smaller than a preset distance threshold according to the Internet interconnection protocol address of the user terminal; and according to a preset fog calculation node screening algorithm, determining a target fog calculation node list for uploading data by the user terminal by combining the block size and the coincidence state of the fog calculation nodes with the distance smaller than a preset distance threshold value with the user terminal.

According to the cloud computing node list, the cloud computing nodes around the user are searched according to the address of the user terminal, and the cloud computing node list meeting the requirements is screened according to the block size and the load state of the cloud computing nodes, so that the target cloud computing node with the lowest transmission delay and the highest transmission speed can be determined, the cloud network data storage efficiency is further improved, and the user experience is improved.

Optionally, after the sending the target cloud computing node list to the user terminal, the method further includes: after the user terminal establishes connection with the target fog computing node, receiving block data uploaded by the user terminal through the target fog computing node; and uploading the block data to a target cloud resource pool through the target fog computing node.

According to the cloud computing method and the cloud computing device, the data of the user terminal are uploaded to the cloud resource pool through the cloud computing node, and the data closer to the network edge are processed, so that the delay of data transmission can be obviously reduced by the cloud computing, the response time is shortened, the security can be improved by processing and analyzing the data at a place closer to a source, the risk that sensitive data are transmitted to a remote data center or cloud through a network is reduced, the security and the transmission efficiency of cloud network data storage are further improved, and the user experience is improved.

Optionally, after the block data uploaded by the user terminal is received by the target fog computing node, the method further includes: performing abstract calculation according to the abstract data and the block serial number of the block data; and if the abstract calculation is wrong, sending error information to the user terminal.

The application can timely find errors of data transmission and timely inform the user terminal to remind the user terminal to retransmit the data, thereby improving the accuracy and reliability of cloud network data storage and further improving user experience.

Optionally, after the uploading, by the target fog computing node, the block data to a target cloud resource pool, the method further includes: and updating the storage record according to a preset format.

After the user data is successfully uploaded, the storage record is updated in time, so that a user can know the storage condition conveniently, and the download of the storage data is realized according to the storage record, so that the user experience is further improved.

Optionally, after updating the storage record according to the preset format, the method further includes:

and responding to a data downloading request of the user terminal, and determining the storage position of the data to be downloaded of the user terminal through the internet protocol address cloud resource pool association table.

The application not only provides a data uploading method for reducing the time delay, but also can rapidly download the data stored by the cloud server, thereby improving the data processing efficiency and further improving the user experience.

In a second aspect, the present application provides a cloud network data storage apparatus, including:

the system comprises a correlation table generation module, a user terminal internet protocol address generation module and a user terminal internet protocol address generation module, wherein the correlation table generation module is used for generating an internet protocol address cloud resource pool correlation table according to cloud resource pool identifiers and the user terminal internet protocol address, wherein connection is realized between any two cloud resource pools through a high-speed link, and the bandwidth of the high-speed link is larger than a preset bandwidth threshold;

the fog computing node establishing module is used for establishing a plurality of fog computing nodes according to the density degree of the user terminal;

A resource locator generating module, configured to generate a resource locator in response to a registration request of a user terminal, and send the resource locator to the user terminal;

the cloud resource pool determining module is used for responding to a file uploading request packet sent by a user terminal, determining a target cloud storage pool for the user terminal according to the Internet interconnection protocol address cloud resource pool association table, and sending the target cloud storage pool address identifier to the user terminal so that the user terminal initiates an uploading data request after receiving the target cloud storage pool address identifier, wherein the file uploading request packet comprises a resource locator of the user terminal;

the fog computing node determining module is used for responding to the uploading data request, determining a target fog computing node list for uploading data to the user terminal according to a preset fog computing node screening algorithm, wherein the target fog computing node list comprises at least one target fog computing node;

and the sending module is used for sending the target fog computing node list to the user terminal so as to enable the user terminal to establish connection with the target fog computing node, and uploading data to the target cloud resource pool through the target fog computing node.

Optionally, the registration request includes a user identification;

correspondingly, the resource locator generation module is specifically configured to:

and responding to the registration request of the user terminal, and carrying out hash calculation on the user identifier and the registration date to obtain the resource locator.

Optionally, the file upload request packet further includes an internet protocol address of the user terminal;

correspondingly, the cloud resource pool determining module is specifically configured to:

and determining the cloud resource pool closest to the user terminal as a target cloud resource pool according to the Internet protocol address of the user terminal and the Internet protocol address cloud resource pool association table.

Optionally, the upload data request includes a chunk number and a chunk size of the data;

correspondingly, the fog computing node determining module is specifically configured to:

determining a fog computing node with the distance from the user terminal smaller than a preset distance threshold according to the Internet interconnection protocol address of the user terminal;

and according to a preset fog calculation node screening algorithm, determining a target fog calculation node list for uploading data by the user terminal by combining the block size and the coincidence state of the fog calculation nodes with the distance smaller than a preset distance threshold value with the user terminal.

Optionally, after the sending module is configured to send the target fog computing node list to the user terminal, the foregoing apparatus further includes an uploading module configured to:

after the user terminal establishes connection with the target fog computing node, receiving block data uploaded by the user terminal through the target fog computing node;

and uploading the block data to a target cloud resource pool through the target fog computing node.

Optionally, after the uploading module is configured to receive, by the target fog computing node, the block data uploaded by the user terminal, the apparatus further includes:

performing abstract calculation according to the abstract data and the block serial number of the block data;

and if the abstract calculation is wrong, sending error information to the user terminal.

Optionally, after the uploading module is configured to upload the block data to a target cloud resource pool through the target fog computing node, the apparatus further includes an updating module configured to:

and updating the storage record according to a preset format.

Optionally, after the updating module updates the storage record according to the preset format, the apparatus further includes a downloading module, configured to:

In a third aspect, the present application provides a cloud network data storage device based on fog calculation, including: at least one processor and memory;

the memory stores computer-executable instructions;

the at least one processor executes the computer-executable instructions stored by the memory, causing the at least one processor to perform the cloud network data storage method based on fog computing as described above in the first aspect and the various possible designs of the first aspect.

In a fourth aspect, the present application provides a computer readable storage medium, in which computer executable instructions are stored, which when executed by a processor, implement the cloud network data storage method based on fog computing according to the first aspect and the various possible designs of the first aspect.

In a fifth aspect, the present application provides a computer program product comprising a computer program which, when executed by a processor, implements the cloud network data storage method based on fog computing as described in the first aspect and the various possible designs of the first aspect.

According to the cloud network data storage method, device, equipment and medium based on fog calculation, which are provided by the application, a high-speed link established between cloud resource pools is established, the data transmission delay is low and high, an Internet interconnection protocol address cloud resource pool association table is generated aiming at a user terminal and the cloud resource pools, a plurality of fog calculation nodes are established based on the degree of density of the user terminal, the fog calculation nodes can provide data uploading service for users, the data can be processed and analyzed at a place closer to a source at the network edge, the data to be stored by the users are rapidly uploaded to the cloud resource pool closest to the users in the Internet interconnection protocol address cloud resource pool association table, more access points are established for the user terminal, so that the data transmission efficiency between the users and the clouds is improved, the business data throughput rate is improved greatly, the transmission delay is reduced, the time delay of the users from the nearest terminal to the storage system is reduced, the user experience is improved, and the convenience of cloud storage is improved.

Drawings

In order to more clearly illustrate the embodiments of the application or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the description below are only some embodiments of the application, and that other drawings can be obtained according to these drawings without inventive faculty for a person skilled in the art.

Fig. 1 is a schematic diagram of a cloud network data storage system architecture based on fog calculation according to an embodiment of the present application;

fig. 2 is a schematic flow chart of a cloud network data storage method based on fog calculation according to an embodiment of the present application;

fig. 3 is a schematic structural diagram of a cloud network data storage device based on fog calculation according to an embodiment of the present application;

fig. 4 is a schematic structural diagram of cloud network data storage equipment based on fog calculation according to an embodiment of the present application.

Specific embodiments of the present disclosure have been shown by way of the above drawings and will be described in more detail below. These drawings and the written description are not intended to limit the scope of the disclosed concepts in any way, but rather to illustrate the disclosed concepts to those skilled in the art by reference to specific embodiments.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.

The terms "first," "second," "third," and "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or fully authorized by each party, and the collection, use and processing of the related data need to comply with related laws and regulations and standards, and provide corresponding operation entries for the user to select authorization or rejection.

For cloud storage, cloud service providers have operated for several years, but for cloud storage, each resource pool is constructed separately, and each cloud resource pool is separately provided with services, that is, a user purchases a storage service of which resource pool, which storage resource pool is used for storage, and once purchased, only one storage pool is used for storage service. Users who often have mobility requirements are not friendly, for example, users purchase a service from the A place, roam to the B place, access the Internet from the B place, route to the A place through the link of the Internet, and route to the storage service of the cloud storage resource pool of the A place through the Internet link of the A place. The delay, packet loss and jitter of long data transmission from the B place to the A place are needed, and the user performance is poor. The cloud network data mode in the prior art has longer data transmission time delay and poor user experience.

In order to solve the technical problems, the embodiment of the application provides a cloud network data storage method, a device, equipment and a medium based on fog computing, which are characterized in that file blocks are scheduled to a cloud resource pool nearest to a user through high-speed inter-cloud network storage scheduling, and are uploaded to the nearest cloud resource pool when the user uploads the file blocks, so that time delay, jitter and packet loss probability when the user uses storage are reduced, further, through the introduction of fog computing nodes, connection points of the user terminals are increased, more access points are established for the user terminals, so that transmission efficiency is improved, transmission time delay is reduced, and service data throughput rate is improved.

Optionally, fig. 1 is a schematic diagram of a cloud network data storage system architecture based on fog calculation according to an embodiment of the present application. As shown in fig. 1, the above architecture includes: the storage management system 101, the user terminal 102, the first cloud computing node 103, the second cloud computing node 104, the third cloud computing node 105, the first cloud resource pool 106, the second cloud resource pool 107, and the third cloud resource pool 108.

The storage management system 101 includes a cloud operator storage management system 1011 and a cloud resource pool storage management system 1012.

The user terminal 102 is connected to the storage management system 101.

The cloud resource pool storage management system 1012 is connected to the first cloud resource pool 106, the second cloud resource pool 107, and the third cloud resource pool 108, and may implement communication with the first cloud resource pool 106, the second cloud resource pool 107, and the third cloud resource pool 108, and manage the first cloud resource pool 106, the second cloud resource pool 107, and the third cloud resource pool 108.

The user terminal 102 is connected to the first mist computing node 103, the second mist computing node 104 and the third mist computing node 105, the storage management system 101 is connected to the first mist computing node 103, the second mist computing node 104 and the third mist computing node 105, and both the user terminal 102 and the storage management system 101 can communicate with any mist computing node.

And communication is realized between any two cloud resource pools through a high-speed link.

Alternatively, the cloud operator storage management system 1011 and the cloud resource pool storage management system 1012 may be cloud servers.

It will be appreciated that the number and specific structure of the storage management system 101, the user terminal 102, the first cloud computing node 103, the second cloud computing node 104, the third cloud computing node 105, the first cloud resource pool 106, the second cloud resource pool 107, and the third cloud resource pool 108 may be determined according to practical situations, and fig. 1 is only schematic, and the number of the nodes is not specifically limited in the embodiment of the present application.

It will be appreciated that the architecture illustrated by the embodiments of the present application does not constitute a specific limitation on the architecture of cloud network data storage systems based on fog computing. In other possible embodiments of the present application, the architecture may include more or less components than those illustrated, or some components may be combined, some components may be split, or different component arrangements may be specifically determined according to the actual application scenario, and the present application is not limited herein. The components shown in fig. 1 may be implemented in hardware, software, or a combination of software and hardware.

In addition, the network architecture and the service scenario described in the embodiments of the present application are for more clearly describing the technical solution of the embodiments of the present application, and do not constitute a limitation on the technical solution provided by the embodiments of the present application, and as a person of ordinary skill in the art can know, with evolution of the network architecture and occurrence of a new service scenario, the technical solution provided by the embodiments of the present application is also applicable to similar technical problems.

The following description of the present application is given by taking several embodiments as examples, and the same or similar concepts or processes may not be described in detail in some embodiments.

Fig. 2 is a schematic flow chart of a cloud network data storage method based on fog calculation according to an embodiment of the present application, where the embodiment of the present application may be applied to the storage management system 101 in fig. 1, and a specific execution body may be determined according to an actual application scenario. As shown in fig. 2, the method comprises the steps of:

S201: and generating an internet protocol address cloud resource pool association table according to the cloud resource pool identification and the user terminal internet protocol address.

The connection between any two cloud resource pools is realized through a high-speed link, and the bandwidth of the high-speed link is larger than a preset bandwidth threshold.

It will be appreciated that the preset bandwidth threshold may be determined according to practical situations, and embodiments of the present application are not limited in this regard.

In one possible implementation manner, after cloud computing builds cloud resource pools of each place, for each cloud resource pool, a high-speed inter-cloud link is built through a dedicated line, wherein the dedicated line can reach 100g bandwidth or exceed 1000g bandwidth, and for each resource pool of a cloud operator, a mesh link can be achieved as much as possible, namely, a direct high-speed link exists between every two cloud resource pools, or the cloud resource pool can be accessed through one hop at most. And completing storage scheduling among cloud pools through the link. For a storage resource pool of the respective cloud resource pool, a cloud resource pool identification, such as a numbered identity number (Identity document, id), is determined. Meanwhile, for all internet protocol (Internet Protocol, ip) addresses of the user terminals, or ip address pools, namely all possible ip addresses of the users, the ip addresses are divided into designated cloud resource pools to form an ip address cloud resource pool association table. The ip address division assignment relation can be continuously refined and divided to update the ip address cloud resource pool association table due to the increase of the number of the cloud resource pools.

S202: and establishing a plurality of fog calculating nodes according to the density degree of the user terminal.

Optionally, the method includes dividing a plurality of areas according to a preset area size, if the number of the user terminals in a certain area is greater than a preset number threshold, establishing mist computing nodes in the user terminals, and establishing different numbers of mist computing nodes according to the number of the user terminals in different areas.

Optionally, the plurality of areas may be divided according to a preset area size, the user terminals in the areas may be sorted from large to small according to the number of the user terminals in the areas, and the number of the fog calculation nodes in different areas may be determined according to the order.

Alternatively, where the user terminal is dense, a large number of fog computing nodes are built, such as in some public offices, a large number of fog computing nodes are deployed, more than 30 fog computing nodes are deployed 100 square meters, etc

S203: in response to a registration request of a user terminal, a resource locator is generated and sent to the user terminal.

Alternatively, in the embodiment of the present application, the communication with the user terminal may be through an application program of the user terminal or through a portal system, a website, or the like.

Alternatively, the registration request may be a login request or a purchase request of the user terminal.

Optionally, after the user logs into the cloud operator portal, the cloud storage service is purchased.

Optionally, the registration request includes a user identification; accordingly, generating a resource locator in response to a registration request of a user terminal, comprising: and responding to the registration request of the user terminal, and carrying out hash calculation on the user identifier and the registration date to obtain the resource locator.

According to the embodiment of the application, the unique link generated for each user is obtained through hash calculation, so that the user information is prevented from being acquired by a middleman, and the safety of user data and the safety of cloud network data storage are improved.

In one possible implementation, the cloud operator provides a resource locator (Uniform Resource Locator, URL) to the user through a portal through which the user can upload data to the cloud storage. The url is a unique link generated by the cloud operator for each user, and is obtained by carrying out sha256 operation on the user id+registration date. Such as: if the user id corresponding to a certain user, registration time and mobile phone number are: zhang &2022 &185, then the link address after hashing the data is md5 (zhang &2022 & 185) or other summarization algorithms such as sha256, sm3, where the purpose of the algorithm is to prevent the middle person from obtaining the user information from the link. The linking information is stored in the cloud operator storage management system.

Optionally, after the user id verification is completed through the cloud operator portal or the application program provided by the cloud operator, the cloud operator storage management system returns url and file block size requirement block size to the user application program or portal system, and the user application program records url, so that the subsequent process is convenient to use.

S204: and responding to the file uploading request packet sent by the user terminal, determining a target cloud storage pool for the user terminal according to the Internet protocol address cloud resource pool association table, and sending the target cloud storage pool address identification to the user terminal so that the user terminal initiates a data uploading request after receiving the target cloud storage pool address identification.

Wherein the file upload request packet includes a resource locator of the user terminal.

Optionally, the file upload request packet further includes an internet protocol address of the user terminal; correspondingly, responding to a file uploading request packet sent by the user terminal, determining a target cloud storage pool for the user terminal according to the internet protocol address cloud resource pool association table, including: and determining the cloud resource pool closest to the user terminal as a target cloud resource pool according to the Internet protocol address of the user terminal and the Internet protocol address cloud resource pool association table.

According to the embodiment of the application, the target cloud resource pool is determined for the user according to the distance between the user terminal and the cloud resource pool, and the cloud resource pools are established with a high-speed link, so that the data can be quickly transmitted, the data transmission efficiency between the user and the cloud is improved, the service data throughput rate can be greatly improved, the transmission delay is reduced, the jitter is reduced, the user can conveniently access the storage system from the nearest end, and the user experience is improved.

In one possible implementation manner, the user terminal sends a file upload request packet to the cloud operator storage management system, where the request packet includes: url, file size (calculated by the application or portal system), file name and user terminal ip address. And after receiving the user request, the cloud operator storage management system returns information in uploading calculation to the user, and selects a cloud resource pool closest to the ip address to allocate storage space for the user according to the ip address of the user terminal. And returning the target cloud storage pool id to the cloud operator storage management system, and returning the target cloud storage pool id and the cloud resource pool storage management system address of the target cloud storage pool to the user terminal by the cloud operator storage management system.

S205: and responding to the data uploading request, and determining a target fog computing node list for uploading data for the user terminal according to a preset fog computing node screening algorithm.

The target fog computing node list comprises at least one target fog computing node.

Optionally, the upload data request includes a chunk number and chunk size of the data; correspondingly, according to a preset fog computing node screening algorithm, determining a target fog computing node list for uploading data to the user terminal, wherein the target fog computing node list comprises: determining fog calculation nodes with the distance from the user terminal smaller than a preset distance threshold according to the Internet interconnection protocol address of the user terminal; and according to a preset fog calculation node screening algorithm, determining a target fog calculation node list for uploading data to the user terminal by combining the block size and the coincidence state of the fog calculation nodes with the distance smaller than a preset distance threshold value with the user terminal.

According to the embodiment of the application, the fog computing nodes around the user are searched according to the address of the user terminal, and the fog computing node list meeting the requirements is screened according to the block size and the load state of the fog computing nodes, so that the target fog computing node with the lowest transmission delay and highest transmission speed can be determined, the cloud network data storage efficiency is further improved, and the user experience is improved.

Optionally, after receiving the cloud resource pool storage management system address, the user terminal blocks the file according to the system requirement blocksize, and sends an uploading data request to the cloud resource pool storage management system interface. The request includes the number of blocks and the block size of the file system.

After receiving the request, the cloud resource pool storage management system judges the position of the user according to the source ip address carried by the information sent by the user terminal, and searches fog computing nodes around the user according to the position of the user in the fog computing operator system. And screening out a mist computing node list meeting the requirements according to the block size and the load state of the mist computing node, and sending the mist computing node list to the user terminal.

Alternatively, the filtering algorithm of the fog computing node may select the following method:

the comprehensive occupation load of the mist calculating node is as follows:

wherein Calculating the comprehensive occupation load of the nodes for fog, c _i For cpu utilization, m _i B is the memory utilization rate _i Bandwidth utilization for this foggy node. q _i A weight coefficient for each index. The index is determined according to the cpu, the memory and the bandwidth proportion of the fog computing operator according to the specific conditions, and the comparison of the embodiment of the application is not particularly limited. And q ₁ +q ₂ +q ₃ =1, n is any positive integer.

wherein And calculating the idle load and the available resource proportion of the node for fog.

Fog computing node s _i Weight of (2) is

The initial weight of each fog computing node is 10. The comprehensive occupation load of each fog computing node is dynamically changed along with the request of the terminal user, and the weight is dynamically adjusted along with the change of the occupation load of the fog computing node.

Total load F of each fog calculating node _(sum) The definition is as follows:

weight H of each fog calculation node _(sum) The definition is as follows:

sum B of node connection numbers of each fog calculation _(sum) The definition is as follows:

the mist computing node sequence which can be used is found out and sent to the user terminal by calculating the ratio of the load to the weight and the ratio of the connection number to the weight.

The screening algorithm is as follows:

calculating two values P for each fog calculation node ₁ 、P ₂ Form vector P _si (P ₁ 、P ₂ )。

For each fog computing node, a vector P is computed _si And then screening out P according to the result _si The fog nodes with vectors all smaller than 1 form a fog node list array (P _i 、……P _m ……P _r )。

And screening out a proper mist computing node array according to the block size and the block number sent by the user terminal, and sending the mist computing node array to the user terminal. It is necessary to find a memory space larger than the block size. The number of this array is in particular dependent on the operating policy of the fog computing node operator. For example, the number of nodes can be calculated for an array of multiple fog of a high-priority user, and the number of nodes can be calculated for a low-priority user.

S206: and sending the target fog computing node list to the user terminal so as to enable the user terminal to establish connection with the target fog computing node, and uploading data to the target cloud resource pool through the target fog computing node.

In one possible implementation, after receiving the information of the target fog computing node, the user terminal establishes a connection with each fog computing node, and sends each file block to the fog computing node through the connections respectively. And simultaneously, when the block data is transmitted, the abstract data and the block serial number of the data block are transmitted. The summary data can adopt algorithm of sha256, md5, sm3 and the like. After the fog computing node receives the data, the fog computing node firstly carries out abstract computation, if the abstract computation is wrong, the fog computing node replies an error to the user terminal, and the user terminal resends the data block. And if the abstract calculation is correct, the fog calculation node sends the data block to the cloud resource pool storage management system. After the cloud resource pool storage management system receives the data blocks, the data blocks are ordered, and whether unreceived data exist or not is checked. If the received block sequence number is not received, the missing block sequence number is sent to the user terminal, and the user terminal continues to select the fog computing node which has received the reply information to resend the missing block. And if the cloud resource pool storage management system is checked to have no problem, the next step is directly performed. And the cloud resource pool storage management system completes uploading of a file block, and updates a record of the file block in the cloud resource pool storage management system. And the cloud resource pool storage management system receives the record and sends file block storage information to the cloud operator storage management system, and after receiving the record, the cloud operator storage management system updates data in the cloud operator storage management system. The data may be displayed in tabular form. The table header of table 1 may include filename, datablock. Wherein filename is the file name, datablock is the data block digest name of the file, and the result of the digest algorithm (which may be md 5/sha 256, sm3, etc.) is calculated. The table header of table 2 may include block, avaliblezone. Where block is the data block summary data. Avalibzone is the target cloud resource pool id of this data block.

The embodiment of the application establishes a high-speed link established between cloud resource pools, has low data transmission delay and high speed, generates an internet protocol address cloud resource pool association table aiming at the user terminal and the cloud resource pools, establishes a plurality of fog computing nodes based on the degree of density of the user terminal, can provide data uploading service for users, processes and analyzes data at a place which is closer to a source at the network edge, rapidly uploads the data to be stored by the users to the cloud resource pool closest to the users in the internet protocol address cloud resource pool association table, and establishes more access points for the user terminal, thereby improving the data transmission efficiency between the users and the cloud, greatly improving the business data throughput rate, reducing the transmission delay, reducing the jitter, facilitating the users to access a storage system from the nearest end, reducing the data transmission delay, improving the user experience and improving the convenience of cloud storage.

Optionally, after the target fog computing node list is sent to the user terminal, the method further includes: after the user terminal establishes connection with the target fog computing node, receiving block data uploaded by the user terminal through the target fog computing node; and uploading the block data to a target cloud resource pool through the target fog computing node.

According to the embodiment of the application, the data of the user terminal is uploaded to the cloud resource pool through the fog computing node, and the delay of data transmission can be obviously reduced through processing the data closer to the network edge, so that the response time is shortened, the security can be improved through processing and analyzing the data in a place closer to a source, the risk that sensitive data is transmitted to a remote data center or cloud through a network is reduced, the security and the transmission efficiency of cloud network data storage are further improved, and the user experience is improved.

Optionally, after receiving the block data uploaded by the user terminal through the target fog computing node, the method further includes: performing abstract calculation according to the abstract data and the block serial number of the block data; if the abstract calculation is wrong, error information is sent to the user terminal.

The embodiment of the application can timely find errors of data transmission and timely inform the user terminal to remind the user terminal to retransmit the data, thereby improving the accuracy and reliability of cloud network data storage and further improving user experience.

Optionally, after uploading the block data to the target cloud resource pool by the target fog computing node, the method further comprises: and updating the storage record according to a preset format.

After the user data is successfully uploaded, the storage records are updated in time, so that a user can know the storage condition conveniently, and the download of the storage data is realized according to the storage records, so that the user experience is further improved.

and responding to the data downloading request of the user terminal, and determining the storage position of the data to be downloaded of the user terminal through the Internet protocol address cloud resource pool association table.

The embodiment of the application not only provides a data uploading method for reducing the time delay, but also can enable a user to rapidly download data stored by the cloud server, thereby improving the efficiency of data processing and further improving the user experience.

Optionally, the embodiment of the application also provides a data downloading mode, which comprises the following specific steps:

download procedure 1.1: the user logs in the cloud operator storage management system through a cloud operator portal or an application program provided by the cloud operator. And sending a request for browsing the file list, wherein the request for browsing the file list comprises a resource locator url1 sent by the cloud operator storage management system when the user terminal is registered and memory available space information of the user terminal. The cloud operator storage management system returns a user file list to the user, wherein the number of entries is determined according to the size of the available space of the memory of the user terminal, for example, 2g of memory, and possibly 400 entries are returned; 8g of memory, 2000 entry records may be returned.

Download procedure 1.2: the user browses the entry record and selects the file name, and initiates a file download request to the cloud operator storage management system. The request includes the user source ip, the file name of the request, for example: my movie. The cloud operator storage management system responds to the message in the generation of the download link to the user terminal.

Download procedure 1.3: and the cloud operator storage management system searches the cloud resource pool divided by the ip in the ip address cloud resource pool association table according to the source ip in the request. And finding a target cloud resource pool id.

Download procedure 1.4: the cloud operator storage management system determines that the file is not in the target cloud resource pool by looking up the table 1 and the table 2. The cloud operator storage management system initiates a storage scheduling flow, a first block of a file block in the cloud resource pool storage management system is transmitted first, then a second block of the file block is transmitted, and the file block is transmitted to a target cloud resource pool through inter-cloud high-speed and high-priority transmission level scheduling, and the table 2 is updated.

Download procedure 1.5: the cloud operator storage management system generates a download link url3 pointing to the target cloud resource pool and returns the download link url3 to the user terminal.

Download procedure 1.6: the user terminal downloads the file block through url 3. The cloud operator storage management system schedules a third file block to the target cloud resource pool. Meanwhile, the cloud operator storage management system generates a download link of the target cloud resource pool of the block 2. The user continues downloading and so on until all the data blocks have been downloaded. Meanwhile, the cloud operator storage management system updates the table 2, and a target cloud resource pool is added in the avalblezone of the third block and the fourth block. And generates a download link. Multithreaded download may also be supported herein. The user sets a thread upper limit in the application. The upper limit set by the user cannot be larger than the upper limit set in the cloud operator storage management system, so that the system is prevented from being dragged by the user system. Within the upper limit, the user terminal receives a link, namely initiates a downloading request, so as to realize multithreading. And updating the last access time of each block in the table 3 of the target cloud resource pool every time the target cloud resource pool finishes downloading one file block.

Optionally, the embodiment of the application also provides another data downloading mode, which comprises the following specific steps:

download procedure 2.1: the user logs in the cloud operator storage management system through a cloud operator portal or an application program provided by the cloud operator. And sending a request for browsing the file list, wherein the request for browsing the file list comprises url1 and memory available space information of the user terminal. The cloud operator storage management system returns a user file list to the user, the number of entries is determined according to the available space size of the memory of the user terminal, for example, 2g of memory, and 400 entries are returned. 8g of memory would return 2000 entry records.

Download procedure 2.2: the user browses the entry record and selects the file name, and initiates a file download request to the cloud operator storage management system. The request includes the user source ip, the file name of the request-my movie. The cloud operator stores information in the link generation in response to the user terminal download by the management system.

Download procedure 2.3: and the cloud operator storage management system searches the target cloud resource pool divided by the ip in the ip address cloud resource pool association table according to the source ip in the request. And finding a target cloud resource pool id.

Download procedure 2.4: the cloud operator storage management system determines that the file exists in the target cloud resource pool through looking up the table 1 and the table 2.

Download procedure 2.5: the cloud operator storage management system generates a download link url3 pointing to the target cloud resource pool and returns the download link url3 to the user terminal. And continuously generating a download link to the user terminal until all the blocks are downloaded. The fog calculation node method in the uploading process can be used, so that the downloading efficiency is improved.

Download procedure 2.6: and updating the last access time of each block in the table 3 of the target cloud resource pool every time the target cloud resource pool finishes downloading one file block.

Optionally, the embodiment of the application can also realize the adjustment and optimization of the data of the storage system:

after a period of operation, perhaps many repeated data of each cloud pool need to be deleted to increase the storage space, which is beneficial to the following scheduling. And the cloud operator storage management system completes deletion optimization of the data according to the parameter configuration of the operator. If the operator configures 10 parts of data at most, traversing the data stored in the table 2, if the table 2 comprises 5 parts of data, determining that the data is not required to be deleted, if the operator configures 3 parts of data at most, and if the operator comprises 5 parts of data, determining that the data is required to be deleted, after the data blocks with more than 3 parts are searched, searching in the table 3 in each relevant cloud storage pool, updating the data in the table 3 according to the update of the table 2, and deleting the relevant data block information in the table 3. For example, the data block of the resource pool corresponding to the longest access time is deleted.

Fig. 3 is a schematic structural diagram of a cloud network data storage device based on fog calculation according to an embodiment of the present application, where, as shown in fig. 3, the device according to the embodiment of the present application includes: an association table generation module 301, a fog computing node establishment module 302, a resource locator generation module 303, a cloud resource pool determination module 304, a fog computing node determination module 305, and a transmission module 306. The cloud network data storage device based on fog calculation can be a server or a terminal device, or a chip or an integrated circuit for realizing the functions of the server or the terminal device. Here, the division of the association table generating module 301, the fog computing node creating module 302, the resource locator generating module 303, the cloud resource pool determining module 304, the fog computing node determining module 305, and the transmitting module 306 is just one logical division, and the two may be integrated or independent physically.

The system comprises a cloud resource pool generation module, a correlation table generation module and a user terminal internet protocol address generation module, wherein the correlation table generation module is used for generating an internet protocol address cloud resource pool correlation table according to a cloud resource pool identifier and a user terminal internet protocol address, wherein any two cloud resource pools are connected through a high-speed link, and the bandwidth of the high-speed link is larger than a preset bandwidth threshold;

the resource locator generation module is used for responding to the registration request of the user terminal, generating a resource locator and sending the resource locator to the user terminal;

the cloud resource pool determining module is used for responding to a file uploading request packet sent by the user terminal, determining a target cloud storage pool for the user terminal according to the Internet interconnection protocol address cloud resource pool association table, and sending a target cloud storage pool address identifier to the user terminal so that the user terminal initiates an uploading data request after receiving the target cloud storage pool address identifier, wherein the file uploading request packet comprises a resource locator of the user terminal;

the fog computing node determining module is used for responding to the uploading data request, determining a target fog computing node list for uploading data for the user terminal according to a preset fog computing node screening algorithm, wherein the target fog computing node list comprises at least one target fog computing node;

and the sending module is used for sending the target fog computing node list to the user terminal so as to enable the user terminal to establish connection with the target fog computing node, and uploading the data to the target cloud resource pool through the target fog computing node.

Optionally, the registration request includes a user identification;

Optionally, the upload data request includes a chunk number and chunk size of the data;

accordingly, the fog computing node determination module is specifically configured to:

determining fog calculation nodes with the distance from the user terminal smaller than a preset distance threshold according to the Internet interconnection protocol address of the user terminal;

and according to a preset fog calculation node screening algorithm, determining a target fog calculation node list for uploading data to the user terminal by combining the block size and the coincidence state of the fog calculation nodes with the distance smaller than a preset distance threshold value with the user terminal.

Optionally, after the sending module is configured to send the target cloud computing node list to the user terminal, the apparatus further includes an uploading module configured to:

if the abstract calculation is wrong, error information is sent to the user terminal.

Optionally, after the uploading module is configured to upload the block data to the target cloud resource pool through the target fog computing node, the apparatus further includes an updating module configured to:

and updating the storage record according to a preset format.

Referring to fig. 4, there is shown a schematic structural diagram of a cloud network data storage device 400 based on fog computing, which may be a terminal device or a server, suitable for use in implementing embodiments of the present disclosure. The terminal device may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a personal digital assistant (Personal Digital Assistant, PDA for short), a tablet (Portable Android Device, PAD for short), a portable multimedia player (Portable Media Player, PMP for short), an in-vehicle terminal (e.g., an in-vehicle navigation terminal), and the like, and a fixed terminal such as a digital TV, a desktop computer, and the like. The cloud network data storage device based on fog computing shown in fig. 4 is only one example and should not impose any limitation on the functionality and scope of use of embodiments of the present disclosure.

As shown in fig. 4, the cloud network data storage apparatus 400 based on fog computing may include a processing device (e.g., a central processor, a graphics processor, etc.) 401 that may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 402 or a program loaded from a storage device 408 into a random access Memory (Random Access Memory, RAM) 403. In the RAM 403, various programs and data required for the operation of the cloud network data storage apparatus 400 based on fog calculation are also stored. The processing device 401, the ROM 402, and the RAM 403 are connected to each other by a bus 404. An input/output (I/O) interface 405 is also connected to bus 404.

In general, the following devices may be connected to the I/O interface 405: input devices 406 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 407 including, for example, a liquid crystal display (Liquid Crystal Display, LCD for short), a speaker, a vibrator, and the like; storage 408 including, for example, magnetic tape, hard disk, etc.; and a communication device 409. The communication means 409 may allow the cloud network data storage device 400 based on fog calculation to communicate with other devices wirelessly or by wire to exchange data. While fig. 4 illustrates a cloud network data storage apparatus 400 based on fog computing having various means, it should be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead.

In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via communications device 409, or from storage 408, or from ROM 402. The above-described functions defined in the methods of the embodiments of the present disclosure are performed when the computer program is executed by the processing device 401.

It should be noted that the computer readable medium described in the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.

The computer readable medium may be contained in the cloud network data storage device based on fog calculation; or may exist alone without being assembled into the cloud network data storage device based on fog computing.

The computer-readable medium carries one or more programs which, when executed by the cloud computing-based cloud network data storage device, cause the cloud computing-based cloud network data storage device to perform the method shown in the above embodiment.

Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (Local Area Network, LAN for short) or a wide area network (Wide Area Network, WAN for short), or it may be connected to an external computer (e.g., connected via the internet using an internet service provider).

In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of elements is merely a logical functional division, and there may be additional divisions of actual implementation, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any adaptations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. The cloud network data storage method based on fog calculation is characterized by comprising the following steps of:

2. The method of claim 1, wherein the registration request includes a user identification;

accordingly, the generating a resource locator in response to the registration request of the user terminal includes:

3. The method according to claim 1, wherein the file upload request packet further comprises an internet protocol address of the user terminal;

correspondingly, the determining, in response to the file upload request packet sent by the user terminal, a target cloud storage pool for the user terminal according to the internet protocol address cloud resource pool association table includes:

4. The method of claim 1, wherein the upload data request includes a chunk number and chunk size of data;

correspondingly, the determining, according to a preset fog computing node screening algorithm, a target fog computing node list for uploading data to the user terminal includes:

5. The method according to any one of claims 1 to 4, further comprising, after said transmitting the target fog computing node list to the user terminal:

6. The method of claim 5, further comprising, after said receiving, by said target mist computing node, block data uploaded by said user terminal:

7. The method of claim 6, further comprising, after the uploading the block data to a target cloud resource pool by the target fog computing node:

and updating the storage record according to a preset format.

8. The method of claim 7, further comprising, after said updating the stored record according to the preset format:

9. Cloud network data storage device based on fog calculation, characterized by comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the cloud network data storage method based on fog computing as claimed in any one of claims 1 to 8.

10. A computer readable storage medium having stored therein computer executable instructions which when executed by a processor are for implementing a cloud network data storage method based on mist computing as claimed in any of claims 1 to 8.