WO2021082401A1 - 一种数据的上传方法、系统、装置及电子设备 - Google Patents
一种数据的上传方法、系统、装置及电子设备 Download PDFInfo
- Publication number
- WO2021082401A1 WO2021082401A1 PCT/CN2020/090654 CN2020090654W WO2021082401A1 WO 2021082401 A1 WO2021082401 A1 WO 2021082401A1 CN 2020090654 W CN2020090654 W CN 2020090654W WO 2021082401 A1 WO2021082401 A1 WO 2021082401A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- block
- block data
- data
- upload
- storage system
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000013500 data storage Methods 0.000 claims abstract description 103
- 238000004590 computer program Methods 0.000 claims description 12
- 230000004044 response Effects 0.000 claims description 4
- 238000004891 communication Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000012546 transfer Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000012790 confirmation Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/64—Protecting data integrity, e.g. using checksums, certificates or signatures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/50—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols using hash chains, e.g. blockchains or hash trees
Definitions
- This application relates to the field of blockchain technology, and in particular to a method, system, device, electronic equipment, and computer-readable medium for uploading data.
- Blockchain refers to a traceable and non-tamperable distributed accounting mechanism realized through cryptography technology, peer-to-peer network, consensus mechanism and block chain data structure organized in time sequence.
- the transaction in the blockchain technology ecology will distribute each transfer to each node and get their confirmation. All parties need to reach a consensus before the transaction can be completed.
- each node also contains the full account book. From the perspective of the entire network, this occupies a larger space of the disk, which is not conducive to the normal operation of the network.
- the present invention uses a block data storage system to store historical accounts.
- the block data storage system is Designed as a distributed file system suitable for running on general-purpose hardware, it can provide high-throughput data access and is very suitable for applications on large-scale data sets. Therefore, the present invention provides a method, device, system and computer readable medium for uploading data.
- the purpose of the embodiments of the present application is to provide a data uploading method, system, device, electronic device, and computer readable medium, so as to solve the problem that multiple nodes of the blockchain occupy a large disk space.
- the specific technical solutions are as follows:
- a method for uploading data which is characterized in that it includes:
- block data corresponding to the block upload request is not uploaded to the block data storage system, obtain the block data and upload the block data to the block data storage system storage;
- an upload success instruction is sent to the slave server, so as to instruct the slave server to delete the block data stored on the lightweight peer in the target node.
- the method further includes:
- block data corresponding to the block upload request has been uploaded to the block data storage system, send an uploaded response to the slave server, so that the slave server deletes the lightweight peer's block data.
- the obtaining the block data includes:
- the obtaining the block data includes:
- the method further includes:
- the block data corresponding to the block upload request is determined.
- the judging whether the block data corresponding to the block upload request has been uploaded to the block data storage system includes:
- the method further includes:
- a data upload system is characterized in that it includes:
- Lightweight peer used to store block data
- the server is configured to receive a block upload request and send the block data to the block data storage system, and send an upload success instruction to the slave server after the upload is completed;
- a block data storage system configured to receive the block data sent by the server, and send upload completion information to the server;
- the slave server is used to receive the upload success instruction from the server and delete the block data of the lightweight peer.
- a device for uploading data is characterized in that it includes:
- the receiving module is used to receive the block upload request sent by the slave server of the target node;
- the judging module is used to judge whether the block data corresponding to the block upload request has been uploaded to the block data storage system
- the obtaining module is configured to obtain the block data if the block data corresponding to the block upload request is not uploaded to the block data storage system, and upload the block data to the block Storage space in the data storage system;
- the sending module is configured to send an upload success instruction to the slave server if the upload is successful, so as to instruct the slave server to delete the block data stored on the lightweight peer in the target node.
- an electronic device including a memory and a processor, and a computer program that can run on the processor is stored in the memory, wherein the processor executes the computer program when the computer program is executed.
- a computer-readable medium having non-volatile program code executable by a processor wherein the program code causes the processor to execute any of the methods.
- the present invention adopts a block data storage system to centrally store the historical ledgers of each node, so that each node only saves the latest information and data, which can reduce the amount of disk occupation , It also reduces the redundant information of the disk.
- the present invention adopts a block data storage system to centrally store the historical ledgers of each node, so that each node only saves the latest information and data, which can reduce the amount of disk occupation , It also reduces the redundant information of the disk.
- any product or method of the present application does not necessarily need to achieve all the advantages described above at the same time.
- Figure 1 is a schematic diagram of the overall framework provided by an embodiment of the application.
- FIG. 2 is a flowchart of a method for uploading data according to an embodiment of the application
- FIG. 3 is a sequence diagram of a method for uploading data according to an embodiment of the application
- FIG. 4 is a flowchart of a data query method provided by an embodiment of the application.
- FIG. 5 is a schematic structural diagram of a data uploading device provided by an embodiment of this application.
- FIG. 6 is a schematic structural diagram of an electronic device provided by an embodiment of the application.
- the embodiment of the present application provides a method for uploading data, which can be applied to a server in a data uploading system.
- a schematic diagram of a data upload framework is provided for an embodiment of the present invention, which includes a subordinate server, a server, a lightweight peer, a full peer, and a block data storage system.
- the lightweight peer is the node that stores the latest block data in the blockchain
- the full peer is the node that stores all the block data in the blockchain.
- the slave server and the lightweight peer are located in the same local area network.
- Block data can be uploaded to the server according to the storage capacity of the lightweight peer's block data, and the block data in the lightweight peer can be deleted.
- the server and the block data storage system exist in the same local area network.
- the server and the subordinate server transmit communication information to each other.
- the server receives the block upload request sent by the subordinate server and determines whether to upload the request to the block data storage system, such as a server Upload the request to the block data storage system, and then send the upload success instruction returned by the block data storage system to the slave server, so that the slave server deletes the block data in the lightweight peer; if the server determines that the request does not need to be uploaded To the block data storage system, the uploaded instruction is sent to the slave server so that the slave server deletes the block data in the lightweight peer.
- Step 201 Receive a block upload request sent by a slave server of the target node.
- Blockchain is a new application mode of computer technology such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm, etc.
- the transaction in the blockchain technology ecology will distribute each transfer to each target node and get their confirmation. The transaction can be completed. Since each target node in the blockchain saves the same block data, it occupies a larger space on the disk, which is not conducive to the long-term stable operation of the system. Therefore, the same block in the target node needs to be Data upload.
- the target node includes a lightweight peer and a subordinate server. Before the target node uploads the block data of the lightweight peer, the subordinate server generates a block upload request corresponding to the block data and sends it to the server, and the server receives the block upload request.
- Step 202 Determine whether the block data corresponding to the block upload request has been uploaded to the block data storage system.
- the server In order to prevent multiple block data from being repeatedly uploaded to the block data storage system, the server must analyze the received block upload request and judge the block upload Whether the block data corresponding to the request has been uploaded to the block data storage system, specifically, the server judges the identification information and history area corresponding to the block upload request by judging the identification information corresponding to the block upload request Whether the historical identification information included in the block upload request is consistent is used to determine whether the block data corresponding to the block upload request has been uploaded to the block data storage system.
- the identification information refers to the characteristics of the block data corresponding to the block upload request, such as: English or numeric codes, symbols, patterns, etc., which are not specifically limited in the embodiment of the present invention.
- the block data storage system is a highly fault-tolerant system that can provide high-throughput data access and is very suitable for applications on large-scale data sets.
- the block data storage system is HDFS (Hadoop Distributed System, Hadoop Distributed File System) as an example for description.
- the file size of HDFS is from GB to TB. Therefore, HDFS is adjusted to support large files. It should provide a high aggregate data bandwidth, support hundreds of nodes in a cluster, and support tens of millions of files in a cluster.
- HDFS opens the file name space and allows user data to be stored in the form of files.
- Step 203 If the block data corresponding to the block upload request is not uploaded to the block data storage system, obtain the block data, and upload the block data to the storage space in the block data storage system.
- the server determines that the identification information contained in the block upload request is inconsistent with the historical identification information contained in the historical block upload request, it indicates that the block data corresponding to the identification information has not been uploaded to the block data storage system. Therefore, the server obtains the area. Block data and upload the block data to the block data storage system.
- Step 204 If the upload is successful, send an upload success instruction to the slave server, so as to instruct the slave server to delete the block data stored on the lightweight peer in the target node.
- the server If the server successfully uploads the block data to the block data storage system, the lightweight peer in the blockchain and the block data storage system contain the same block data. In order to reduce the disk occupation, the server will Send the upload success instruction back to the slave server, and record the block upload request corresponding to this upload success instruction as a historical block upload request, so as to compare with the new block upload request. After receiving the upload success instruction, the slave server deletes the block data stored on the lightweight peer.
- the block upload request of the lightweight peer that has uploaded the block data is a historical block upload request.
- the server receives other lightweight peer block upload requests, the block data identification information contained in the block upload request of other lightweight peers and the block data identification information contained in the historical block upload request If the information is consistent, the server will determine that the block data of other lightweight peers has been uploaded to the block data storage system, and the server will send an uploaded response to the slave server, indicating that the block data does not need to be re-uploaded, and instruct the slave server to send The block data of the lightweight peer requested by the block upload is deleted.
- the embodiment of the present invention may also be that after the server successfully uploads the block data of a lightweight peer to the block data storage system, due to the block data contained in the lightweight peer and the lightweight peer in other target nodes
- the server automatically sends a delete instruction to each slave server to facilitate each slave server to delete the block data of the lightweight peer in their respective target node in time, avoiding the server from repeatedly receiving the block upload request of the lightweight peer from other target nodes. Reduce the amount of data calculation on the server, and reduce the amount of data communication occupied by the block upload requests corresponding to the lightweight peers of other target nodes.
- the slave server in the blockchain, and the slave server is in the same local area network as the lightweight peer.
- the slave server monitors the regional data storage capacity of the lightweight peer, and the regional data storage capacity of the lightweight peer exceeds the setting. In the case of a threshold value, the slave server sends a block upload request to the server, where the block upload request is used to request the block data to be uploaded to the block data storage system.
- the slave server When the regional data storage capacity of the lightweight peer exceeds the set threshold, the slave server sends a block upload request to the server, so that the block data can be uploaded to the block data storage system through the server and delete the data in the lightweight peer. Block data to reduce the amount of disk space occupied.
- the slave server can also periodically send a block upload request to the server to avoid excessive block data stored in the lightweight peer and waste of disk space occupation.
- the server obtains the block data and uploads the block data to the block data storage system, including two obtaining methods:
- the server obtains the block data from the lightweight peer and uploads the block data to the block data storage system, where the lightweight peer and the slave server are in the same local area network.
- the server acquires the block data from all peers, and uploads the block data to the block data storage system.
- the full peer, server and block data storage system are in the same local area network, and the component peer and the slave server are in another local area network. Therefore, the server obtains block data from the full peer in the same local area network, which can speed up block data acquisition speed.
- the specific method for obtaining block data is: the server parses the block data identification information from the block upload request according to the block upload request received from the slave server, and based on the identification information, from the zone
- the lightweight peer of the blockchain obtains the block data corresponding to the identification information.
- the identification information of the block data is parsed from the block upload request; based on the identification information, the block data corresponding to the block upload request is determined.
- each block data to be uploaded corresponds to a block upload request
- the server can parse the corresponding block data identification information from the block upload request, where the identification information refers to
- the features of the block data corresponding to the block upload request such as English or numeric codes, symbols, patterns, etc., are not specifically limited in the embodiment of the present invention.
- the server determines the block data corresponding to the block upload request.
- record the historical block upload request determine whether the identification information included in the block upload request is consistent with the historical identification information included in the historical block upload request; if they are consistent, determine the block corresponding to the block upload request The data has been uploaded to the block data storage system.
- each block upload request contains the identification information corresponding to the block data
- the server will record and upload the block data every time after uploading the block data to the block data storage system.
- the historical block upload request corresponding to the identification information of the block data.
- the server receives a new block upload request, it will determine whether the identification information contained in the new block upload request is consistent with the historical identification information contained in the historical block upload request, so as to determine the area corresponding to the identification information. Whether the block data has been uploaded to the block data storage system, if the judgment is consistent, it is determined that the block data corresponding to the block upload request has been uploaded to the block data storage system.
- the number of block data storage systems is less than the total number of light-weight peers in the disk.
- the block data of each light-weight peer Only by uploading to the block data storage system will the duplicate block data be reduced and the disk space occupied.
- the number of block data storage systems is three, then after all the lightweight peers upload the block data to the three block data storage systems respectively , If all the block data in ten lightweight peers are deleted, the disk contains only three identical block data, which are stored in the three block data storage systems, which can reduce the amount of disk space and save expenses. It can also achieve the purpose of backup and prevent block data loss.
- the embodiment of the present invention also provides a data query method, as shown in FIG. 4, including the following steps:
- Step 401 Receive a data query request sent by a slave server, where the data query request is used to request to query the block data of the lightweight peer.
- each lightweight peer Since each lightweight peer has uploaded the block data to the block data storage system, each lightweight peer only contains the latest block data. If the user wants to query the target block data that has been uploaded to the block data storage system , The slave server will send a data query request corresponding to the target block data to the server, where the data query request is used to request to query the block data of the lightweight peer.
- Step 402 Send the data query request to the block data storage system, so that the block data storage system searches for block storage data containing block data according to the data query request.
- the server After the server receives the data query request sent by the slave server, it sends the data query request to the block data storage system, and the block data storage system searches for the block containing the block data according to the block data corresponding to the data query request Store the data, and send the block storage data to the server.
- the embodiment of the present invention takes HDFS as an example.
- the internal mechanism of HDFS is to divide a file into one or more blocks. These blocks are stored in a group of data nodes.
- the name node is used to operate file or directory operations in the file namespace, such as Open, close, rename, etc. It also determines the mapping between blocks and data nodes, which are responsible for read and write requests from file system clients.
- HDFS stores the block data uploaded by multiple servers in the same group of data nodes in advance. After the server receives the data query request sent by the slave server, it sends the data query request to HDFS, and HDFS according to the area corresponding to the data query request Block data, query the block storage data containing the block data, and send the block storage data to the server.
- Step 403 Receive the block storage data sent by the block data storage system.
- the block data storage system sends the block storage data to the server, and the server receives the block storage data sent by the block data storage system.
- Step 404 Send the block storage data to the slave server, so that the slave server can send the block storage data to the lightweight peer, and query the block data from the block storage data through the lightweight peer.
- the server block storage data is sent to the slave server so that the slave server can send the block storage data to the lightweight peer. Since the block storage data contains multiple block data, the lightweight peer can query from multiple block data To the block data corresponding to the data query request, the query of the block data is completed.
- the embodiment of the present application also provides a sequence diagram of data upload, as shown in Fig. 3:
- Step S301 upload block data request
- the slave server After the block data storage capacity of the component peer exceeds the preset storage capacity, the slave server sends a block upload request to the server.
- Step S302 Obtain corresponding block data
- the server After receiving the block upload request, the server parses the identification information of the block data contained in the block upload request. If the server determines that the identification information contained in the block upload request is inconsistent with the historical identification information contained in the historical block upload request, Then the server obtains the block data corresponding to the block upload request from the full amount of peers.
- Step S303 upload block data
- the server uploads the block data to the block data storage system, and after receiving the block data, the block data storage system sends an upload success instruction to the server.
- Step S304 Record the block upload request
- the server receives the successful upload instruction, and records the block upload request corresponding to the successful upload instruction.
- Step S305 Send an upload success instruction
- the server returns the upload success instruction to the slave server.
- Step S306 Delete component node block data
- the slave server After the slave server receives the upload success instruction, it deletes the block data on the lightweight peer.
- an embodiment of the present application also provides a device for uploading data.
- the device includes:
- the first receiving module 501 is configured to receive a block upload request sent by a slave server of the target node;
- the judging module 502 is used to judge whether the block data corresponding to the block upload request has been uploaded to the block data storage system;
- the obtaining module 503 is used to obtain the block data when the block data corresponding to the block upload request is not uploaded to the block data storage system, and upload the block data to the storage space in the block data storage system;
- the first sending module 504 is configured to send an upload success instruction to the slave server if the upload is successful, so as to instruct the slave server to delete the block data stored on the lightweight peer in the target node.
- the first sending module 504 may also be used to:
- an uploaded response is sent to the slave server, so that the slave server can delete the block data of the lightweight peer.
- the obtaining module 503 may also be used for:
- the obtaining module 503 may also be used for:
- the obtaining module 503 includes:
- the parsing unit is used to parse the block data identification information from the block upload request
- the obtaining unit is configured to determine the block data corresponding to the block upload request based on the identification information.
- the judgment module 502 includes:
- the recording unit is used to record the upload request of the historical block
- the judging unit is used to judge whether the identification information included in the block upload request is consistent with the historical identification information included in the historical block upload request.
- the determining unit is configured to determine that the block data corresponding to the block upload request has been uploaded to the block data storage system if the determination is consistent.
- the device also includes:
- the second receiving module is configured to receive a data query request sent by the slave server, where the data query request is used to request to query the block data of the lightweight peer;
- the second sending module is used to send the data query request to the block data storage system, so that the block data storage system can find the block storage data containing the block data according to the data query request;
- the third receiving module is used to receive the block storage data sent by the block data storage system
- the third sending module is used to send the block storage data to the slave server, so that the slave server can send the block storage data to the lightweight peer, and query the block data from the block storage data through the lightweight peer.
- the present invention adopts a block data storage system to centrally store the historical ledgers of each node, so that each node only saves the latest information and data, which can reduce the amount of disk occupancy , It also reduces the redundant information of the disk.
- an embodiment of the present application also provides a data upload system, which includes a lightweight peer, a slave server, a server, and a block data storage system, where:
- Lightweight peer used to store block data
- the server is used to receive the block upload request and send the block data to the block data storage system, and send the upload success instruction to the slave server after the upload is completed;
- Block data storage system used to receive block data sent by the server, and send the upload end information to the server;
- the slave server is used to receive the upload success instruction from the server and delete the block data of the lightweight peer.
- an embodiment of the present invention also provides an electronic device, as shown in FIG. 6, including a processor 601, a communication interface 602, a memory 603, and a communication bus 604.
- the processor 601, the communication interface 602, The memory 603 completes mutual communication through the communication bus 604,
- the memory 603 is used to store computer programs
- the processor 601 is configured to implement the steps of the method for uploading data when executing the program stored in the memory 603.
- the communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc.
- PCI Peripheral Component Interconnect
- EISA Extended Industry Standard Architecture
- the communication bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
- the communication interface is used for communication between the above-mentioned electronic device and other devices.
- the memory may include random access memory (Random Access Memory, RAM), and may also include non-volatile memory (Non-Volatile Memory, NVM), such as at least one disk storage.
- NVM non-Volatile Memory
- the memory may also be at least one storage device located far away from the foregoing processor.
- the above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (Network Processor, NP), etc.; it may also be a digital signal processor (Digital Signal Processing, DSP), a dedicated integrated Circuit (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
- CPU central processing unit
- NP Network Processor
- DSP Digital Signal Processing
- ASIC Application Specific Integrated Circuit
- FPGA Field-Programmable Gate Array
- a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it realizes the uploading of any of the above-mentioned data. Method steps.
- a computer program product containing instructions, which when running on a computer, causes the computer to execute any data uploading method in the above-mentioned embodiments.
- the computer may be implemented in whole or in part by software, hardware, firmware, or any combination thereof.
- software it can be implemented in the form of a computer program product in whole or in part.
- the computer program product includes one or more computer instructions.
- the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
- the computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.
- the computer instructions may be transmitted from a website, computer, server, or data center.
- the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center integrated with one or more available media.
- the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Bioethics (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Description
Claims (11)
- 一种数据的上传方法,其特征在于,包括:接收目标节点的从属服务器发送的区块上传请求;判断所述区块上传请求所对应的区块数据是否已上传至区块数据存储系统;若所述区块上传请求所对应的区块数据未上传至所述区块数据存储系统,则获取所述区块数据,并将所述区块数据上传至所述区块数据存储系统中的存储空间;若上传成功,则发送上传成功指令至所述从属服务器,以便于指示所述从属服务器删除所述目标节点中的轻量peer上储存的所述区块数据。
- 根据权利要求1所述的方法,其特征在于,所述方法还包括:若所述区块上传请求所对应的区块数据已上传至所述区块数据存储系统,则向所述从属服务器发送已上传响应,以便于所述从属服务器删除所述轻量peer的区块数据。
- 根据权利要求1所述的方法,其特征在于,所述获取所述区块数据包括:从全量peer获取所述区块数据,其中所述全量peer与所述区块数据存储系统处于同一局域网。
- 根据权利要求1所述的方法,其特征在于,所述获取所述区块数据包括:从所述轻量peer获取所述区块数据,其中所述轻量peer与所述 从属服务器处于同一局域网。
- 根据权利要求1所述的方法,其特征在于,所述方法还包括:从所述区块上传请求中解析出所述区块数据的标识信息;基于所述标识信息,确定所述区块上传请求对应的所述区块数据。
- 根据权利要求5所述的方法,其特征在于,所述判断所述区块上传请求所对应的区块数据是否已上传至区块数据存储系统包括:记录历史区块上传请求;判断所述区块上传请求中包含的所述标识信息与所述历史区块上传请求中包含的历史标识信息是否一致;若一致,则判定所述区块上传请求所对应的区块数据已上传至所述区块数据存储系统。
- 根据权利要求1所述的方法,其特征在于,所述方法还包括:接收所述从属服务器发送的数据查询请求,其中,所述数据查询请求用于请求查询所述轻量peer的区块数据;将所述数据查询请求发送至所述区块数据存储系统,以便于所述区块数据存储系统根据所述数据查询请求,查找包含所述区块数据的区块存储数据;接收所述区块数据存储系统发送的所述区块存储数据;将所述区块存储数据发送至所述从属服务器,以便于所述从属服务器将所述区块存储数据发送至所述轻量peer,通过所述轻量peer从所述区块存储数据中查询所述区块数据。
- 一种数据上传系统,其特征在于,包括:轻量peer,用于存储区块数据;服务器,用于接收区块上传请求并将所述区块数据发送至区块数据存储系统,以及在上传结束后发送上传成功指令至从属服务器;区块数据存储系统,用于接收所述服务器发送的所述区块数据,并将上传结束信息发送至所述服务器;从属服务器,用于从所述服务器接收所述上传成功指令并删除所述轻量peer的区块数据。
- 一种数据的上传装置,其特征在于,包括:接收模块,用于接收目标节点的从属服务器发送的区块上传请求;判断模块,用于判断所述区块上传请求所对应的区块数据是否已上传至区块数据存储系统;获取模块,用于若所述区块上传请求所对应的区块数据未上传至所述区块数据存储系统,则获取所述区块数据,并将所述区块数据上传至所述区块数据存储系统中的存储空间;发送模块,用于若上传成功,则发送上传成功指令至所述从属服务器,以便于指示所述从属服务器删除所述目标节点中的轻量peer上储存的所述区块数据。
- 一种电子设备,包括存储器、处理器,所述存储器中存储有可在所述处理器上运行的计算机程序,其特征在于,所述处理器执行所述计算机程序时实现上述权利要求1至7任一项所述的方法的步骤。
- 一种具有处理器可执行的非易失的程序代码的计算机可读介质,其特征在于,所述程序代码使所述处理器执行所述权利要求1-7任一方法。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2022519757A JP2022550401A (ja) | 2019-10-30 | 2020-05-15 | データのアップロード方法、システム、装置及び電子機器 |
EP20883303.8A EP4024815A4 (en) | 2019-10-30 | 2020-05-15 | METHOD, SYSTEM AND DEVICE FOR UPLOADING DATA AND ELECTRONIC DEVICE |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201911048190.3 | 2019-10-30 | ||
CN201911048190.3A CN110958300B (zh) | 2019-10-30 | 2019-10-30 | 一种数据的上传方法、系统、装置、电子设备和计算机可读介质 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021082401A1 true WO2021082401A1 (zh) | 2021-05-06 |
Family
ID=69975831
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/090654 WO2021082401A1 (zh) | 2019-10-30 | 2020-05-15 | 一种数据的上传方法、系统、装置及电子设备 |
Country Status (4)
Country | Link |
---|---|
EP (1) | EP4024815A4 (zh) |
JP (1) | JP2022550401A (zh) |
CN (1) | CN110958300B (zh) |
WO (1) | WO2021082401A1 (zh) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114925406A (zh) * | 2022-06-01 | 2022-08-19 | 北京百度网讯科技有限公司 | 数据校验方法、装置及计算机程序产品 |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110958300B (zh) * | 2019-10-30 | 2022-06-07 | 京东科技信息技术有限公司 | 一种数据的上传方法、系统、装置、电子设备和计算机可读介质 |
CN111600720B (zh) * | 2020-05-20 | 2021-09-28 | 腾讯科技(深圳)有限公司 | 基于区块链的数据处理方法、装置、设备及可读存储介质 |
CN112291358B (zh) * | 2020-11-02 | 2021-03-16 | 暗链科技(深圳)有限公司 | 一种分段式区块链的数据关联方法与系统 |
CN114928622A (zh) * | 2022-05-07 | 2022-08-19 | 杭州复杂美科技有限公司 | 多子网区块链网络及其数据存储方法、设备和存储介质 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107423426A (zh) * | 2017-08-02 | 2017-12-01 | 众安信息技术服务有限公司 | 一种区块链块数据的数据归档方法及电子设备 |
CN108540566A (zh) * | 2018-04-18 | 2018-09-14 | 暴风集团股份有限公司 | 文件上传方法、装置、系统以及客户端和服务器 |
US20190306176A1 (en) * | 2016-10-17 | 2019-10-03 | Arm Limited | Blockchain mining using trusted nodes |
CN110958300A (zh) * | 2019-10-30 | 2020-04-03 | 北京海益同展信息科技有限公司 | 一种数据的上传方法、系统、装置、电子设备和计算机可读介质 |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2003196129A (ja) * | 2001-12-28 | 2003-07-11 | Toshiba Corp | ファイル転送システム、ファイル転送装置及びファイル転送方法 |
JP2007201861A (ja) * | 2006-01-27 | 2007-08-09 | Eastman Kodak Co | ファイル管理方法 |
US9559889B1 (en) * | 2012-10-31 | 2017-01-31 | Amazon Technologies, Inc. | Cache population optimization for storage gateways |
CN107770115B (zh) * | 2016-08-15 | 2021-01-05 | 华为技术有限公司 | 在对等网络中分发数字内容的方法和系统 |
US11631077B2 (en) * | 2017-01-17 | 2023-04-18 | HashLynx Inc. | System for facilitating secure electronic communications between entities and processing resource transfers |
CN107332876B (zh) * | 2017-05-31 | 2020-05-08 | 深圳前海微众银行股份有限公司 | 区块链状态的同步方法及装置 |
US10749879B2 (en) * | 2017-08-31 | 2020-08-18 | Topia Technology, Inc. | Secure decentralized file sharing systems and methods |
CN108875411A (zh) * | 2018-07-11 | 2018-11-23 | 成都理工大学 | 基于区块链的智能手环数据存储与共享方法 |
CN109299336B (zh) * | 2018-09-30 | 2022-07-01 | 腾讯科技(深圳)有限公司 | 数据备份方法、装置、存储介质及计算设备 |
CN110046894B (zh) * | 2019-04-19 | 2021-11-09 | 电子科技大学 | 一种基于纠删码的分组可重构区块链建立方法 |
-
2019
- 2019-10-30 CN CN201911048190.3A patent/CN110958300B/zh active Active
-
2020
- 2020-05-15 JP JP2022519757A patent/JP2022550401A/ja active Pending
- 2020-05-15 WO PCT/CN2020/090654 patent/WO2021082401A1/zh active Application Filing
- 2020-05-15 EP EP20883303.8A patent/EP4024815A4/en active Pending
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190306176A1 (en) * | 2016-10-17 | 2019-10-03 | Arm Limited | Blockchain mining using trusted nodes |
CN107423426A (zh) * | 2017-08-02 | 2017-12-01 | 众安信息技术服务有限公司 | 一种区块链块数据的数据归档方法及电子设备 |
CN108540566A (zh) * | 2018-04-18 | 2018-09-14 | 暴风集团股份有限公司 | 文件上传方法、装置、系统以及客户端和服务器 |
CN110958300A (zh) * | 2019-10-30 | 2020-04-03 | 北京海益同展信息科技有限公司 | 一种数据的上传方法、系统、装置、电子设备和计算机可读介质 |
Non-Patent Citations (1)
Title |
---|
See also references of EP4024815A4 * |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114925406A (zh) * | 2022-06-01 | 2022-08-19 | 北京百度网讯科技有限公司 | 数据校验方法、装置及计算机程序产品 |
Also Published As
Publication number | Publication date |
---|---|
EP4024815A4 (en) | 2022-08-31 |
CN110958300B (zh) | 2022-06-07 |
JP2022550401A (ja) | 2022-12-01 |
EP4024815A1 (en) | 2022-07-06 |
CN110958300A (zh) | 2020-04-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2021082401A1 (zh) | 一种数据的上传方法、系统、装置及电子设备 | |
KR102476531B1 (ko) | 분산 시스템을 위한 데이터 동기화 방법 및 장치, 매체, 및 전자 디바이스 | |
US10803016B2 (en) | Predictive models of file access patterns by application and file type | |
US9792344B2 (en) | Asynchronous namespace maintenance | |
CN109783438B (zh) | 基于librados的分布式NFS系统及其构建方法 | |
US9699017B1 (en) | Dynamic utilization of bandwidth for a quorum-based distributed storage system | |
US20150215405A1 (en) | Methods of managing and storing distributed files based on information-centric network | |
CN113168404B (zh) | 用于在分布式数据库系统中复制数据的系统和方法 | |
CN111475483B (zh) | 数据库迁移方法、装置及计算设备 | |
US8805849B1 (en) | Enabling use of analytic functions for distributed storage system data | |
CN105025053A (zh) | 基于云存储技术的分布式文件的上传方法及其系统 | |
WO2008131653A1 (fr) | Système et procédé pour la réalisation d'une mémoire d'abonnement à un réseau et serveur d'abonnement | |
WO2015039569A1 (zh) | 副本存储装置及副本存储方法 | |
CN108540510B (zh) | 一种云主机创建方法、装置及云服务系统 | |
WO2013172405A1 (ja) | ストレージシステムおよびデータアクセス方法 | |
US11144407B1 (en) | Synchronous database geo-mirroring using delayed visibility write operations | |
CN111225003B (zh) | 一种nfs节点配置方法和装置 | |
US10545667B1 (en) | Dynamic data partitioning for stateless request routing | |
CN112148206A (zh) | 一种数据读写方法、装置、电子设备及介质 | |
CN105207993A (zh) | 一种cdn中数据的访问、调度方法和系统 | |
US9432238B2 (en) | Communicating large amounts of data over a network with improved efficiency | |
WO2023093608A1 (zh) | 一种自动化分布式云存储调度交互方法、装置及设备 | |
US20240134549A1 (en) | Method and system for data uploading, and computer-readable storage medium | |
CN110196881B (zh) | 一种基于区块链的数据读写方法及区块链网络结构 | |
US11386072B1 (en) | Automatic consistency for database write forwarding |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20883303 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2022519757 Country of ref document: JP Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 2020883303 Country of ref document: EP Effective date: 20220328 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 17769511 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |