WO2021082401A1 - 一种数据的上传方法、系统、装置及电子设备 - Google Patents

一种数据的上传方法、系统、装置及电子设备 Download PDF

Info

Publication number
WO2021082401A1
WO2021082401A1 PCT/CN2020/090654 CN2020090654W WO2021082401A1 WO 2021082401 A1 WO2021082401 A1 WO 2021082401A1 CN 2020090654 W CN2020090654 W CN 2020090654W WO 2021082401 A1 WO2021082401 A1 WO 2021082401A1
Authority
WO
WIPO (PCT)
Prior art keywords
block
block data
data
upload
storage system
Prior art date
Application number
PCT/CN2020/090654
Other languages
English (en)
French (fr)
Inventor
曹龙
马超
孙海波
王�义
赵铭
Original Assignee
北京海益同展信息科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京海益同展信息科技有限公司 filed Critical 北京海益同展信息科技有限公司
Priority to JP2022519757A priority Critical patent/JP2022550401A/ja
Priority to EP20883303.8A priority patent/EP4024815A4/en
Publication of WO2021082401A1 publication Critical patent/WO2021082401A1/zh

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1097Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60Protecting data
    • G06F21/64Protecting data integrity, e.g. using checksums, certificates or signatures
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/104Peer-to-peer [P2P] networks
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/50Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols using hash chains, e.g. blockchains or hash trees

Definitions

  • This application relates to the field of blockchain technology, and in particular to a method, system, device, electronic equipment, and computer-readable medium for uploading data.
  • Blockchain refers to a traceable and non-tamperable distributed accounting mechanism realized through cryptography technology, peer-to-peer network, consensus mechanism and block chain data structure organized in time sequence.
  • the transaction in the blockchain technology ecology will distribute each transfer to each node and get their confirmation. All parties need to reach a consensus before the transaction can be completed.
  • each node also contains the full account book. From the perspective of the entire network, this occupies a larger space of the disk, which is not conducive to the normal operation of the network.
  • the present invention uses a block data storage system to store historical accounts.
  • the block data storage system is Designed as a distributed file system suitable for running on general-purpose hardware, it can provide high-throughput data access and is very suitable for applications on large-scale data sets. Therefore, the present invention provides a method, device, system and computer readable medium for uploading data.
  • the purpose of the embodiments of the present application is to provide a data uploading method, system, device, electronic device, and computer readable medium, so as to solve the problem that multiple nodes of the blockchain occupy a large disk space.
  • the specific technical solutions are as follows:
  • a method for uploading data which is characterized in that it includes:
  • block data corresponding to the block upload request is not uploaded to the block data storage system, obtain the block data and upload the block data to the block data storage system storage;
  • an upload success instruction is sent to the slave server, so as to instruct the slave server to delete the block data stored on the lightweight peer in the target node.
  • the method further includes:
  • block data corresponding to the block upload request has been uploaded to the block data storage system, send an uploaded response to the slave server, so that the slave server deletes the lightweight peer's block data.
  • the obtaining the block data includes:
  • the obtaining the block data includes:
  • the method further includes:
  • the block data corresponding to the block upload request is determined.
  • the judging whether the block data corresponding to the block upload request has been uploaded to the block data storage system includes:
  • the method further includes:
  • a data upload system is characterized in that it includes:
  • Lightweight peer used to store block data
  • the server is configured to receive a block upload request and send the block data to the block data storage system, and send an upload success instruction to the slave server after the upload is completed;
  • a block data storage system configured to receive the block data sent by the server, and send upload completion information to the server;
  • the slave server is used to receive the upload success instruction from the server and delete the block data of the lightweight peer.
  • a device for uploading data is characterized in that it includes:
  • the receiving module is used to receive the block upload request sent by the slave server of the target node;
  • the judging module is used to judge whether the block data corresponding to the block upload request has been uploaded to the block data storage system
  • the obtaining module is configured to obtain the block data if the block data corresponding to the block upload request is not uploaded to the block data storage system, and upload the block data to the block Storage space in the data storage system;
  • the sending module is configured to send an upload success instruction to the slave server if the upload is successful, so as to instruct the slave server to delete the block data stored on the lightweight peer in the target node.
  • an electronic device including a memory and a processor, and a computer program that can run on the processor is stored in the memory, wherein the processor executes the computer program when the computer program is executed.
  • a computer-readable medium having non-volatile program code executable by a processor wherein the program code causes the processor to execute any of the methods.
  • the present invention adopts a block data storage system to centrally store the historical ledgers of each node, so that each node only saves the latest information and data, which can reduce the amount of disk occupation , It also reduces the redundant information of the disk.
  • the present invention adopts a block data storage system to centrally store the historical ledgers of each node, so that each node only saves the latest information and data, which can reduce the amount of disk occupation , It also reduces the redundant information of the disk.
  • any product or method of the present application does not necessarily need to achieve all the advantages described above at the same time.
  • Figure 1 is a schematic diagram of the overall framework provided by an embodiment of the application.
  • FIG. 2 is a flowchart of a method for uploading data according to an embodiment of the application
  • FIG. 3 is a sequence diagram of a method for uploading data according to an embodiment of the application
  • FIG. 4 is a flowchart of a data query method provided by an embodiment of the application.
  • FIG. 5 is a schematic structural diagram of a data uploading device provided by an embodiment of this application.
  • FIG. 6 is a schematic structural diagram of an electronic device provided by an embodiment of the application.
  • the embodiment of the present application provides a method for uploading data, which can be applied to a server in a data uploading system.
  • a schematic diagram of a data upload framework is provided for an embodiment of the present invention, which includes a subordinate server, a server, a lightweight peer, a full peer, and a block data storage system.
  • the lightweight peer is the node that stores the latest block data in the blockchain
  • the full peer is the node that stores all the block data in the blockchain.
  • the slave server and the lightweight peer are located in the same local area network.
  • Block data can be uploaded to the server according to the storage capacity of the lightweight peer's block data, and the block data in the lightweight peer can be deleted.
  • the server and the block data storage system exist in the same local area network.
  • the server and the subordinate server transmit communication information to each other.
  • the server receives the block upload request sent by the subordinate server and determines whether to upload the request to the block data storage system, such as a server Upload the request to the block data storage system, and then send the upload success instruction returned by the block data storage system to the slave server, so that the slave server deletes the block data in the lightweight peer; if the server determines that the request does not need to be uploaded To the block data storage system, the uploaded instruction is sent to the slave server so that the slave server deletes the block data in the lightweight peer.
  • Step 201 Receive a block upload request sent by a slave server of the target node.
  • Blockchain is a new application mode of computer technology such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm, etc.
  • the transaction in the blockchain technology ecology will distribute each transfer to each target node and get their confirmation. The transaction can be completed. Since each target node in the blockchain saves the same block data, it occupies a larger space on the disk, which is not conducive to the long-term stable operation of the system. Therefore, the same block in the target node needs to be Data upload.
  • the target node includes a lightweight peer and a subordinate server. Before the target node uploads the block data of the lightweight peer, the subordinate server generates a block upload request corresponding to the block data and sends it to the server, and the server receives the block upload request.
  • Step 202 Determine whether the block data corresponding to the block upload request has been uploaded to the block data storage system.
  • the server In order to prevent multiple block data from being repeatedly uploaded to the block data storage system, the server must analyze the received block upload request and judge the block upload Whether the block data corresponding to the request has been uploaded to the block data storage system, specifically, the server judges the identification information and history area corresponding to the block upload request by judging the identification information corresponding to the block upload request Whether the historical identification information included in the block upload request is consistent is used to determine whether the block data corresponding to the block upload request has been uploaded to the block data storage system.
  • the identification information refers to the characteristics of the block data corresponding to the block upload request, such as: English or numeric codes, symbols, patterns, etc., which are not specifically limited in the embodiment of the present invention.
  • the block data storage system is a highly fault-tolerant system that can provide high-throughput data access and is very suitable for applications on large-scale data sets.
  • the block data storage system is HDFS (Hadoop Distributed System, Hadoop Distributed File System) as an example for description.
  • the file size of HDFS is from GB to TB. Therefore, HDFS is adjusted to support large files. It should provide a high aggregate data bandwidth, support hundreds of nodes in a cluster, and support tens of millions of files in a cluster.
  • HDFS opens the file name space and allows user data to be stored in the form of files.
  • Step 203 If the block data corresponding to the block upload request is not uploaded to the block data storage system, obtain the block data, and upload the block data to the storage space in the block data storage system.
  • the server determines that the identification information contained in the block upload request is inconsistent with the historical identification information contained in the historical block upload request, it indicates that the block data corresponding to the identification information has not been uploaded to the block data storage system. Therefore, the server obtains the area. Block data and upload the block data to the block data storage system.
  • Step 204 If the upload is successful, send an upload success instruction to the slave server, so as to instruct the slave server to delete the block data stored on the lightweight peer in the target node.
  • the server If the server successfully uploads the block data to the block data storage system, the lightweight peer in the blockchain and the block data storage system contain the same block data. In order to reduce the disk occupation, the server will Send the upload success instruction back to the slave server, and record the block upload request corresponding to this upload success instruction as a historical block upload request, so as to compare with the new block upload request. After receiving the upload success instruction, the slave server deletes the block data stored on the lightweight peer.
  • the block upload request of the lightweight peer that has uploaded the block data is a historical block upload request.
  • the server receives other lightweight peer block upload requests, the block data identification information contained in the block upload request of other lightweight peers and the block data identification information contained in the historical block upload request If the information is consistent, the server will determine that the block data of other lightweight peers has been uploaded to the block data storage system, and the server will send an uploaded response to the slave server, indicating that the block data does not need to be re-uploaded, and instruct the slave server to send The block data of the lightweight peer requested by the block upload is deleted.
  • the embodiment of the present invention may also be that after the server successfully uploads the block data of a lightweight peer to the block data storage system, due to the block data contained in the lightweight peer and the lightweight peer in other target nodes
  • the server automatically sends a delete instruction to each slave server to facilitate each slave server to delete the block data of the lightweight peer in their respective target node in time, avoiding the server from repeatedly receiving the block upload request of the lightweight peer from other target nodes. Reduce the amount of data calculation on the server, and reduce the amount of data communication occupied by the block upload requests corresponding to the lightweight peers of other target nodes.
  • the slave server in the blockchain, and the slave server is in the same local area network as the lightweight peer.
  • the slave server monitors the regional data storage capacity of the lightweight peer, and the regional data storage capacity of the lightweight peer exceeds the setting. In the case of a threshold value, the slave server sends a block upload request to the server, where the block upload request is used to request the block data to be uploaded to the block data storage system.
  • the slave server When the regional data storage capacity of the lightweight peer exceeds the set threshold, the slave server sends a block upload request to the server, so that the block data can be uploaded to the block data storage system through the server and delete the data in the lightweight peer. Block data to reduce the amount of disk space occupied.
  • the slave server can also periodically send a block upload request to the server to avoid excessive block data stored in the lightweight peer and waste of disk space occupation.
  • the server obtains the block data and uploads the block data to the block data storage system, including two obtaining methods:
  • the server obtains the block data from the lightweight peer and uploads the block data to the block data storage system, where the lightweight peer and the slave server are in the same local area network.
  • the server acquires the block data from all peers, and uploads the block data to the block data storage system.
  • the full peer, server and block data storage system are in the same local area network, and the component peer and the slave server are in another local area network. Therefore, the server obtains block data from the full peer in the same local area network, which can speed up block data acquisition speed.
  • the specific method for obtaining block data is: the server parses the block data identification information from the block upload request according to the block upload request received from the slave server, and based on the identification information, from the zone
  • the lightweight peer of the blockchain obtains the block data corresponding to the identification information.
  • the identification information of the block data is parsed from the block upload request; based on the identification information, the block data corresponding to the block upload request is determined.
  • each block data to be uploaded corresponds to a block upload request
  • the server can parse the corresponding block data identification information from the block upload request, where the identification information refers to
  • the features of the block data corresponding to the block upload request such as English or numeric codes, symbols, patterns, etc., are not specifically limited in the embodiment of the present invention.
  • the server determines the block data corresponding to the block upload request.
  • record the historical block upload request determine whether the identification information included in the block upload request is consistent with the historical identification information included in the historical block upload request; if they are consistent, determine the block corresponding to the block upload request The data has been uploaded to the block data storage system.
  • each block upload request contains the identification information corresponding to the block data
  • the server will record and upload the block data every time after uploading the block data to the block data storage system.
  • the historical block upload request corresponding to the identification information of the block data.
  • the server receives a new block upload request, it will determine whether the identification information contained in the new block upload request is consistent with the historical identification information contained in the historical block upload request, so as to determine the area corresponding to the identification information. Whether the block data has been uploaded to the block data storage system, if the judgment is consistent, it is determined that the block data corresponding to the block upload request has been uploaded to the block data storage system.
  • the number of block data storage systems is less than the total number of light-weight peers in the disk.
  • the block data of each light-weight peer Only by uploading to the block data storage system will the duplicate block data be reduced and the disk space occupied.
  • the number of block data storage systems is three, then after all the lightweight peers upload the block data to the three block data storage systems respectively , If all the block data in ten lightweight peers are deleted, the disk contains only three identical block data, which are stored in the three block data storage systems, which can reduce the amount of disk space and save expenses. It can also achieve the purpose of backup and prevent block data loss.
  • the embodiment of the present invention also provides a data query method, as shown in FIG. 4, including the following steps:
  • Step 401 Receive a data query request sent by a slave server, where the data query request is used to request to query the block data of the lightweight peer.
  • each lightweight peer Since each lightweight peer has uploaded the block data to the block data storage system, each lightweight peer only contains the latest block data. If the user wants to query the target block data that has been uploaded to the block data storage system , The slave server will send a data query request corresponding to the target block data to the server, where the data query request is used to request to query the block data of the lightweight peer.
  • Step 402 Send the data query request to the block data storage system, so that the block data storage system searches for block storage data containing block data according to the data query request.
  • the server After the server receives the data query request sent by the slave server, it sends the data query request to the block data storage system, and the block data storage system searches for the block containing the block data according to the block data corresponding to the data query request Store the data, and send the block storage data to the server.
  • the embodiment of the present invention takes HDFS as an example.
  • the internal mechanism of HDFS is to divide a file into one or more blocks. These blocks are stored in a group of data nodes.
  • the name node is used to operate file or directory operations in the file namespace, such as Open, close, rename, etc. It also determines the mapping between blocks and data nodes, which are responsible for read and write requests from file system clients.
  • HDFS stores the block data uploaded by multiple servers in the same group of data nodes in advance. After the server receives the data query request sent by the slave server, it sends the data query request to HDFS, and HDFS according to the area corresponding to the data query request Block data, query the block storage data containing the block data, and send the block storage data to the server.
  • Step 403 Receive the block storage data sent by the block data storage system.
  • the block data storage system sends the block storage data to the server, and the server receives the block storage data sent by the block data storage system.
  • Step 404 Send the block storage data to the slave server, so that the slave server can send the block storage data to the lightweight peer, and query the block data from the block storage data through the lightweight peer.
  • the server block storage data is sent to the slave server so that the slave server can send the block storage data to the lightweight peer. Since the block storage data contains multiple block data, the lightweight peer can query from multiple block data To the block data corresponding to the data query request, the query of the block data is completed.
  • the embodiment of the present application also provides a sequence diagram of data upload, as shown in Fig. 3:
  • Step S301 upload block data request
  • the slave server After the block data storage capacity of the component peer exceeds the preset storage capacity, the slave server sends a block upload request to the server.
  • Step S302 Obtain corresponding block data
  • the server After receiving the block upload request, the server parses the identification information of the block data contained in the block upload request. If the server determines that the identification information contained in the block upload request is inconsistent with the historical identification information contained in the historical block upload request, Then the server obtains the block data corresponding to the block upload request from the full amount of peers.
  • Step S303 upload block data
  • the server uploads the block data to the block data storage system, and after receiving the block data, the block data storage system sends an upload success instruction to the server.
  • Step S304 Record the block upload request
  • the server receives the successful upload instruction, and records the block upload request corresponding to the successful upload instruction.
  • Step S305 Send an upload success instruction
  • the server returns the upload success instruction to the slave server.
  • Step S306 Delete component node block data
  • the slave server After the slave server receives the upload success instruction, it deletes the block data on the lightweight peer.
  • an embodiment of the present application also provides a device for uploading data.
  • the device includes:
  • the first receiving module 501 is configured to receive a block upload request sent by a slave server of the target node;
  • the judging module 502 is used to judge whether the block data corresponding to the block upload request has been uploaded to the block data storage system;
  • the obtaining module 503 is used to obtain the block data when the block data corresponding to the block upload request is not uploaded to the block data storage system, and upload the block data to the storage space in the block data storage system;
  • the first sending module 504 is configured to send an upload success instruction to the slave server if the upload is successful, so as to instruct the slave server to delete the block data stored on the lightweight peer in the target node.
  • the first sending module 504 may also be used to:
  • an uploaded response is sent to the slave server, so that the slave server can delete the block data of the lightweight peer.
  • the obtaining module 503 may also be used for:
  • the obtaining module 503 may also be used for:
  • the obtaining module 503 includes:
  • the parsing unit is used to parse the block data identification information from the block upload request
  • the obtaining unit is configured to determine the block data corresponding to the block upload request based on the identification information.
  • the judgment module 502 includes:
  • the recording unit is used to record the upload request of the historical block
  • the judging unit is used to judge whether the identification information included in the block upload request is consistent with the historical identification information included in the historical block upload request.
  • the determining unit is configured to determine that the block data corresponding to the block upload request has been uploaded to the block data storage system if the determination is consistent.
  • the device also includes:
  • the second receiving module is configured to receive a data query request sent by the slave server, where the data query request is used to request to query the block data of the lightweight peer;
  • the second sending module is used to send the data query request to the block data storage system, so that the block data storage system can find the block storage data containing the block data according to the data query request;
  • the third receiving module is used to receive the block storage data sent by the block data storage system
  • the third sending module is used to send the block storage data to the slave server, so that the slave server can send the block storage data to the lightweight peer, and query the block data from the block storage data through the lightweight peer.
  • the present invention adopts a block data storage system to centrally store the historical ledgers of each node, so that each node only saves the latest information and data, which can reduce the amount of disk occupancy , It also reduces the redundant information of the disk.
  • an embodiment of the present application also provides a data upload system, which includes a lightweight peer, a slave server, a server, and a block data storage system, where:
  • Lightweight peer used to store block data
  • the server is used to receive the block upload request and send the block data to the block data storage system, and send the upload success instruction to the slave server after the upload is completed;
  • Block data storage system used to receive block data sent by the server, and send the upload end information to the server;
  • the slave server is used to receive the upload success instruction from the server and delete the block data of the lightweight peer.
  • an embodiment of the present invention also provides an electronic device, as shown in FIG. 6, including a processor 601, a communication interface 602, a memory 603, and a communication bus 604.
  • the processor 601, the communication interface 602, The memory 603 completes mutual communication through the communication bus 604,
  • the memory 603 is used to store computer programs
  • the processor 601 is configured to implement the steps of the method for uploading data when executing the program stored in the memory 603.
  • the communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc.
  • PCI Peripheral Component Interconnect
  • EISA Extended Industry Standard Architecture
  • the communication bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
  • the communication interface is used for communication between the above-mentioned electronic device and other devices.
  • the memory may include random access memory (Random Access Memory, RAM), and may also include non-volatile memory (Non-Volatile Memory, NVM), such as at least one disk storage.
  • NVM non-Volatile Memory
  • the memory may also be at least one storage device located far away from the foregoing processor.
  • the above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (Network Processor, NP), etc.; it may also be a digital signal processor (Digital Signal Processing, DSP), a dedicated integrated Circuit (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
  • CPU central processing unit
  • NP Network Processor
  • DSP Digital Signal Processing
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it realizes the uploading of any of the above-mentioned data. Method steps.
  • a computer program product containing instructions, which when running on a computer, causes the computer to execute any data uploading method in the above-mentioned embodiments.
  • the computer may be implemented in whole or in part by software, hardware, firmware, or any combination thereof.
  • software it can be implemented in the form of a computer program product in whole or in part.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • the computer instructions may be transmitted from a website, computer, server, or data center.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center integrated with one or more available media.
  • the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Theoretical Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Bioethics (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

本申请提供了一种数据的上传方法,属于区块链技术领域。所述方法包括:接收目标节点的从属服务器发送的区块上传请求;判断所述区块上传请求所对应的区块数据是否已上传至区块数据存储系统;若所述区块上传请求所对应的区块数据未上传至所述区块数据存储系统,则获取所述区块数据,并将所述区块数据上传至所述区块数据存储系统中的存储空间;若上传成功,则发送上传成功指令至所述从属服务器,以便于指示所述从属服务器删除所述目标节点中的轻量peer上储存的所述区块数据。本发明实施例优点:本发明采用区块数据存储系统集中存储各节点的历史账本,使各节点仅保存最新的信息数据,可以减少磁盘占有量,也减少了磁盘的冗余信息。

Description

一种数据的上传方法、系统、装置及电子设备
本申请要求于2019年10月30日提交中国专利局、申请号为201911048190.3、发明名称为“一种数据的上传方法、系统、装置、电子设备和计算机可读介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
技术领域
本申请涉及区块链技术领域,尤其涉及一种数据的上传方法、系统、装置、电子设备和计算机可读介质。
背景技术
区块链是指通过密码学技术、点对点网络、共识机制和按时序组织的块链式数据结构实现的可追溯、不可篡改的分布式记账机制。区块链技术生态中的交易会将每笔转账分发到各个节点并得到它们的确认,需要各方达成共识,才可完成该笔交易,但同时每个节点也就都包含了全量账本,从整个网络的角度来看,这样占据了磁盘的较大空间,不利于网络正常运行,为了能够减小磁盘占有量,本发明采用区块数据存储系统来存储历史账本,区块数据存储系统是被设计成适合运行在通用硬件上的分布式文件系统,能提供高吞吐量的数据访问,非常适合大规模数据集上的应用。因此,本发明提供了一种数据的上传方法、装置、系统和计算机可读介质。
发明内容
本申请实施例的目的在于提供一种数据的上传方法、系统、装置、电子设备和计算机可读介质,以解决区块链的多个节点占用磁盘较大空间的问题。具体技术方案如下:
第一方面,提供了一种数据的上传方法,其特征在于,包括:
接收目标节点的从属服务器发送的区块上传请求;
判断所述区块上传请求所对应的区块数据是否已上传至区块数据存储系统;
若所述区块上传请求所对应的区块数据未上传至所述区块数据存储系统,则获取所述区块数据,并将所述区块数据上传至所述区块数据存储系统中的存储空间;
若上传成功,则发送上传成功指令至所述从属服务器,以便于指示所述从属服务器删除所述目标节点中的轻量peer上储存的所述区块数据。
可选的,所述方法还包括:
若所述区块上传请求所对应的区块数据已上传至所述区块数据存储系统,则向所述从属服务器发送已上传响应,以便于所述从属服务器删除所述轻量peer的区块数据。
可选的,所述获取所述区块数据包括:
从全量peer获取所述区块数据,其中所述全量peer与所述区块数据存储系统处于同一局域网。
可选的,所述获取所述区块数据包括:
从所述轻量peer获取所述区块数据,其中所述轻量peer与所述从属服务器处于同一局域网。
可选的,所述方法还包括:
从所述区块上传请求中解析出所述区块数据的标识信息;
基于所述标识信息,确定所述区块上传请求对应的所述区块数据。
可选的,所述判断所述区块上传请求所对应的区块数据是否已上 传至区块数据存储系统包括:
记录历史区块上传请求;
判断所述区块上传请求中包含的所述标识信息与所述历史区块上传请求中包含的历史标识信息是否一致;
若一致,则判定所述区块上传请求所对应的区块数据已上传至所述区块数据存储系统。
可选的,所述方法还包括:
接收所述从属服务器发送的数据查询请求,其中,所述数据查询请求用于请求查询所述轻量peer的区块数据;
将所述数据查询请求发送至所述区块数据存储系统,以便于所述区块数据存储系统根据所述数据查询请求,查找包含所述区块数据的区块存储数据;
接收所述区块数据存储系统发送的所述区块存储数据;
将所述区块存储数据发送至所述从属服务器,以便于所述从属服务器将所述区块存储数据发送至所述轻量peer,通过所述轻量peer从所述区块存储数据中查询所述区块数据。
第二方面,一种数据上传系统,其特征在于,包括:
轻量peer,用于存储区块数据;
服务器,用于接收区块上传请求并将所述区块数据发送至区块数据存储系统,以及在上传结束后发送上传成功指令至从属服务器;
区块数据存储系统,用于接收所述服务器发送的所述区块数据,并将上传结束信息发送至所述服务器;
从属服务器,用于从所述服务器接收所述上传成功指令并删除所 述轻量peer的区块数据。
第三方面,一种数据的上传装置,其特征在于,包括:
接收模块,用于接收目标节点的从属服务器发送的区块上传请求;
判断模块,用于判断所述区块上传请求所对应的区块数据是否已上传至区块数据存储系统;
获取模块,用于若所述区块上传请求所对应的区块数据未上传至所述区块数据存储系统,则获取所述区块数据,并将所述区块数据上传至所述区块数据存储系统中的存储空间;
发送模块,用于若上传成功,则发送上传成功指令至所述从属服务器,以便于指示所述从属服务器删除所述目标节点中的轻量peer上储存的所述区块数据。
第四方面,提供了一种电子设备,包括存储器、处理器,所述存储器中存储有可在所述处理器上运行的计算机程序,其特征在于,所述处理器执行所述计算机程序时实现上述权利要求任一项所述的方法的步骤。
第五方面,提供了一种具有处理器可执行的非易失的程序代码的计算机可读介质,其特征在于,所述程序代码使所述处理器执行所任一所述方法。
本发明实施例提供的上述技术方案与现有技术相比具有如下优点:本发明采用区块数据存储系统集中存储各节点的历史账本,使各节点仅保存最新的信息数据,可以减少磁盘占有量,也减少了磁盘的冗余信息。当然,实施本申请的任一产品或方法并不一定需要同时达到以上所述的所有优点。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分,示出了符合本申请的实施例,并与说明书一起用于解释本申请的原理。
为了更清楚地说明本申请实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,对于本领域普通技术人员而言,在不付出创造性劳动性的前提下,还可以根据这些附图获得其他的附图。
图1为本申请实施例提供的整体框架示意图;
图2为本申请实施例提供的一种数据上传的方法流程图;
图3为本申请实施例提供的一种数据上传的方法时序图;
图4为本申请实施例提供的一种数据查询的方法流程图;
图5为本申请实施例提供的一种数据上传的装置结构示意图;
图6为本申请实施例提供的一种电子设备的结构示意图。
具体实施方式
为使本申请实施例的目的、技术方案和优点更加清楚,下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例是本申请的一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例,都属于本申请保护的范围。
本申请实施例提供了一种数据的上传方法,可以应用于数据上传系统中的服务器。如图1所示,为本发明实施例提供了一种数据上传框架示意图,包括从属服务器、服务器、轻量peer、全量peer和区块数据存储系统。其中,轻量peer为区块链中存储最新区块数据的节点,全量peer为区块链中存储全部区块数据的节点,当区块链中节点数量较多时,会占用磁盘的大量空间。从属服务器与轻量peer位于同一个 局域网中,可以根据轻量peer的区块数据存储量上传区块数据至服务器,并可删除轻量peer中的区块数据。服务器与区块数据存储系统存在于同一局域网中,服务器与从属服务器之间互相传输通信信息,服务器接收从属服务器发送的区块上传请求并判断是否将该请求上传至区块数据存储系统,如服务器将该请求上传至区块数据存储系统,则将区块数据存储系统返回的上传成功指令发送至从属服务器,以使从属服务器删除轻量peer中的区块数据;如服务器判断无需将该请求上传至区块数据存储系统,则发送已上传指令至从属服务器,以使从属服务器删除轻量peer中的区块数据。
下面将结合具体实施方式,对本申请实施例提供的一种数据的上传方法进行详细的说明,如图2所示,具体步骤如下:
步骤201,接收目标节点的从属服务器发送的区块上传请求。
区块链是分布式数据存储、点对点传输、共识机制、加密算法等计算机技术的新型应用模式,区块链技术生态中的交易会将每笔转账分发到各个目标节点并得到它们的确认,才可完成该笔交易,由于区块链中的各个目标节点都保存了相同的区块数据,占据了磁盘的较大空间,不利于系统的长期稳定运行,因此需要将目标节点中的相同区块数据上传。
目标节点包括轻量peer和从属服务器,在目标节点将轻量peer的区块数据上传之前,从属服务器生成与区块数据对应的区块上传请求并发送至服务器,服务器接收该区块上传请求。
步骤202:判断区块上传请求所对应的区块数据是否已上传至区块数据存储系统。
区块链中存在含有相同区块数据的多个轻量peer,为了防止多个区块数据重复上传至区块数据存储系统,服务器要对接收到的区块上传请求进行分析,判断区块上传请求所对应的区块数据是否已上传至 区块数据存储系统,具体来说,服务器通过对区块上传请求所对应的标识信息进行判断,判断该区块上传请求所对应的标识信息与历史区块上传请求包含的历史标识信息是否一致,以此来判定区块上传请求所对应的区块数据是否已上传至区块数据存储系统。其中,标识信息指与区块上传请求对应的区块数据的特征,例如:英文或数字代码、符号、图案等,本发明实施例不做具体限制。
其中,区块数据存储系统是是一个高度容错性的系统,能提供高吞吐量的数据访问,非常适合大规模数据集上的应用。本发明实施例以区块数据存储系统是HDFS(Hadoop Distributed System,Hadoop分布式文件系统)为例来进行描述,HDFS文件大小是GB到TB的级别,所以,HDFS被调整成支持大文件,它应该提供很高的聚合数据带宽,一个集群中支持数百个节点,一个集群中还应该支持千万级别的文件,HDFS对外开放文件命名空间并允许用户数据以文件形式存储。
步骤203:若区块上传请求所对应的区块数据未上传至区块数据存储系统,则获取区块数据,并将区块数据上传至区块数据存储系统中的存储空间。
若服务器判定区块上传请求中包含的标识信息与历史区块上传请求中包含的历史标识信息不一致,表明该标识信息对应的区块数据未曾上传过区块数据存储系统,因此,服务器获取该区块数据,并将区块数据上传至区块数据存储系统中。
步骤204:若上传成功,则发送上传成功指令至从属服务器,以便于指示从属服务器删除目标节点中的轻量peer上储存的区块数据。
如果服务器成功将区块数据上传至区块数据存储系统,则区块链中轻量peer和区块数据存储系统中含有相同的区块数据,为了减少磁盘占有量,服务器在上传成功后,会发送上传成功指令返回至从属服务器,并将与此次上传成功指令对应的区块上传请求记录为历史区块 上传请求,以便于和新的区块上传请求做对比。从属服务器在接收到上传成功指令后,删除该轻量peer上存储的区块数据。
由于各轻量peer存储的区块数据相同,在其中一个轻量peer将区块数据至区块数据存储系统后,该已上传区块数据的轻量peer的区块上传请求为历史区块上传请求。在这种情形下,如果服务器接收到其他轻量peer区块上传请求,由于其他轻量peer的区块上传请求所包含区块数据的标识信息与历史区块上传请求所包含区块数据的标识信息一致,服务器会判定其他轻量peer的区块数据已上传至区块数据存储系统,则服务器会向从属服务器发送给已上传响应,表示该区块数据无需重新上传,并指示从属服务器将发送区块上传请求的轻量peer的区块数据删除。
可选的,本发明实施例还可以为,服务器成功将一个轻量peer的区块数据上传至区块数据存储系统后,由于该轻量peer和其他目标节点中轻量peer包含的区块数据相同,服务器自动发送删除指令至各从属服务器,以方便各从属服务器及时将各自目标节点中轻量peer的区块数据删除,避免了服务器重复接收其他目标节点的轻量peer的区块上传请求,减少服务器的数据计算量,并减少其他目标节点的轻量peer分别对应的区块上传请求所占的数据通讯量。
在本发明实施例中,区块链中存在从属服务器,该从属服务器与轻量peer处于相同局域网,从属服务器监测轻量peer的区域数据存储量,在轻量peer的区域数据存储量超过设定阈值的情形下,从属服务器发送区块上传请求至服务器,其中,区块上传请求用于请求将区块数据上传至区块数据存储系统。
从属服务器在轻量peer的区域数据存储量超过设定阈值的情形下,发送区块上传请求至服务器,以便于将区块数据通过服务器上传至区块数据存储系统,并删除轻量peer中的区块数据,减少磁盘空间 占有量。
可选的,从属服务器也可以定时发送区块上传请求至服务器,避免轻量peer中存储的区块数据过多,浪费磁盘空间占有量。
在本发明实施例中,服务器获取该区块数据,并将区块数据上传至区块数据存储系统中,包括两种获取方式:
获取方式一:服务器从轻量peer获取该区块数据,并将区块数据上传至区块数据存储系统中,其中轻量peer与从属服务器处于同一局域网。
获取方式二:服务器从全量peer获取该区块数据,并将区块数据上传至区块数据存储系统中。全量peer、服务器和区块数据存储系统处于同一个局域网中,分量peer和从属服务器处于另外的局域网中,因此,服务器从处于同一个局域网中的全量peer获取区块数据,可以加快区块数据获取速度。
在本发明实施例中,获取区块数据的具体方式为:服务器根据从从属服务器接收到区块上传请求,从该区块上传请求中解析出区块数据的标识信息,基于标识信息,从区块链的轻量peer获取与标识信息对应的区块数据。
可选的,从区块上传请求中解析出区块数据的标识信息;基于标识信息,确定区块上传请求对应的区块数据。
在本发明实施例中,每个要上传的区块数据都对应着一个区块上传请求,服务器可以从区块上传请求中解析出对应的区块数据的标识信息,其中,该标识信息为指与区块上传请求对应的区块数据的特征,例如:英文或数字代码、符号、图案等,本发明实施例不做具体限制。基于该标识信息,服务器确定区块上传请求对应的区块数据。
可选的,记录历史区块上传请求;判断区块上传请求中包含的标 识信息与历史区块上传请求中包含的历史标识信息是否一致;若一致,则判定区块上传请求所对应的区块数据已上传至区块数据存储系统。
对于轻量peer的区块数据,每条区块上传请求中都含有对应该区块数据的标识信息,服务器在每次将区块数据上传至区块数据存储系统结束后,都会记录与上传的区块数据的标识信息对应的历史区块上传请求。当服务器接收新的区块上传请求时,会判断该新的区块上传请求中包含的标识信息与历史区块上传请求中包含的历史标识信息是否一致,以此来判定该标识信息对应的区块数据是否曾经上传过区块数据存储系统,若判断一致,则判定区块上传请求所对应的区块数据已上传至区块数据存储系统。
为了达到减少磁盘空间占有量的目的,对于本发明实施例,区块数据存储系统的数量要少于磁盘内各轻量peer的总数量,这种情况下,将各轻量peer的区块数据上传至区块数据存储系统才会减少重复的区块数据,减少磁盘占有空间。举例来说,磁盘内含有包含相同区块数据的十个轻量peer,区块数据存储系统数量为三个,那么在所有轻量peer将区块数据分别上传至三个区块数据存储系统后,十个轻量peer中的区块数据全部删除,则磁盘中只包含三个相同的区块数据,分别存在于三个区块数据存储系统中,既可以减少磁盘空间占有量,节省开支,也可以达到备份的目的,防止区块数据丢失。
本发明实施例还提供了一种数据查询方法,如图4所示,包括以下步骤:
步骤401:接收从属服务器发送的数据查询请求,其中,数据查询请求用于请求查询轻量peer的区块数据。
由于各轻量peer将区块数据已上传至区块数据存储系统,因此各轻量peer只含有最新的区块数据,如果用户想要查询已上传至区块数据存储系统中的目标区块数据,则从属服务器会发送与目标区块数据 对应的数据查询请求至服务器,其中,数据查询请求用于请求查询轻量peer的区块数据。
步骤402:将数据查询请求发送至区块数据存储系统,以便于区块数据存储系统根据数据查询请求,查找包含区块数据的区块存储数据。
服务器接收到从属服务器发送的数据查询请求后,将该数据查询请求发送至区块数据存储系统,区块数据存储系统根据与该数据查询请求对应的区块数据,查找包含区块数据的区块存储数据,并将该区块存储数据发送至服务器。
本发明实施例以HDFS为例,HDFS内部机制是将一个文件分割成一个或多个块,这些块被存储在一组数据节点中,名字节点用来操作文件命名空间的文件或目录操作,如打开,关闭,重命名等等。它同时确定块与数据节点的映射,数据节点负责来自文件系统客户的读写请求。HDFS事先将多个服务器上传的区块数据存储在同一组数据节点中,服务器接收到从属服务器发送的数据查询请求后,将该数据查询请求发送至HDFS,HDFS根据与该数据查询请求对应的区块数据,查询包含区块数据的区块存储数据,并将区块存储数据发送至服务器。
步骤403:接收区块数据存储系统发送的区块存储数据。
区块数据存储系统将该区块存储数据发送至服务器,服务器接收区块数据存储系统发送的区块存储数据。
步骤404:将区块存储数据发送至从属服务器,以便于从属服务器将区块存储数据发送至轻量peer,通过轻量peer从区块存储数据中查询区块数据。
服务器区块存储数据发送至从属服务器,以便于从属服务器将区块存储数据发送至轻量peer,由于区块存储数据中包含多个区块数据,轻量peer可以从多个区块数据中查询到与数据查询请求对应的区块数 据,完成对区块数据的查询。
基于相同的技术构思,本申请实施例还提供了一种数据上传的时序图,如图3所示:
步骤S301:上传区块数据请求;
分量peer的区块数据存储量超过预设存储量后,从属服务器将区块上传请求发送至服务器。
步骤S302:获取对应的区块数据;
服务器在接收到区块上传请求后,解析区块上传请求包含的区块数据的标识信息,若服务器判定区块上传请求中包含的标识信息与历史区块上传请求中包含的历史标识信息不一致,则服务器从全量peer中获取与区块上传请求对应的区块数据。
步骤S303:上传区块数据;
服务器将该区块数据上传至区块数据存储系统,区块数据存储系统接收到该区块数据后,会发送上传成功指令至服务器。
步骤S304:记录区块上传请求;
服务器接收到上传成功指令,会记录与该上传成功指令对应的区块上传请求。
步骤S305:发送上传成功指令;
服务器将该上传成功指令返还至从属服务器。
步骤S306:删除分量节点区块数据;
从属服务器接收到上传成功指令后,会删除轻量peer上的区块数据。
基于相同的技术构思,本申请实施例还提供了一种数据上传的装 置,如图5所示,该装置包括:
第一接收模块501,用于接收目标节点的从属服务器发送的区块上传请求;
判断模块502,用于判断区块上传请求所对应的区块数据是否已上传至区块数据存储系统;
获取模块503,用于区块上传请求所对应的区块数据未上传至区块数据存储系统,则获取区块数据,并将区块数据上传至区块数据存储系统中的存储空间;
第一发送模块504,用于若上传成功,则发送上传成功指令至从属服务器,以便于指示从属服务器删除目标节点中的轻量peer上储存的区块数据。
可选的,第一发送模块504还可以用于:
若区块上传请求所对应的区块数据已上传至区块数据存储系统,则向从属服务器发送已上传响应,以便于从属服务器删除轻量peer的区块数据。
可选的,获取模块503还可以用于:
从全量peer获取区块数据,其中全量peer与区块数据存储系统处于同一局域网。
可选的,获取模块503还可以用于:
从轻量peer获取区块数据,其中轻量peer与从属服务器处于同一局域网。
可选的,获取模块503包括:
解析单元,用于从区块上传请求中解析出区块数据的标识信息;
获取单元,用于基于标识信息,确定区块上传请求对应的区块数据。
可选的,判断模块502包括:
记录单元,用于记录历史区块上传请求;
判断单元,用于判断区块上传请求中包含的标识信息与历史区块上传请求中包含的历史标识信息是否一致。
判定单元,用于若判断一致,则判定区块上传请求所对应的区块数据已上传至区块数据存储系统。
可选的,装置还包括:
第二接收模块,用于接收从属服务器发送的数据查询请求,其中,数据查询请求用于请求查询轻量peer的区块数据;
第二发送模块,用于将数据查询请求发送至区块数据存储系统,以便于区块数据存储系统根据数据查询请求,查找包含区块数据的区块存储数据;
第三接收模块,用于接收区块数据存储系统发送的区块存储数据;
第三发送模块,用于将区块存储数据发送至从属服务器,以便于从属服务器将区块存储数据发送至轻量peer,通过轻量peer从区块存储数据中查询区块数据。
本发明实施例提供的上述技术方案与现有技术相比具有如下优点:本发明采用区块数据存储系统集中存储各节点的历史账本,使各节点仅保存最新的信息数据,可以减少磁盘占有量,也减少了磁盘的冗余信息。
基于相同技术构思,本申请实施例还提供了一种数据上传系统,该数据上传系统包括轻量peer、从属服务器、服务器和区块数据存储 系统,其中,
轻量peer,用于存储区块数据;
服务器,用于接收区块上传请求并将区块数据发送至区块数据存储系统,以及在上传结束后发送上传成功指令至从属服务器;
区块数据存储系统,用于接收服务器发送的区块数据,并将上传结束信息发送至服务器;
从属服务器,用于从服务器接收上传成功指令并删除轻量peer的区块数据。
基于相同的技术构思,本发明实施例还提供了一种电子设备,如图6所示,包括处理器601、通信接口602、存储器603和通信总线604,其中,处理器601,通信接口602,存储器603通过通信总线604完成相互间的通信,
存储器603,用于存放计算机程序;
处理器601,用于执行存储器603上所存放的程序时,实现数据的上传方法步骤。
上述电子设备提到的通信总线可以是外设部件互连标准(Peripheral Component Interconnect,PCI)总线或扩展工业标准结构(Extended Industry Standard Architecture,EISA)总线等。该通信总线可以分为地址总线、数据总线、控制总线等。为便于表示,图中仅用一条粗线表示,但并不表示仅有一根总线或一种类型的总线。
通信接口用于上述电子设备与其他设备之间的通信。
存储器可以包括随机存取存储器(Random Access Memory,RAM),也可以包括非易失性存储器(Non-Volatile Memory,NVM),例如至少一个磁盘存储器。可选的,存储器还可以是至少一个位于远 离前述处理器的存储装置。
上述的处理器可以是通用处理器,包括中央处理器(Central Processing Unit,CPU)、网络处理器(Network Processor,NP)等;还可以是数字信号处理器(Digital Signal Processing,DSP)、专用集成电路(Application Specific Integrated Circuit,ASIC)、现场可编程门阵列(Field-Programmable Gate Array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。
在本发明提供的又一实施例中,还提供了一种计算机可读存储介质,该计算机可读存储介质内存储有计算机程序,所述计算机程序被处理器执行时实现上述任一数据上传的方法的步骤。
在本发明提供的又一实施例中,还提供了一种包含指令的计算机程序产品,当其在计算机上运行时,使得计算机执行上述实施例中任一数据上传的方法。
在上述实施例中,可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时,可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时,全部或部分地产生按照本发明实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中,或者从一个计算机可读存储介质向另一个计算机可读存储介质传输,例如,所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可 用介质可以是磁性介质,(例如,软盘、硬盘、磁带)、光介质(例如,DVD)、或者半导体介质(例如固态硬盘Solid State Disk(SSD))等。
需要说明的是,在本文中,诸如“第一”和“第二”等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来,而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且,术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含,从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素,而且还包括没有明确列出的其他要素,或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下,由语句“包括一个……”限定的要素,并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
以上所述仅是本申请的具体实施方式,使本领域技术人员能够理解或实现本申请。对这些实施例的多种修改对本领域的技术人员来说将是显而易见的,本文中所定义的一般原理可以在不脱离本申请的精神或范围的情况下,在其它实施例中实现。因此,本申请将不会被限制于本文所示的这些实施例,而是要符合与本文所申请的原理和新颖特点相一致的最宽的范围。

Claims (11)

  1. 一种数据的上传方法,其特征在于,包括:
    接收目标节点的从属服务器发送的区块上传请求;
    判断所述区块上传请求所对应的区块数据是否已上传至区块数据存储系统;
    若所述区块上传请求所对应的区块数据未上传至所述区块数据存储系统,则获取所述区块数据,并将所述区块数据上传至所述区块数据存储系统中的存储空间;
    若上传成功,则发送上传成功指令至所述从属服务器,以便于指示所述从属服务器删除所述目标节点中的轻量peer上储存的所述区块数据。
  2. 根据权利要求1所述的方法,其特征在于,所述方法还包括:
    若所述区块上传请求所对应的区块数据已上传至所述区块数据存储系统,则向所述从属服务器发送已上传响应,以便于所述从属服务器删除所述轻量peer的区块数据。
  3. 根据权利要求1所述的方法,其特征在于,所述获取所述区块数据包括:
    从全量peer获取所述区块数据,其中所述全量peer与所述区块数据存储系统处于同一局域网。
  4. 根据权利要求1所述的方法,其特征在于,所述获取所述区块数据包括:
    从所述轻量peer获取所述区块数据,其中所述轻量peer与所述 从属服务器处于同一局域网。
  5. 根据权利要求1所述的方法,其特征在于,所述方法还包括:
    从所述区块上传请求中解析出所述区块数据的标识信息;
    基于所述标识信息,确定所述区块上传请求对应的所述区块数据。
  6. 根据权利要求5所述的方法,其特征在于,所述判断所述区块上传请求所对应的区块数据是否已上传至区块数据存储系统包括:
    记录历史区块上传请求;
    判断所述区块上传请求中包含的所述标识信息与所述历史区块上传请求中包含的历史标识信息是否一致;
    若一致,则判定所述区块上传请求所对应的区块数据已上传至所述区块数据存储系统。
  7. 根据权利要求1所述的方法,其特征在于,所述方法还包括:
    接收所述从属服务器发送的数据查询请求,其中,所述数据查询请求用于请求查询所述轻量peer的区块数据;
    将所述数据查询请求发送至所述区块数据存储系统,以便于所述区块数据存储系统根据所述数据查询请求,查找包含所述区块数据的区块存储数据;
    接收所述区块数据存储系统发送的所述区块存储数据;
    将所述区块存储数据发送至所述从属服务器,以便于所述从属服务器将所述区块存储数据发送至所述轻量peer,通过所述轻量peer从所述区块存储数据中查询所述区块数据。
  8. 一种数据上传系统,其特征在于,包括:
    轻量peer,用于存储区块数据;
    服务器,用于接收区块上传请求并将所述区块数据发送至区块数据存储系统,以及在上传结束后发送上传成功指令至从属服务器;
    区块数据存储系统,用于接收所述服务器发送的所述区块数据,并将上传结束信息发送至所述服务器;
    从属服务器,用于从所述服务器接收所述上传成功指令并删除所述轻量peer的区块数据。
  9. 一种数据的上传装置,其特征在于,包括:
    接收模块,用于接收目标节点的从属服务器发送的区块上传请求;
    判断模块,用于判断所述区块上传请求所对应的区块数据是否已上传至区块数据存储系统;
    获取模块,用于若所述区块上传请求所对应的区块数据未上传至所述区块数据存储系统,则获取所述区块数据,并将所述区块数据上传至所述区块数据存储系统中的存储空间;
    发送模块,用于若上传成功,则发送上传成功指令至所述从属服务器,以便于指示所述从属服务器删除所述目标节点中的轻量peer上储存的所述区块数据。
  10. 一种电子设备,包括存储器、处理器,所述存储器中存储有可在所述处理器上运行的计算机程序,其特征在于,所述处理器执行所述计算机程序时实现上述权利要求1至7任一项所述的方法的步骤。
  11. 一种具有处理器可执行的非易失的程序代码的计算机可读介质,其特征在于,所述程序代码使所述处理器执行所述权利要求1-7任一方法。
PCT/CN2020/090654 2019-10-30 2020-05-15 一种数据的上传方法、系统、装置及电子设备 WO2021082401A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2022519757A JP2022550401A (ja) 2019-10-30 2020-05-15 データのアップロード方法、システム、装置及び電子機器
EP20883303.8A EP4024815A4 (en) 2019-10-30 2020-05-15 METHOD, SYSTEM AND DEVICE FOR UPLOADING DATA AND ELECTRONIC DEVICE

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911048190.3 2019-10-30
CN201911048190.3A CN110958300B (zh) 2019-10-30 2019-10-30 一种数据的上传方法、系统、装置、电子设备和计算机可读介质

Publications (1)

Publication Number Publication Date
WO2021082401A1 true WO2021082401A1 (zh) 2021-05-06

Family

ID=69975831

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/090654 WO2021082401A1 (zh) 2019-10-30 2020-05-15 一种数据的上传方法、系统、装置及电子设备

Country Status (4)

Country Link
EP (1) EP4024815A4 (zh)
JP (1) JP2022550401A (zh)
CN (1) CN110958300B (zh)
WO (1) WO2021082401A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114925406A (zh) * 2022-06-01 2022-08-19 北京百度网讯科技有限公司 数据校验方法、装置及计算机程序产品

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110958300B (zh) * 2019-10-30 2022-06-07 京东科技信息技术有限公司 一种数据的上传方法、系统、装置、电子设备和计算机可读介质
CN111600720B (zh) * 2020-05-20 2021-09-28 腾讯科技(深圳)有限公司 基于区块链的数据处理方法、装置、设备及可读存储介质
CN112291358B (zh) * 2020-11-02 2021-03-16 暗链科技(深圳)有限公司 一种分段式区块链的数据关联方法与系统
CN114928622A (zh) * 2022-05-07 2022-08-19 杭州复杂美科技有限公司 多子网区块链网络及其数据存储方法、设备和存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107423426A (zh) * 2017-08-02 2017-12-01 众安信息技术服务有限公司 一种区块链块数据的数据归档方法及电子设备
CN108540566A (zh) * 2018-04-18 2018-09-14 暴风集团股份有限公司 文件上传方法、装置、系统以及客户端和服务器
US20190306176A1 (en) * 2016-10-17 2019-10-03 Arm Limited Blockchain mining using trusted nodes
CN110958300A (zh) * 2019-10-30 2020-04-03 北京海益同展信息科技有限公司 一种数据的上传方法、系统、装置、电子设备和计算机可读介质

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003196129A (ja) * 2001-12-28 2003-07-11 Toshiba Corp ファイル転送システム、ファイル転送装置及びファイル転送方法
JP2007201861A (ja) * 2006-01-27 2007-08-09 Eastman Kodak Co ファイル管理方法
US9559889B1 (en) * 2012-10-31 2017-01-31 Amazon Technologies, Inc. Cache population optimization for storage gateways
CN107770115B (zh) * 2016-08-15 2021-01-05 华为技术有限公司 在对等网络中分发数字内容的方法和系统
US11631077B2 (en) * 2017-01-17 2023-04-18 HashLynx Inc. System for facilitating secure electronic communications between entities and processing resource transfers
CN107332876B (zh) * 2017-05-31 2020-05-08 深圳前海微众银行股份有限公司 区块链状态的同步方法及装置
US10749879B2 (en) * 2017-08-31 2020-08-18 Topia Technology, Inc. Secure decentralized file sharing systems and methods
CN108875411A (zh) * 2018-07-11 2018-11-23 成都理工大学 基于区块链的智能手环数据存储与共享方法
CN109299336B (zh) * 2018-09-30 2022-07-01 腾讯科技(深圳)有限公司 数据备份方法、装置、存储介质及计算设备
CN110046894B (zh) * 2019-04-19 2021-11-09 电子科技大学 一种基于纠删码的分组可重构区块链建立方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190306176A1 (en) * 2016-10-17 2019-10-03 Arm Limited Blockchain mining using trusted nodes
CN107423426A (zh) * 2017-08-02 2017-12-01 众安信息技术服务有限公司 一种区块链块数据的数据归档方法及电子设备
CN108540566A (zh) * 2018-04-18 2018-09-14 暴风集团股份有限公司 文件上传方法、装置、系统以及客户端和服务器
CN110958300A (zh) * 2019-10-30 2020-04-03 北京海益同展信息科技有限公司 一种数据的上传方法、系统、装置、电子设备和计算机可读介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4024815A4 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114925406A (zh) * 2022-06-01 2022-08-19 北京百度网讯科技有限公司 数据校验方法、装置及计算机程序产品

Also Published As

Publication number Publication date
EP4024815A4 (en) 2022-08-31
CN110958300B (zh) 2022-06-07
JP2022550401A (ja) 2022-12-01
EP4024815A1 (en) 2022-07-06
CN110958300A (zh) 2020-04-03

Similar Documents

Publication Publication Date Title
WO2021082401A1 (zh) 一种数据的上传方法、系统、装置及电子设备
KR102476531B1 (ko) 분산 시스템을 위한 데이터 동기화 방법 및 장치, 매체, 및 전자 디바이스
US10803016B2 (en) Predictive models of file access patterns by application and file type
US9792344B2 (en) Asynchronous namespace maintenance
CN109783438B (zh) 基于librados的分布式NFS系统及其构建方法
US9699017B1 (en) Dynamic utilization of bandwidth for a quorum-based distributed storage system
US20150215405A1 (en) Methods of managing and storing distributed files based on information-centric network
CN113168404B (zh) 用于在分布式数据库系统中复制数据的系统和方法
CN111475483B (zh) 数据库迁移方法、装置及计算设备
US8805849B1 (en) Enabling use of analytic functions for distributed storage system data
CN105025053A (zh) 基于云存储技术的分布式文件的上传方法及其系统
WO2008131653A1 (fr) Système et procédé pour la réalisation d'une mémoire d'abonnement à un réseau et serveur d'abonnement
WO2015039569A1 (zh) 副本存储装置及副本存储方法
CN108540510B (zh) 一种云主机创建方法、装置及云服务系统
WO2013172405A1 (ja) ストレージシステムおよびデータアクセス方法
US11144407B1 (en) Synchronous database geo-mirroring using delayed visibility write operations
CN111225003B (zh) 一种nfs节点配置方法和装置
US10545667B1 (en) Dynamic data partitioning for stateless request routing
CN112148206A (zh) 一种数据读写方法、装置、电子设备及介质
CN105207993A (zh) 一种cdn中数据的访问、调度方法和系统
US9432238B2 (en) Communicating large amounts of data over a network with improved efficiency
WO2023093608A1 (zh) 一种自动化分布式云存储调度交互方法、装置及设备
US20240134549A1 (en) Method and system for data uploading, and computer-readable storage medium
CN110196881B (zh) 一种基于区块链的数据读写方法及区块链网络结构
US11386072B1 (en) Automatic consistency for database write forwarding

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20883303

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022519757

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2020883303

Country of ref document: EP

Effective date: 20220328

WWE Wipo information: entry into national phase

Ref document number: 17769511

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE