CN113987062A

CN113987062A - Data uplink storage method, system, device and storage medium

Info

Publication number: CN113987062A
Application number: CN202111080869.8A
Authority: CN
Inventors: 刘祥涛
Original assignee: Shenzhen Fandora Information Technology Co ltd
Current assignee: Shenzhen Fandora Information Technology Co ltd
Priority date: 2021-09-15
Filing date: 2021-09-15
Publication date: 2022-01-28

Abstract

The application discloses a data uplink storage method, a system, a device and a storage medium, which are used for distributing data and performing uplink storage, improving the safety of data storage and providing support for data asset and asset value. The method comprises the following steps: acquiring target data, and analyzing the target data to judge whether the target data is stored through a block chain; if so, generating a data identifier uniquely corresponding to the target data; generating an uplink request message according to the data identifier and the target data, and writing the uplink request message into a message queue; analyzing an uplink request message in the message queue, and uploading the data identifier and the target data to a block chain; and acquiring uplink information uniquely corresponding to the data identifier from the block chain, and storing the uplink information.

Description

Data uplink storage method, system, device and storage medium

Technical Field

The present application relates to the field of data processing, and in particular, to a method, a system, an apparatus, and a storage medium for storing data uplink.

Background

The internet development has experienced 3 times of web1.0, web2.0 and web3.0, and in the web1.0 time, a central server mainly produces data contents and establishes a link relationship through an HTTP protocol for a user to access, and typical representatives thereof include yahoo.com, sohu.com and the like; in the web2.0 era, users mainly produce data and share the data with each other, and a producer of the data is not an owner of the data, that is, the data produced by the users does not become data assets of the users, and representatives of the data assets include Facebook, Amazon, WeChat, fast hand, tremble, Taobao, Jingdong and the like; in the era of web3.0, data is also produced by users, and is stored in a blockchain manner, which is typically represented by NFT (Non-homogeneous Tokens), and is generally generated and bound to users in a manner based on a highly secure blockchain contract, thereby becoming data assets of the users.

Internet is a mainstream nowadays in web2.0, and in the era of rapid growth of web3.0, users can generate massive user data assets (including user behavior data, virtual asset data, transaction data, article data and the like) in information systems of games, social networks, e-commerce and the like, most of the data are generated by relying on the web2.0 information system, the information systems are generally operated by a certain central organization and stored on a central server, and the problems of data security such as data being easy to be tampered, data change being incapable of being traced, data producers having no data ownership, data single point failure and the like exist.

The rapid growth of Web3.0 is mainly driven by technical elements of a blockchain, the blockchain has the characteristics of decentralization, difficulty in tampering, traceability, user data capitalization, consensus and high security brought by distributed nodes and the like, and the data security problem of a Web2.0 information system can be solved by chaining and storing the user data assets; generally, a uplink method provided in the prior art realizes uplink and receives uplink results by directly calling a function and creating a channel, but the method simply realizes uplink, can complete a task under the condition that the uplink timeliness requirement is not very high, and is difficult to cope with the scenes of high TPS (Transactions Per Second, TPS) and high timeliness requirement; meanwhile, the blockchain itself is technology-driven, and various security mechanisms are used to ensure the security and consensus of data, and the uplink data is difficult to be popularized and used for users and most information systems without the support of middleware technology.

Disclosure of Invention

In order to solve the above technical problem, the present application provides a data uplink storage method, system, apparatus and storage medium.

A first aspect of the present application provides a data uplink storage method, including:

acquiring target data, and analyzing the target data to judge whether the target data is stored through a block chain;

if so, generating a data identifier uniquely corresponding to the target data;

generating an uplink request message according to the data identifier and the target data, and writing the uplink request message into a message queue;

analyzing an uplink request message in the message queue, and uploading the data identifier and the target data to a block chain;

and acquiring uplink information uniquely corresponding to the data identifier from the block chain, and storing the uplink information.

Optionally, the storing the uplink information includes:

generating an uplink response message according to the uplink information, and returning the uplink response message to the message queue;

and analyzing the uplink response message in the message queue and storing the uplink information.

Optionally, after determining whether to store the target data through the blockchain, if it is determined that the target data is not stored through the blockchain, the method further includes:

and storing the target data to a database.

Optionally, the method further includes:

and carrying out validity verification on the target data, and analyzing the target data if the target data passes the validity verification.

Optionally, the verifying the validity of the target data includes:

and verifying the hash value of the target data.

Optionally, the verifying the validity of the target data includes:

and verifying the timestamp corresponding to the target data.

Optionally, before the verifying the validity of the target data, the method further includes:

and verifying the user identity of the calling user corresponding to the target data, and if the verification is passed, verifying the legality of the target data.

Optionally, before the uploading the data identifier and the target data to the blockchain, the method further includes:

and performing data summary verification on the uplink request message, and uploading the data identifier and the target data to a block chain if the verification is passed.

A second aspect of the present application provides a split-flow relay station system for data uplink storage, the system comprising:

the system comprises a chain divider, a block chain and a data processing unit, wherein the chain divider is used for acquiring target data and analyzing the target data to judge whether the target data is stored through the block chain;

if so, the data link shunt generates a data identifier uniquely corresponding to the target data;

the data transmission device comprises a data queue, a data operation middleware and a data transmission module, wherein the data queue comprises a plurality of data identifiers and a target data block chain;

the data identifier is used for identifying a block chain, and the data identifier is used for identifying a data identifier of the block chain.

A third aspect of the present application provides a data uplink storage apparatus, the apparatus comprising:

the device comprises a processor, a memory, an input and output unit and a bus;

the processor is connected with the memory, the input and output unit and the bus;

the memory holds a program that the processor calls to perform the method of any of the first aspect and the first aspect.

A fourth aspect of the present application provides a computer readable storage medium having a program stored thereon, which when executed on a computer performs the method of any one of the first aspect and the first aspect.

According to the technical scheme, the method has the following advantages:

according to the method, the target data can be analyzed to judge whether the target data needs to be stored through a block chain, the traditional centralized user behavior data, the virtual asset data, the article data and other data (namely, the data, namely, the assets) can be chain-stored, the data is decentralized, traceable, not easy to tamper and cannot be repudiated, and support is provided for data asset realization and asset valuation. In addition, in the method provided by the present application, the target data and the data identifier are uplinked through the message queue, which can provide support for high TPS (Transactions Per Second).

Drawings

In order to more clearly illustrate the technical solutions in the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

Fig. 1 is a flowchart illustrating an embodiment of a data uplink storage method according to the present application;

FIG. 2 is a flowchart illustrating another embodiment of a method for storing data uplink provided herein;

fig. 3 is a schematic structural diagram of a split-flow relay system for data uplink storage according to an embodiment of the present invention;

fig. 4 is a schematic structural diagram of a split-flow relay system for uplink data storage according to another embodiment of the present invention;

fig. 5 is a schematic structural diagram of an embodiment of a data uplink storage device according to the present application.

Detailed Description

When data is stored, data is generally stored in a traditional database in a unified manner or in a block chain, the traditional database is generally controlled by a certain central organization, and the problems that the data is easy to be tampered, the data change cannot be traced, a data producer does not have data ownership, a data single point fault and the like exist, so that more and more data needs to be stored in an uplink manner.

By utilizing the characteristics of decentralized block chain, difficult data tampering, traceability, ownership and data binding and the like, the uplink data is stored to ensure the multi-directional security of the data. The blockchain can be understood as a distributed database, the uplink method provided in the prior art generally implements uplink and uplink receiving results by directly calling a function and creating a channel, but the method simply implements uplink, can complete a task when the uplink timeliness requirement is not very high, and is difficult to cope with scenarios with high TPS (Transactions Per Second, TPS) and high timeliness requirements; meanwhile, the blockchain itself is technology-driven, and various security mechanisms are used to ensure the security and consensus of data, and the uplink data is difficult to be popularized and used for users and most information systems without the support of middleware technology.

Based on the above, the application provides a data uplink storage method, which solves the problem of low TPS (secure data protocol) of the traditional data uplink method, and meanwhile, simplifies the data uplink mode through a middleware technology and a shunting uplink interface, breaks through the popularization and application bottlenecks caused by various safety technical mechanisms, obtains multi-directional safety of data uplink storage, and provides support for data capitalization and asset valuation.

In order to provide a clearer understanding of what is provided herein, the following discussion is intended to explain terms that may be used in the present application:

winding: uplink in this application refers to uploading data to a blockchain;

shunting: in the application, data are stored in different modes according to the characteristics of the data;

shunting middle platform system: the data link distribution center System (SFMOMS, hereinafter referred to as "data link distribution center" or "distribution center") provides data link center services for traditional centralized systems (e.g., games, social networks, e-commerce systems, internet of things systems, etc.).

Fig. 1 is a flowchart illustrating an embodiment of a data uplink storage method provided in the present application, where the data uplink storage method includes:

101. acquiring target data, and analyzing the target data to judge whether the target data is stored through a block chain; if yes, go to step 102, otherwise go to step 106;

in practical application, a user can implement shunt storage on target data through the method provided by the application, when the data needs to be stored, the user initiates call of an interface by sending an interface call request, responds to the interface call request, a server obtains the target data through a shunt interface, and analyzes the target data, wherein the target data can include user identity data, data, a hash value of the data, a signature of the hash value, and the like. And if no uplink is needed, storing the target data in a conventional database.

102. Generating a data identifier uniquely corresponding to the target data;

if the target data needs to be uplinked, the server first assigns a data identifier to the target data, where the data identifier may be a globally unique number chain ID (Identity number) for identifying the target data, and in the present application, the target data may be a piece of data or a batch of data, and correspondingly, the data identifier may be for a piece of data or a batch of data.

Optionally, the data identifier may be generated by a central server program or a block chain contract, so as to obtain an NFT attribute (NFT, Non-variant Token) of the data asset; the former method has the advantages of higher generation speed, no need of consuming Gas fee and the like, and the latter method has the advantages of data tamper resistance, high safety and the like.

103. Generating an uplink request message according to the data identifier and the target data, and writing the uplink request message into a message queue;

in the application, the target data is uplinked through the message queue, specifically, an uplink request message is generated according to the data identifier and the target data, and the uplink request message is cached in a first-in first-out message queue. Message buffering and second-level message transmission services can be provided through the message queue, the long transaction is decomposed into a plurality of short transactions in a serial connection mode and synchronous asynchronization is achieved, and meanwhile concurrency performance of the system is expanded to the TPS10 ten thousand level is supported.

104. Analyzing an uplink request message in the message queue, and uploading the data identifier and the target data to a block chain;

the uplink request message in the consumption message queue is specifically to analyze the uplink request message, obtain a data identifier and corresponding target data, and further upload the data identifier and the target data to a block chain.

Meanwhile, when the target data is uplink-linked, a proper chain can be selected from multiple chains or multiple chains are combined for simultaneous uplink (for example, the TPS of the Solana public chain exceeds 5 ten thousand, and the time delay is less than 1s) according to the TPS and timeliness index requirements of the user, so that data cross-link and data multi-chain simultaneous storage are realized.

105. And obtaining uplink information uniquely corresponding to the data identifier from the block chain, and storing the uplink information.

After the server uploads the data id and the target data to the blockchain, uplink information is obtained, where the uplink information may include an uplink result returned by the blockchain, and the uplink result indicates whether the target data and the data id have been uploaded on the blockchain, for example, the uplink result may include an uplink failure and an uplink success. The uplink information may further include metadata of uplink results, and the metadata of uplink results may include: the data identifier, the link identity, whether the uplink is successful, the success/failure time, the failure reason and the like, before uplink, the link identity can be acquired for the target data, a legal identity is provided for the data uplink operation, and the normal operation of the uplink is ensured.

In another possible implementation manner, the storing the uplink information may include:

and generating an uplink response message according to the uplink information, returning the uplink response message to the message queue, analyzing the uplink response message in the message queue, and storing the uplink information. After the uplink, obtaining the uplink information, generating an uplink response message according to the uplink information, caching the uplink response message into a message queue so as to respond to the uplink request message corresponding to the data identifier, and further analyzing the uplink response message in the message queue so as to obtain the uplink information corresponding to the data identifier, thereby storing the uplink information into a warehouse.

106. And storing the target data to a database.

In practical applications, if it is determined that the target data is not stored through the blockchain, the target data may also be stored in other manners, and this embodiment is exemplified by storing the target data through a conventional database.

In practical application, in order to ensure the security of the user identity and the integrity of data called by the interface, the user identity of the corresponding calling user, the hash value of the data, and the like may be verified. Specific embodiments are described in detail below with reference to the accompanying drawings.

Referring to fig. 2, fig. 2 is a flowchart illustrating an embodiment of a data uplink storage method provided in the present application, where the embodiment includes:

201. acquiring target data;

the target data can be obtained from the interface parameters of the splitter interface in the splitting intermediate station system for calling uplink storage of data splitting in the application.

202. Verifying the user identity of the calling user corresponding to the target data, if the user identity passes the verification, performing validity verification on the target data, if the user identity passes the verification, executing the step 203, and if the user identity does not pass the verification, executing the step 212;

in practical application, a calling user can send an interface calling request to call an interface to perform shunting uplink storage on target data, and when the calling user calls the interface calling request, a server can analyze the interface calling request to obtain a user identity of the calling user and verify the user identity, for example, the user identity is verified based on the user identity data and a public and private key + digital signature mechanism, and if the verification is passed, the integrity of the target data can be continuously verified.

203. Verifying the legality of the target data, if the target data passes the verification, executing a step 204, and if the target data does not pass the verification, executing a step 212;

in order to ensure the integrity of the target data, the data field may be verified by using an SHA3-256 (e.g., Keccak-256) algorithm, specifically, a hash value of the data field in the target data is obtained, the hash value is compared with the sent hash value, if the two hash values are consistent, the target data may be considered as not being tampered, and after that, the signature verification may be continued by using the public key of the calling user. Further, to ensure that the data is not attacked by replay, a timestamp-based authentication mechanism may be added, for example, it is required to ensure that an interface call request of a message within plus or minus N seconds (for example, N may be 3 to 10 seconds) is a legal call request.

204. Analyzing the target data to determine whether the target data is stored through a block chain, if so, executing step 205, and if not, executing step 206;

205. generating a data identifier uniquely corresponding to the target data;

206. storing the target data to a database;

207. generating an uplink request message according to the data identifier and the target data, and writing the uplink request message into a message queue;

208. analyzing uplink request messages in the message queue;

the contents of step 205 to step 208 in this embodiment are similar to those in the above embodiments, and are not described herein again.

209. Performing data summary verification on the uplink request message, if the verification is passed, executing step 210, and if the verification is not passed, executing step 212;

and performing data abstract verification on the uplink request message, and performing uplink processing on the target data and the data identifier after the verification is passed.

A method of data digest verification may be: analyzing the original hash value of the obtained target data from the uplink request message, obtaining the corresponding hash value for the target data again, comparing the obtained hash value with the original hash value in the uplink request message, if the two hash values are consistent, passing the verification of the data abstract of the uplink request message, if the two hash values are not consistent, failing the verification, and generating response information after determining that the verification fails, which is detailed in step 212.

210. Uploading the data identification and the target data to a block chain;

211. and obtaining uplink information uniquely corresponding to the data identifier from the block chain, and storing the uplink information.

212. Response information is generated.

If the verification of any verification link is not passed, corresponding wrong response information can be generated, and then a client program of the system is called to process the response information.

In this embodiment, the security of the user identity and the data integrity called by the interface can be ensured by, for example, a SHA-3 hash algorithm and a private key signature algorithm with high security level, the interface is ensured not to be attacked by message replay through timestamp verification, a data desensitization mechanism can be adopted to desensitize the user private data, and the core data is subjected to data signature and encrypted storage to ensure the data integrity and confidentiality; the above mechanism ensures the interface security and the storage security of high security level.

The above embodiments describe the data uplink storage method provided in the present application in detail, and the following describes the system, apparatus and storage medium for offloading data uplink storage provided in the present application in detail.

Referring to fig. 3, fig. 3 is a schematic structural diagram of an embodiment of a split intermediate station system for uplink data storage provided in the present application, where the embodiment includes:

the chain divider 301 is configured to acquire target data and analyze the target data to determine whether to store the target data through a block chain;

if so, the data link splitter 301 generates a data identifier uniquely corresponding to the target data;

the data link splitter 301 generates an uplink request message according to the data identifier and the target data, and writes the uplink request message into a message queue;

the digital chain operation middleware 302 is configured to parse the uplink request message in the message queue, and upload the data identifier and the target data to the block chain;

the daisy-chain operation middleware 302 is further configured to obtain uplink information uniquely corresponding to the data identifier from the blockchain, and the daisy-chain splitter 301 stores the uplink information in the database.

Optionally, the several-chain operation middleware 302 is specifically configured to:

chain splitter 301 is specifically configured to:

and analyzing the uplink response message in the message queue and storing uplink information.

Chain splitter 301 is also used to:

storing target data in a database

Referring to fig. 4, fig. 4 is a schematic structural diagram of a split-flow relay system for uplink data storage according to another embodiment of the present application;

in fig. 4, the chain splitter 301 includes: the system comprises an identity authentication module 3011, a data signature verification module 3012, a data chain splitting module 3013, and a splitter interface 3014, where the data chain splitting module 3013 is configured to obtain target data through the splitter interface 3014, and analyze the target data to determine whether the target data is stored through a block chain. The digital chain operation middleware 302 includes a consumption message module 3021, a data digest verification module 3022, and a chain identity obtaining module 3023, where the consumption message module 3021 is configured to parse the uplink request message in the message queue, and the chain identity obtaining module 3023 is configured to obtain a chain identity of the target data.

Optionally, the identity authentication module 3011 is configured to:

and verifying the user identity of the calling user corresponding to the target data.

Optionally, the data signature verification module 3012 is configured to:

and carrying out validity verification on the target data.

Optionally, the data signature verification module 3012 is specifically configured to:

and verifying the hash value of the target data.

and verifying the timestamp corresponding to the target data.

Optionally, the data digest verification module 3022 is configured to:

and performing data summary verification on the uplink request message, and uploading the data identifier and the target data to the block chain if the verification is passed.

Optionally, a method for verifying a data digest may be: the data chain offload module 3013 generates a hash value of the target data (the hash value may be obtained by using SHA-3 algorithm), and sends the target data and the hash value to the message queue during sending, after the consumption message module 3021 parses and obtains the data and the hash value, the data digest verification module 3022 performs re-hashing on the target data to obtain the hash value of the target data again, and compares the re-obtained hash value with the original hash value sent, if the two hash values are consistent, the data digest verification is passed, and if the two hash values are not consistent, the data digest verification is not passed.

The present application further provides a data uplink storage device, comprising:

a processor 501, a memory 502, an input/output unit 503, and a bus 504;

the processor 501 is connected with the memory 502, the input/output unit 503 and the bus 504;

the memory 502 holds a program that the processor 501 invokes to perform any of the above data cochain storage methods.

The present application also relates to a computer readable storage medium having a program stored thereon, wherein the program, when executed on a computer, causes the computer to perform any of the above data uplink storage methods.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and the like.

Claims

1. A method for data uplink storage, the method comprising:

if so, generating a data identifier uniquely corresponding to the target data;

2. The method of claim 1, wherein the storing the uplink information comprises:

3. The method of claim 1 wherein after determining whether to store the target data via a blockchain, if it is determined that the target data is not stored via the blockchain, the method further comprises:

and storing the target data to a database.

4. The method of claim 1, wherein prior to said analyzing the target data, the method further comprises:

5. The method of claim 4 wherein the validating the target data for validity comprises:

and verifying the hash value of the target data.

6. The method of claim 4 wherein the validating the target data for validity comprises:

and verifying the timestamp corresponding to the target data.

7. The method of claim 4 wherein prior to said validating said target data for validity, said method further comprises:

8. The method of any of claims 1-7, wherein prior to said uploading said data identity and said target data block chain, said method further comprises:

9. A split-flow relay system for data uplink storage, the system comprising:

10. An apparatus for data uplink storage, the apparatus comprising:

the device comprises a processor, a memory, an input and output unit and a bus;

the memory holds a program that the processor calls to perform the method of any of claims 1 to 8.

11. A computer-readable storage medium having a program stored thereon, which when executed on a computer performs the method of any one of claims 1 to 8.