CN115190136B - Data storage method, data transmission method and computing equipment - Google Patents

Data storage method, data transmission method and computing equipment Download PDF

Info

Publication number
CN115190136B
CN115190136B CN202210800036.2A CN202210800036A CN115190136B CN 115190136 B CN115190136 B CN 115190136B CN 202210800036 A CN202210800036 A CN 202210800036A CN 115190136 B CN115190136 B CN 115190136B
Authority
CN
China
Prior art keywords
data
node
stored
data block
current
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210800036.2A
Other languages
Chinese (zh)
Other versions
CN115190136A (en
Inventor
卢桢
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Uniontech Software Technology Co Ltd
Original Assignee
Uniontech Software Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Uniontech Software Technology Co Ltd filed Critical Uniontech Software Technology Co Ltd
Priority to CN202210800036.2A priority Critical patent/CN115190136B/en
Publication of CN115190136A publication Critical patent/CN115190136A/en
Application granted granted Critical
Publication of CN115190136B publication Critical patent/CN115190136B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1097Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00Network architectures or network communication protocols for network security
    • H04L63/04Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
    • H04L63/0428Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00Network architectures or network communication protocols for network security
    • H04L63/12Applying verification of the received information
    • H04L63/123Applying verification of the received information received data contents, e.g. message integrity
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/06Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols the encryption apparatus using shift registers or memories for block-wise or stream coding, e.g. DES systems or RC4; Hash functions; Pseudorandom sequence generators
    • H04L9/0643Hash functions, e.g. MD5, SHA, HMAC or f9 MAC

Abstract

The invention discloses a data storage method, a data transmission method and computing equipment, wherein the data storage method comprises the following steps: the method comprises the steps of obtaining data to be stored, splitting a data object to be stored into a plurality of data blocks, assigning a unique name to each data block, and storing each data block into nodes of a preset storage structure, wherein each node in the preset storage structure is correspondingly a data server, each node is assigned with a unique node ID, the maximum storage capacity of each node is the same, all leaf nodes except the rightmost leaf node store data with the maximum capacity, each node in the preset storage structure is used for storing data of a first type, the data of the first type comprises one or more of father node IDs, directory names, data block names and node IDs, and the directory names of the data blocks stored by the current node corresponding to child nodes are stored, so that when node information is modified, only node information and information of related nodes need to be modified, and the probability of node conflict is greatly reduced.

Description

Data storage method, data transmission method and computing equipment
The present application is a divisional application of patent application 2021104296870 filed on 21/04/2021.
Technical Field
The present invention relates to the field of cloud storage technologies, and in particular, to a data storage method, a data transmission method, a computing device, and a storage medium.
Background
With the continuous development of socioeconomic performance, data is being exploded and increased, each industry faces a huge data file storage problem, and enterprises want to rely on own power to store data and maintain the data at high cost, and professional team is needed. Thus, cloud storage is an option for each enterprise to store data.
Cloud storage is a model of online data storage services provided by network providers, that is, storing data on multiple virtual servers hosted by third parties. The third party operates a large data center, and the user needs to purchase the storage space operated by the third party and access the storage space through the application program interface to store data.
Cloud storage is implemented based on a cloud storage structure. Currently, cloud storage structures mainly include a central storage structure and a directory storage structure. The central storage structure, i.e. the directory entries of the files are all stored in one node (the node is a data server), and the directory storage structure, i.e. each node stores one directory, including the directory entries of all files under the directory. However, the cloud storage structure described above modifies the central storage structure when files are changed or deleted, and conflicts easily occur when multiple clients operate the files simultaneously.
To this end, the invention provides a data storage method capable of avoiding collision.
Disclosure of Invention
To this end, the present invention provides a data storage method in an effort to solve or at least alleviate the above-identified problems.
According to one aspect of the present invention, there is provided a data storage method, for execution in a computing device, the method comprising the steps of:
acquiring data to be stored;
splitting a data object to be stored into a plurality of data blocks, and distributing a unique name for each data block;
storing each data block into a node of a preset storage structure, wherein each node in the preset storage structure corresponds to a data server, each node is assigned with a unique node ID, the maximum storage capacity of each node is the same, all leaf nodes except the rightmost leaf node store data with the maximum capacity, each node in the preset storage structure is used for storing first-class data, the first-class data comprises one or more of a father node ID, a catalog name, the data block name and the node ID, and the catalog name is the catalog name of the data block stored by the child node corresponding to the current node.
Optionally, the step of splitting the data object to be stored into a plurality of data blocks comprises:
Determining the current offset of a data pointer of a data object to be stored;
reading data of a first preset value from the data to be stored by taking the determined current offset of the data pointer as a starting position, and taking the data as a data block;
updating the current offset of the data pointer to be the sum of the determined current offset of the data pointer and a first preset value.
Optionally, with the determined current offset of the data pointer as a starting position, the step of reading the data of the first preset value from the data to be stored as a data block includes:
judging whether the product of the number of times of currently reading the data to be stored and the first preset value is smaller than the size of the data to be stored, if yes, reading the data of the first preset value by taking the determined current offset of the data pointer as a starting position, if no, determining the number of the data to be stored to be read according to the following formula, and taking the determined current offset of the data pointer as the starting position to read the data of the second value as the second value:
L=m-(a-1)*b
wherein L is the number of the data to be stored, m is the size of the storage space occupied by the data to be stored, a is the number of times of currently reading the data to be stored, and b is a first preset value;
the read data is used as a data block.
Optionally, the step of saving each data block into a node of the predetermined storage structure comprises:
judging whether the current node has a child node or not, if so, storing the parent node ID, the current node ID and the directory name of the current node, and if not, storing the parent node ID, the data block name and the current node ID of the current node.
Optionally, the method comprises the steps of:
judging whether each node in the preset storage structure has a left node or not, and if so, storing the left node ID of the node in the current node.
Optionally, the method further comprises the steps of:
setting version information for each data block, wherein the versions of the data blocks are distinguished through the version information of the data blocks;
the step of judging whether the current node has a child node comprises the following steps:
judging whether the current node has a child node or not, if so, storing the parent node ID, the current node ID, the left node ID, the directory name and the data block version information in the current node, and if not, storing the parent node ID, the current node ID, the left node ID, the data block name and the data block version information in the current node.
Optionally, the method further comprises the steps of:
responding to the operation of replacing the data block in one leaf node of the preset storage structure by the target data block, judging whether the data block name of the target data block is the same as the data block name in the current node, if not, not replacing, and if so, judging whether the data block version information of the target data block is the same as the data block version information in the current node;
If the data block version information of the target data block is the same as the data block version information in the current node, replacing the data block in the current node with the target data block, and if the data block version information is different from the data block version information in the current node, not replacing.
Optionally, the method further comprises the steps of:
in response to a data block name operation stored in one node of the renaming predefined storage structure, the data block name stored in the current node is updated to be the renamed name.
Optionally, the first preset value is equal to a maximum storage capacity of each node in the predetermined storage structure.
According to another aspect of the present invention, there is provided a data transmission method, performed in a computing device, the method comprising:
acquiring data to be transmitted, wherein the data to be transmitted is data stored by each node in a preset storage structure and stored according to the data storage method;
carrying out hash operation on data to be transmitted to obtain a first hash value;
combining data to be transmitted and the first hash value to serve as target data;
encrypting the target data through a data encryption algorithm to obtain encrypted data;
and sending the encrypted data to a receiving end.
Optionally, the operation steps of the receiving end after receiving the encrypted data are as follows:
Decrypting the encrypted data through a data encryption algorithm to obtain target data;
splitting target data to obtain first data to be transmitted and the first hash value;
carrying out hash operation on the first data to be transmitted to obtain a second hash value;
and judging whether the second hash value is the same as the first hash value, if so, using the data to be transmitted, and if not, not using the data to be transmitted.
Optionally, the data encryption algorithm is an ABE encryption algorithm.
According to one aspect of the present invention, there is provided a computing device comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be adapted to be executed by the at least one processor, the program instructions comprising instructions for performing the method as described above.
According to one aspect of the present invention, there is provided a readable storage medium storing program instructions which, when read and executed by a computing device, cause the computing device to perform a method as described above.
According to the technical scheme of the invention, the data to be stored is stored in the preset storage structure, specifically, each node in the preset storage structure stores the father node ID instead of the father node ID, and the father node ID stores the file catalogue (including all file catalogue items of the file catalogue), so that when the node information is modified, only the information of the node and the information of the related node are required to be modified, the change of the father node is not involved, the probability of collision of the node is greatly reduced, and the modification complexity is also reduced. And the node also stores the version information of the data block, so that an attacker cannot replace the file by different versions of the same file, the integrity of the data is ensured, and the safety of data storage is improved.
In addition, since in the cloud storage structure, all leaf nodes except the rightmost leaf node store data of the maximum capacity (compared with each leaf node storing only part of the data and not storing the data of the maximum capacity), the number of nodes can be reduced, thereby reducing the storage space waste. And because the maximum capacity of the leaf node storage data is the same, when the stored data size is determined, the stored data size of each node is not required to be obtained and summed, the stored data size can be determined by multiplying the node number by 1 and the node maximum capacity and adding the rightmost leaf node data size, which is equivalent to the stored data size calculated by only accessing the rightmost leaf node data size, and the stored data size can be rapidly determined, so that the time consumption is reduced.
The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present invention more readily apparent.
Drawings
To the accomplishment of the foregoing and related ends, certain illustrative aspects are described herein in connection with the following description and the annexed drawings, which set forth the various ways in which the principles disclosed herein may be practiced, and all aspects and equivalents thereof are intended to fall within the scope of the claimed subject matter. The above, as well as additional objects, features, and advantages of the present disclosure will become more apparent from the following detailed description when read in conjunction with the accompanying drawings. Like reference numerals generally refer to like parts or elements throughout the present disclosure.
FIG. 1 illustrates a block diagram of a data storage system 100, according to one embodiment of the invention
FIG. 2 shows a schematic diagram of a computing device 200 according to one embodiment of the invention;
FIG. 3 illustrates a flow chart of a data storage method 300 according to one embodiment of the invention;
FIG. 4 shows a schematic diagram of a predetermined memory structure according to one embodiment of the invention;
FIG. 5 illustrates a schematic diagram of a predetermined storage structure correspondence map according to one embodiment of the present invention; and
fig. 6 shows a flow chart of a data transmission method 600 according to an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
With the continuous development of socioeconomic performance, data is being exploded and increased, each industry faces a huge data file storage problem, and enterprises want to rely on own power to store data and maintain the data at high cost, and professional team is needed. Thus, cloud storage is an option for each enterprise to store data.
Cloud storage is a model of online data storage services provided by network providers, that is, storing data on multiple virtual servers hosted by third parties. The third party operates a large data center, and the user needs to purchase the storage space operated by the third party and access the storage space through the application program interface to store data.
Cloud storage is implemented based on a cloud storage structure. Currently, cloud storage structures mainly include a central storage structure and a directory storage structure. The central storage structure, namely the directory entry of the file is all stored in one node (the node is a data server), the central storage structure is modified when the file is changed or deleted, the conflict is easy to occur when a plurality of clients operate the file at the same time, and the central node of the storage structure can be judged by observing the change of the file after the file is operated, so that node information is exposed. In the directory storage structure, each node stores a directory including directory entries of all files under the directory, and when path searching is performed, all directory nodes are traversed and decrypted, so path searching is slow, similarly, if multiple clients operate files simultaneously, conflicts can occur at a father node, so that only one version of node is reserved, and an orphan node can appear for the clients which are not reserved.
To this end, the invention defines a memory structure defined as follows: the maximum storage capacity of each node in the storage structure is the same, all leaf nodes except the rightmost leaf node in the predetermined storage structure store data of the maximum capacity, and each node in the storage structure stores a parent node ID, a current node ID, a directory name (directory name of a data block stored by a child node corresponding to the current node), a left node ID, and data block version information (different versions of the data block are distinguished), or each node in the storage structure stores a parent node ID, a current node ID, a left node ID, a data block name, and data block version information.
On the basis of defining a storage structure, the invention provides a data storage method capable of avoiding conflict. The method is to store the data to be stored in the storage structure defined above. Specifically, each node in the storage structure stores a parent node ID, instead of the parent node storing a child node ID, so that when a data node is modified, only the information of the node and the information of an associated node are required to be modified, and the change of the parent node is not involved, so that when a plurality of nodes are modified, the probability of collision of the nodes is greatly reduced, and the modification complexity is also reduced.
Since all leaf nodes except the rightmost leaf node store the maximum capacity data (compared to each leaf node storing only a portion of the data and not storing the maximum capacity data), the number of nodes is reduced, thereby reducing the storage space waste. And because the maximum capacity of the leaf node storage data is the same, when the stored data size is determined, the stored data size of each node is not required to be acquired, the stored data size can be determined by multiplying the node number by 1 and the node maximum capacity and adding the rightmost leaf node data size, which is equivalent to the stored data size calculated by only accessing the rightmost leaf node data size, and the stored data size can be rapidly determined, so that the time consumption is reduced.
And the node also stores the version information of the data block, so that an attacker cannot replace the file by different versions of the same file, the integrity of the data is ensured, and the safety of data storage is improved.
The data storage method is realized based on a data storage system. Fig. 1 is a schematic diagram of a data storage system, where fig. 1 includes a control center server 110 and a plurality of data servers 120, and the control center server 110 is respectively communicatively connected to the plurality of data servers 120, and the connection manner between the ends in the system is not limited in the present invention. For example, one end can access the internet in a wired or wireless mode and be connected with the other end through a data interface, so that interaction among all ends in the system is realized.
It should be noted that, each data server 110 in the data storage system represents one node in the storage structure, that is, each data server 110 is configured to store a parent node ID, a current node ID, a directory name (directory name is a directory name of a data block stored by a child node corresponding to the current node), a left node ID, and data block version information (differentiating between different versions of the data block), or each data server 110 is configured to store the parent node ID, the current node ID, the left node ID, the data block name, and the data block version information. .
The control center server 110 and the data server 120 may each be a computing device, and in one embodiment, FIG. 2 illustrates a block diagram of a computing device 200 in accordance with one embodiment of the invention. As shown in FIG. 2, a block diagram of a computing device 200, in a basic configuration 202, the computing device 200 typically includes a system memory 206 and one or more processors 204. A memory bus 208 may be used for communication between the processor 204 and the system memory 206.
The process 204 may be any type of process depending on the desired configuration, including, but not limited to: a microprocessor (μp), a microcontroller (μc), a digital information processor (DSP), or any combination thereof. Processor 204 may include one or more levels of cache, such as a first level cache 210 and a second level cache 212, a processor core 214, and registers 216. The example processor core 214 may include an Arithmetic Logic Unit (ALU), a Floating Point Unit (FPU), a digital signal processing core (DSP core), or any combination thereof. The example memory controller 218 may be used with the processor 204, or in some implementations, the memory controller 218 may be an internal part of the processor 204.
Depending on the desired configuration, system memory 206 may be any type of memory including, but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. The system memory 206 may include an operating system 120, one or more applications 222, and program data 224. In some implementations, the application 122 may be arranged to operate on an operating system with the program data 224. Program data 224 includes instructions, and in computing device 200 according to the present invention, program data 224 includes instructions for performing data storage method 300.
Computing device 200 also includes a storage device 232, where storage device 232 includes removable storage 236 and non-removable storage 238, where removable storage 236 and non-removable storage 238 are each connected to storage interface bus 234. In the present invention, the data related to each event occurring during the execution of the program and the time information indicating the occurrence of each event may be stored in the storage device 232, and the operating system 220 is adapted to manage the storage device 232. Wherein the storage device 232 may be a magnetic disk.
Computing device 200 may also include an interface bus 240 that facilitates communication from various interface devices (e.g., output devices 242, peripheral interfaces 244, and communication devices 246) to basic configuration 202 via bus/interface controller 230. The exemplary output device 42 includes a graphics processing unit 248 and an audio processing unit 250. They may be configured to facilitate communication with various external devices, such as a display or speakers, via one or more a/V ports 252. The example peripheral interface 244 may include a serial interface controller 254 and a parallel interface controller 256, which may be configured to facilitate communication via one or more I/O ports 258 and external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device) or other peripherals (e.g., printer, scanner, etc.). The example communication device 246 may include a network controller 260 that may be arranged to facilitate communication with one or more other computing devices 262 over a network communication link via one or more communication ports 264.
The network communication link may be one example of a communication medium. Communication media may typically be embodied by computer readable instructions, data structures, program modules, and may include any information delivery media in a modulated data signal, such as a carrier wave or other transport mechanism. A "modulated data signal" may be a signal that has one or more of its data set or changed in such a manner as to encode information in the signal. By way of non-limiting example, communication media may include wired media such as a wired network or special purpose network, and wireless media such as acoustic, radio Frequency (RF), microwave, infrared (IR) or other wireless media. The term computer readable media as used herein may include both storage media and communication media.
Computing device 200 may be implemented as a server, such as a file server, database server, application server, WEB server, etc., as part of a small-sized portable (or mobile) electronic device, such as a cellular telephone, personal Digital Assistant (PDA), personal media player device, wireless WEB-watch device, personal headset device, application-specific device, or a hybrid device that may include any of the above functions. Computing device 200 may also be implemented as a personal computer including desktop and notebook computer configurations. In some embodiments, the operating system 220 of the computing device 200 is configured to perform a data storage method 300 and a data transmission method 600 according to the present invention.
FIG. 3 illustrates a flow chart of a data storage method 300 according to one embodiment of the invention. The method 300 is adapted to be executed in the control center server 110. As shown in fig. 3, the data storage method 300 starts at step S310, and in executing step S310, data to be stored is acquired, specifically: and acquiring data to be stored, searching a mounting point, mounting the data to be stored to the mounting point, and distributing a unique node ID for the mounting point. Each node ID is unique, and the node ID may be a random number, a random character, a random combination of numbers and characters, etc., which is not limited in this regard by the present invention, e.g., the node ID is rootID, id_ A, ID _a_1, etc.
Then, in step S320, the data to be stored is split into a plurality of data blocks, and each data block is assigned a unique name, where the data block name may be a random number, a random character, a random combination of a number and a character, etc., which is not limited in this invention, for example, the data block name is file A1.
Taking splitting a data block as an example, the following procedure is as follows: determining the current offset of a data pointer of the data to be stored, judging whether the product of the number of times of currently reading the data to be stored and a first preset value is smaller than the size of the data to be encrypted, if yes, reading the data of the first preset value by taking the determined current offset of the data pointer as a starting position, if no, determining the number of the data to be stored to be read according to the following formula, taking the determined current offset of the data pointer as a second value, and reading the data of the second value by taking the determined current offset of the data pointer as the starting position:
L=m-(a-1)*b
Wherein L is the number of data to be stored, m is the size of the memory occupied by the data to be stored, a is the number of times of currently reading the data to be stored, b is a first preset value, and the first value is the same as the maximum storage capacity of each node in the predetermined storage structure, that is, the maximum capacity of the leaf node corresponding to the data to be stored in the predetermined storage structure for storing the data is the same. The maximum storage capacity of each node in the predetermined storage structure may be set according to the storage capacity of each node, which is not limited in the present invention, for example, the storage capacity of each node is 5G.
Because the maximum capacity of the leaf node storage data of the preset storage structure is the same, and the data blocks are stored in the leaf node, after the data to be stored is stored by utilizing the preset storage structure, the stored data size is acquired, the stored data size of each node is not required to be acquired, the node number is reduced by 1 and then multiplied by the maximum capacity of the node, the data size of the rightmost leaf node is added, the data size to be encrypted can be determined, the stored data size can be calculated only by accessing the data size of the rightmost leaf node, the data size can be calculated quickly, and the time consumption is reduced.
And then taking the read data as a data block, and updating the current offset of the data pointer to be the sum of the determined current offset of the data pointer and a first preset value. And repeating the above process to divide the data to be stored into a plurality of data blocks.
In the process of splitting data to be stored into a plurality of data blocks, other leaf nodes except the rightmost leaf node store the data with the maximum capacity, and as the other leaf nodes store the data with the maximum capacity, the number of the nodes is reduced when the data with the same size are stored, so that the occupation of storage space is reduced in the data storage process, the system operation efficiency is improved, and the data storage efficiency is further improved.
After splitting the data to be stored into a plurality of data blocks, step S330 is executed to save each data block to a node of a predetermined storage structure, specifically:
version information is set for each data block, whether each node in a preset storage structure has a left node or not is judged, and if so, the left node ID of the node is stored in the current node. If the left node ID is not saved, which is equivalent to the sequence of the sub-nodes which are not saved, the data block sequence cannot be determined, so that the left node ID is saved in each node in the invention, thereby distinguishing the storage sequence of the data blocks stored by the nodes of the same level, and quickly constructing a mapping table after traversing the block file.
And continuously judging whether the current node has a child node or not, if yes (the current node is not a leaf node, the directory name of the data block stored by the child node corresponding to the current node is stored in the node), storing the father node ID, the current node ID, the directory name, the left node ID and the data block version information in the current node, and if no (the current node is a leaf node, the data block is stored in the node), storing the father node ID, the current node ID, the left node ID, the data block name and the data block version information in the current node.
The version information of the data block can be random numbers, random characters, random combinations of numbers and characters, and the like, and the invention is not limited to the random numbers, the random characters, the random combinations of numbers and characters, and the like, so long as the set version information is unique. In one embodiment, in response to an operation of replacing a data block in a leaf node of a predetermined storage structure by a target data block, it is determined whether the data block name of the target data block is the same as the data block name in the current node, if not, replacement is not performed, if yes, it is determined whether the data block version information of the target data block is the same as the data block version information in the current node, if so, the data block in the current node is replaced with the target data block, and if not, replacement is not performed.
By storing the data block names and the data block version information in the nodes, the node formats for storing the data are standardized, so that an attacker can be prevented from replacing the data blocks stored by the current node ID by using the data blocks with different names, and the attacker can be prevented from replacing the data blocks of the current version by using the data blocks of the previous version, the data integrity is ensured, and the safety of data storage is improved.
Taking a file split into 4 data blocks as an example, fig. 4 is a schematic diagram of a predetermined storage structure according to an embodiment. The memory structure in FIG. 4 includes node rootID, ID_ A, ID _ B, ID _ C, ID _A_1, ID_B_1, and ID_B_2, node rootID is the root node of the memory structure, node ID_ A, ID _ B, ID _C is the child node of node rootID, node ID_A_1 is the child node of node ID_A, and nodes ID_B_1 and ID_B_2 are the child nodes of node ID_B, respectively. In the memory structure shown in fig. 4, the nodes id_ C, ID _a_1, id_b_1 and id_b_2 are leaf nodes, in which data blocks are stored, and since id_b_2 is the rightmost leaf node, the nodes id_ C, ID _a_1 and id_b_1 may each store maximum capacity data, while the node id_b_2 may not store maximum capacity data, and there may be a part of free memory space at the node.
In this embodiment, the node rootID is the root node, and since each node stores the parent node ID and the data block name, the parent node in the node rootID is null and the data block name is null. Since the node id_ A, ID _b has child nodes, the node id_ A, ID _b stores a parent node ID, a current node ID, a directory name, a left node ID, and data block version information. Since the node id_c has no child node, the node id_c stores a parent node ID, a current node ID, a left node ID, a data block name, and data block version information. Node id_a_1 is a child node of node id_a and the node has no child node, so node id_a_1 stores parent node ID, current node ID, left node ID, data block name, and data block version information. Node id_b_1 and id_b_2 are child nodes of node id_b and there are no child nodes, so node id_b_1 and id_b_2 are parent node ID, current node ID, left node ID, data block name, and data block version information, respectively.
According to the storage structure, the data to be stored is mapped into a mapping table after being stored in a predetermined storage structure, and taking the storage structure shown in fig. 4 as an example, the mapping table corresponding to the mapping table is shown in fig. 5, fig. 5 is a schematic diagram of the mapping table corresponding to the storage structure of the file to be stored, and splitting the data to be stored into four data blocks (the names of the data blocks are file A1, file B2 and file C respectively) is taken as an example.
In fig. 5, the node rootID is the root node of the storage structure, and there is no left node, so the node stores the node ID (rootID) and the data block version information (version), and the stored parent node ID, left node ID and the data block name are all null. The node id_ A, ID _b is a child node of the node rootID, and there is a child node and no left node in the two nodes, so the parent node ID, the current node ID, the directory name, the left node ID and the data block Version information are respectively stored in the node id_ A, ID _b, specifically, the node id_ A, ID _b stores the node ID (id_ A, ID _b respectively), the parent node ID (rootID), the directory name (directory a and directory B respectively), the left node ID (null) and the data block Version information (Version). The node id_c is a child node of the node rootID, and the node has no child node, no left node, and thus the node id_c stores a parent node ID, a current node ID, a left node ID, a data block name, and data block Version information, specifically, the node id_c stores a node ID (id_c), a parent node ID (rootID), a left node ID (null), a data block (content), a data block name (file C), and data block Version information (Version). The node id_a_1 is a child node of the node id_a, and the node has no child node and no left node, so the node id_a_1 stores the node ID (id_a_1), the parent node ID (id_a), the left node ID (null), the data block (content), the data block name (fileA 1), and the data block Version information (Version). Node id_b_1 and id_b_2 are child nodes of node id_b, respectively, and two nodes have no child nodes, but node id_b_2 has left node id_b_1, so node id_b_1 and id_b_2 store node IDs (id_b_1, id_b_2, respectively), parent node IDs (id_b, respectively), left node IDs (null, id_b_1, respectively), data blocks (parent), data block names (file B1, file B2, respectively), and data block Version information (Version).
In one embodiment, in response to a data block name operation stored in one node of the renaming preset storage structure, the data block name stored in the current node is updated to be a renamed name without changing parent node information of the current node, so that after data is stored based on the storage structure defined by the invention, the renaming is low in execution complexity and high in efficiency.
Based on the above, the data storage method provided by the invention stores the data to be stored in a predetermined storage structure, specifically, each node in the predetermined storage structure stores parent node information, but child node information (including all file directory entries of the file directory) is not stored in the parent node, so that when the node information is modified, only the information of the node itself and the information (such as right node information) of the associated node are modified, and the change of the parent node is not involved, and therefore, when multiple clients modify the node information at the same time, the probability of collision of the nodes is greatly reduced, and the modification complexity is also reduced. In addition, the left node ID is also stored in the node, so that the data order can be ensured, and the mapping table can be obtained quickly by traversing the preset storage structure. In addition, the node also stores the version information of the data block, so that an attacker cannot replace the file by different versions of the same file, the integrity of the data is ensured, and the safety of data storage is improved.
In addition, since all leaf nodes except the rightmost leaf node store the maximum capacity data in the predetermined storage structure (compared with each leaf node storing only a part of data and not storing the maximum capacity data), the number of nodes can be reduced, thereby reducing the storage space waste. And because the maximum capacity of the stored data of the leaf nodes is the same, when the stored data size is determined, the stored data size of each node is not required to be acquired, but the stored data size can be calculated only by accessing the data size of the rightmost leaf node, so that the stored data size can be calculated quickly, and the time consumption is reduced.
However, since the above predetermined storage structure stores data, the child node information is not stored in the parent node in order to reduce the collision, and thus partial data sharing cannot be realized. To solve this problem, partial data sharing may be achieved by secondary encryption, i.e., data to be shared is encrypted and then sent to a, which shares data to B, but when a shares data to B, the data needs to be encrypted again, which is a typical one-to-one partial data sharing manner, resulting in partial data sharing being inflexible.
In order to solve the above problems, the present invention proposes a data transmission method. Fig. 6 shows a flow chart of a data transmission method 600 according to an embodiment of the invention. The method 600 is adapted to be executed in the control center server 110. As shown in fig. 6, the data transmission method 600 starts at step S610. In the executing step S610, data to be transmitted (shared data) is acquired, where the data to be transmitted is data stored by each node in the preset storage structure, that is, the data to be transmitted is part of the data stored in the preset storage structure. Next, in step S620, a hash operation is performed on the data to be transmitted to obtain a first hash value, in step S620, the data to be transmitted and the first hash value are combined to be used as target data, then in step S630, the target data is encrypted by a data encryption algorithm to obtain encrypted data, and finally in step S640, the encrypted data is sent to the receiving end. The data encryption algorithm is an ABE (Attribute Based Bncryption, attribute-based encryption system) encryption algorithm, and further, the data encryption algorithm may be a CP-ABE (Ciphertext Policy Attribute Based Bncryption, ciphertext-policy attribute-based encryption system) encryption algorithm.
The operation steps of the receiving end after receiving the encrypted data are as follows: decrypting the encrypted data through a data encryption algorithm to obtain target data, splitting the target data to obtain first data to be transmitted and a first hash value, carrying out hash operation on the first data to be transmitted to obtain a second hash value, judging whether the second hash value is identical to the first hash value, if so, using the data to be transmitted, and if not, not using the data to be transmitted.
According to the data transmission method provided by the invention, in the process of transmitting data, the integrity check of partial data (shared data) stored in the preset storage structure in the transmission process is realized based on the hash abstract, and different access strategies can be realized after the shared data is processed by the ABE encryption algorithm in the transmission process, so that a one-to-many partial data sharing mode is realized, and the granularity of data sharing is thinned.
The various techniques described herein may be implemented in connection with hardware or software or, alternatively, with a combination of both. Thus, the methods and apparatus of the present invention, or certain aspects or portions of the methods and apparatus of the present invention, may take the form of program code (i.e., instructions) embodied in tangible media, such as removable hard drives, U-drives, floppy diskettes, CD-ROMs, or any other machine-readable storage medium, wherein, when the program is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention.
In the case of program code execution on programmable computers, the computing device will generally include a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device. Wherein the memory is configured to store program code; the processor is configured to perform the data storage method and the data transmission method of the present invention according to instructions in said program code stored in the memory.
By way of example, and not limitation, readable media comprise readable storage media and communication media. The readable storage medium stores information such as computer readable instructions, data structures, program modules, or other data. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. Combinations of any of the above are also included within the scope of readable media.
In the description provided herein, algorithms and displays are not inherently related to any particular computer, virtual system, or other apparatus. Various general-purpose systems may also be used with examples of the invention. The required structure for a construction of such a system is apparent from the description above. In addition, the present invention is not directed to any particular programming language. It will be appreciated that the teachings of the present invention described herein may be implemented in a variety of programming languages, and the above description of specific languages is provided for disclosure of enablement and best mode of the present invention.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be construed as reflecting the intention that: i.e., the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in a device as described in this embodiment, or alternatively may be located in one or more devices different from the devices in this example. The modules in the foregoing examples may be combined into one module or may be further divided into a plurality of sub-modules.
Those skilled in the art will appreciate that the modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component and, furthermore, they may be divided into a plurality of sub-modules or sub-units or sub-components. Any combination of all features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or units of any method or apparatus so disclosed, may be used in combination, except insofar as at least some of such features and/or processes or units are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features but not others included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.
Furthermore, some of the embodiments are described herein as methods or combinations of method elements that may be implemented by a processor of a computer system or by other means of performing the functions. Thus, a processor with the necessary instructions for implementing the described method or method element forms a means for implementing the method or method element. Furthermore, the elements of the apparatus embodiments described herein are examples of the following apparatus: the apparatus is for carrying out the functions performed by the elements for carrying out the objects of the invention.
As used herein, unless otherwise specified the use of the ordinal terms "first," "second," "third," etc., to describe a general object merely denote different instances of like objects, and are not intended to imply that the objects so described must have a given order, either temporally, spatially, in ranking, or in any other manner.
While the invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of the above description, will appreciate that other embodiments are contemplated within the scope of the invention as described herein. Furthermore, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter. Accordingly, many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is intended to be illustrative, but not limiting, of the scope of the invention, which is defined by the appended claims.

Claims (12)

1. A data storage method, performed in a computing device, the method comprising the steps of:
acquiring data to be stored;
splitting a data object to be stored into a plurality of data blocks, and distributing a unique name for each data block;
storing each data block into a node of a preset storage structure, wherein each node in the preset storage structure corresponds to a data server, each node is assigned with a unique node ID, the maximum storage capacity of each node is the same, all leaf nodes except the rightmost leaf node store data with the maximum capacity, each node in the preset storage structure is used for storing first-class data, the first-class data comprises one or more of a father node ID, a directory name, the data block, a data block name and a node ID, and the directory name is the directory name of the data block stored by the child node corresponding to the current node;
judging whether each node in the preset storage structure has a left node or not, if so, storing the left node ID of the node in the current node;
wherein the step of splitting the data object to be stored into a plurality of data blocks comprises:
determining the current offset of a data pointer of the data object to be stored;
Reading data of a first preset value from the data to be stored by taking the determined current offset of the data pointer as a starting position, and taking the data as a data block;
updating the current offset of the data pointer to be the sum of the determined current offset of the data pointer and the first preset value.
2. The method of claim 1, wherein the step of reading the data of the first preset value from the data to be stored as a data block using the determined current offset of the data pointer as a starting position comprises:
judging whether the product of the number of times of currently reading the data to be stored and the first preset value is smaller than the size of the data to be stored, if yes, reading the data of the first preset value by taking the determined current offset of the data pointer as a starting position, and if not, determining the number of the data to be stored to be read according to the following formula, and taking the determined current offset of the data pointer as the starting position to read the data of the second value as the second value:
L=m-(a-1)*b
wherein L is the number of the data to be stored, m is the size of the storage space occupied by the data to be stored, a is the number of times of currently reading the data to be stored, and b is the first preset value;
the read data is used as a data block.
3. The method of claim 1 or 2, said step of saving each of said data blocks into a node of a predetermined storage structure comprising:
judging whether the current node has a child node or not, if so, storing the parent node ID, the current node ID and the directory name of the current node, and if not, storing the parent node ID, the data block name and the current node ID of the current node.
4. A method as claimed in claim 3, further comprising the step of:
setting version information for each data block, wherein the versions of the data blocks are distinguished through the version information of the data blocks;
wherein the judging whether the current node has the child node comprises:
judging whether the current node has a child node or not, if so, storing the parent node ID, the current node ID, the left node ID, the directory name and the data block version information in the current node, and if not, storing the parent node ID, the current node ID, the left node ID, the data block name and the data block version information in the current node.
5. The method of claim 4, further comprising the step of:
responding to the operation of replacing the data block in one leaf node of the preset storage structure by the target data block, judging whether the data block name of the target data block is the same as the data block name in the current node, if not, not replacing, and if so, judging whether the data block version information of the target data block is the same as the data block version information in the current node;
If the data block version information of the target data block is the same as the data block version information in the current node, replacing the data block in the current node with the target data block, and if the data block version information is different from the data block version information in the current node, not replacing.
6. The method according to claim 1 or 2, further comprising the step of:
in response to a renaming operation of a data block name stored in a node of the predetermined storage structure, the data block name stored in the current node is updated to be the renamed name.
7. The method of claim 1, wherein the first predetermined value is equal to a maximum storage capacity of each node in the predetermined storage structure.
8. A data transmission method performed in a computing device, the method comprising:
acquiring data to be transmitted, wherein the data to be transmitted is data stored by each node in a predetermined storage structure and stored by a data storage method according to any one of claims 1 to 7;
carrying out hash operation on the data to be transmitted to obtain a first hash value;
combining the data to be transmitted and the first hash value as target data;
encrypting the target data through a data encryption algorithm to obtain encrypted data;
And sending the encrypted data to a receiving end.
9. The method of claim 8, wherein the operation of the receiving end after receiving the encrypted data is as follows:
decrypting the encrypted data through the data encryption algorithm to obtain the target data;
splitting the target data to obtain first data to be transmitted and the first hash value;
performing hash operation on the first data to be transmitted to obtain a second hash value;
and judging whether the second hash value is the same as the first hash value, if so, using the data to be transmitted, and if not, not using the data to be transmitted.
10. The method of claim 8 or 9, wherein the data encryption algorithm is an ABE encryption algorithm.
11. A computing device, comprising:
at least one processor; and
a memory storing program instructions, wherein the program instructions are configured to be adapted to be executed by the at least one processor, the program instructions comprising instructions for performing the method of any of claims 1-10.
12. A readable storage medium storing program instructions which, when read and executed by a computing device, cause the computing device to perform the method of any of claims 1-10.
CN202210800036.2A 2021-04-21 2021-04-21 Data storage method, data transmission method and computing equipment Active CN115190136B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210800036.2A CN115190136B (en) 2021-04-21 2021-04-21 Data storage method, data transmission method and computing equipment

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110429687.0A CN113194127B (en) 2021-04-21 2021-04-21 Data storage method, data transmission method and computing equipment
CN202210800036.2A CN115190136B (en) 2021-04-21 2021-04-21 Data storage method, data transmission method and computing equipment

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CN202110429687.0A Division CN113194127B (en) 2021-04-21 2021-04-21 Data storage method, data transmission method and computing equipment

Publications (2)

Publication Number Publication Date
CN115190136A CN115190136A (en) 2022-10-14
CN115190136B true CN115190136B (en) 2024-03-01

Family

ID=76977821

Family Applications (2)

Application Number Title Priority Date Filing Date
CN202210800036.2A Active CN115190136B (en) 2021-04-21 2021-04-21 Data storage method, data transmission method and computing equipment
CN202110429687.0A Active CN113194127B (en) 2021-04-21 2021-04-21 Data storage method, data transmission method and computing equipment

Family Applications After (1)

Application Number Title Priority Date Filing Date
CN202110429687.0A Active CN113194127B (en) 2021-04-21 2021-04-21 Data storage method, data transmission method and computing equipment

Country Status (1)

Country Link
CN (2) CN115190136B (en)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105243334A (en) * 2015-09-17 2016-01-13 浪潮(北京)电子信息产业有限公司 Data storage protection method and system
CN110162525A (en) * 2019-04-17 2019-08-23 平安科技(深圳)有限公司 Read/write conflict solution, device and storage medium based on B+ tree
CN110569213A (en) * 2018-05-18 2019-12-13 北京果仁宝软件技术有限责任公司 File access method, device and equipment
JP2020038623A (en) * 2018-09-04 2020-03-12 北京百度网▲訊▼科技有限公司Beijing Baidu Netcom Science And Technology Co.,Ltd. Method, device, and system for storing data
CN111309523A (en) * 2020-02-16 2020-06-19 西安奥卡云数据科技有限公司 Data reading and writing method, data remote copying method and device and distributed storage system
CN111309260A (en) * 2020-02-16 2020-06-19 西安奥卡云数据科技有限公司 Data storage node selection method
CN111858146A (en) * 2019-04-26 2020-10-30 伊姆西Ip控股有限责任公司 Method, apparatus and computer program product for recovering data
WO2020220641A1 (en) * 2019-04-28 2020-11-05 平安科技(深圳)有限公司 Data storage method and apparatus based on point-to-point network, and medium and server
WO2021042923A1 (en) * 2019-09-06 2021-03-11 平安国际智慧城市科技股份有限公司 Data sharding storage method and apparatus, and terminal and computer-readable storage mediium

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7002964B1 (en) * 1998-02-24 2006-02-21 Canon Kabushiki Kaisha Communication system, method for a communication system and controller for a communication system
JP4655986B2 (en) * 2006-04-12 2011-03-23 ブラザー工業株式会社 Node device, storage control program, and information storage method
CN103282899B (en) * 2011-12-23 2016-06-15 华为技术有限公司 The storage method of data, access method and device in file system
US9432192B1 (en) * 2014-03-28 2016-08-30 Emc Corporation Content aware hierarchical encryption for secure storage systems
CN108628753B (en) * 2017-03-24 2021-02-23 华为技术有限公司 Memory space management method and device
CN111316251B (en) * 2017-11-06 2024-04-16 海量数据有限公司 Scalable storage system
CN111414426A (en) * 2020-03-26 2020-07-14 北京云图科瑞科技有限公司 Data processing method and system based on block chain

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105243334A (en) * 2015-09-17 2016-01-13 浪潮(北京)电子信息产业有限公司 Data storage protection method and system
CN110569213A (en) * 2018-05-18 2019-12-13 北京果仁宝软件技术有限责任公司 File access method, device and equipment
JP2020038623A (en) * 2018-09-04 2020-03-12 北京百度网▲訊▼科技有限公司Beijing Baidu Netcom Science And Technology Co.,Ltd. Method, device, and system for storing data
CN110162525A (en) * 2019-04-17 2019-08-23 平安科技(深圳)有限公司 Read/write conflict solution, device and storage medium based on B+ tree
CN111858146A (en) * 2019-04-26 2020-10-30 伊姆西Ip控股有限责任公司 Method, apparatus and computer program product for recovering data
WO2020220641A1 (en) * 2019-04-28 2020-11-05 平安科技(深圳)有限公司 Data storage method and apparatus based on point-to-point network, and medium and server
WO2021042923A1 (en) * 2019-09-06 2021-03-11 平安国际智慧城市科技股份有限公司 Data sharding storage method and apparatus, and terminal and computer-readable storage mediium
CN111309523A (en) * 2020-02-16 2020-06-19 西安奥卡云数据科技有限公司 Data reading and writing method, data remote copying method and device and distributed storage system
CN111309260A (en) * 2020-02-16 2020-06-19 西安奥卡云数据科技有限公司 Data storage node selection method

Also Published As

Publication number Publication date
CN115190136A (en) 2022-10-14
CN113194127B (en) 2022-08-12
CN113194127A (en) 2021-07-30

Similar Documents

Publication Publication Date Title
WO2020063820A1 (en) Asset transaction method, storage medium and computer device
CN111914520A (en) Document collaborative editing method and device, computer device and storage medium
CN109522314B (en) Block chain-based data archiving method and terminal equipment
US20180332008A1 (en) Systems and methods for regional data storage and data anonymization
WO2021174836A1 (en) Differential package generation method and apparatus, computer device, and storage medium
WO2022222350A1 (en) Method for encrypting data, and computing device
CN112632071A (en) Database primary key id generation method, device, equipment and storage medium
CN111611249A (en) Data management method, device, equipment and storage medium
US11360966B2 (en) Information processing system and method of controlling information processing system
CN112241474B (en) Information processing method, apparatus and storage medium
JP2022520141A (en) Methods and systems for efficiently processing the high transaction processing volume required by DApp on the blockchain
CN115190136B (en) Data storage method, data transmission method and computing equipment
CN111324645B (en) Block chain data processing method and device
WO2021012554A1 (en) Method and apparatus for updating data field in block chain, medium, and electronic device
CN116233254A (en) Business cut-off method, device, computer equipment and storage medium
CN116842012A (en) Method, device, equipment and storage medium for storing Redis cluster in fragments
US11625497B2 (en) File management system and non-transitory computer readable medium
JP2012164166A (en) Information processing system
CN110795604A (en) Method, apparatus, computing device and medium for generating transaction serial number
CN110889040B (en) Method and device for pushing information
CN116431566B (en) Data migration method, device, electronic equipment and medium
US20080262861A1 (en) User identification management system and method
CN110633273A (en) Authority management method and device
CN111404979B (en) Method and device for processing service request and computer readable storage medium
US11709845B2 (en) Federation of data during query time in computing systems

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant