CN113010103B - Data storage method and device, related equipment and storage medium - Google Patents
Data storage method and device, related equipment and storage medium Download PDFInfo
- Publication number
- CN113010103B CN113010103B CN202110059716.9A CN202110059716A CN113010103B CN 113010103 B CN113010103 B CN 113010103B CN 202110059716 A CN202110059716 A CN 202110059716A CN 113010103 B CN113010103 B CN 113010103B
- Authority
- CN
- China
- Prior art keywords
- data
- target
- node device
- disk
- attribute
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses a data storage method, a data storage device, related equipment and a storage medium, wherein the method comprises the following steps: responding to storage triggering operation aiming at the data to be stored, acquiring content data of the data to be stored from a client, and writing the content data into a target disk; acquiring attribute data of data to be stored, and writing the attribute data into a target disk when target node equipment is main node equipment in a node set; if the target node equipment is not the main node equipment in the node set, the attribute data is sent to the main node equipment, so that the main node equipment writes the attribute information into the target disk, and the performance of data storage in the object storage system can be improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data storage method and apparatus, a related device, and a storage medium.
Background
With the large-scale popularization and application of the internet, the explosive increase of data volume marks the coming of the big data era. The development of mass data brings convenience to people's lives, such as cloud storage, electronic payment, online shopping, and the like, but there is a serious challenge to store mass data.
Object storage systems are capable of supporting storage on the scale of billions or even billions of files. However, when the storage system stores files and data and metadata are written into the storage system, a problem of redundant data copy exists in the data transmission process, and the data is forcibly flushed twice every time the data is written, so that the performance of data storage is affected. Therefore, how to improve data storage performance becomes a current research hotspot.
Disclosure of Invention
The embodiment of the invention provides a data storage method, a data storage device, related equipment and a storage medium, which can improve the data storage performance of target node equipment.
In one aspect, an embodiment of the present invention provides a data storage method, where the data storage method is executed by a target node device in a node set, and the data storage method includes:
responding to a storage triggering operation aiming at the data to be stored, acquiring content data of the data to be stored from a client, and storing the content data into a target disk;
acquiring attribute data of data to be stored, and storing the attribute data into a target disk when target node equipment is main node equipment in a node set;
and if the target node equipment is not the main node equipment in the node set, sending the attribute data to the main node equipment so that the main node equipment stores the attribute information in the target disk.
The device comprises an acquisition unit, a storage unit and a processing unit, wherein the acquisition unit is used for responding to storage triggering operation aiming at data to be stored and acquiring content data of the data to be stored from a client;
a storage unit configured to write the content data to a target disk;
the acquiring unit is further configured to acquire attribute data of the data to be stored;
the storage unit is further configured to write the attribute data to the target disk when the target node device is a master node device in the node set;
a sending unit, configured to send the attribute data to the master node device if the target node device is not the master node device in the node set, so that the master node device writes the attribute information in the target disk.
The target node equipment is any node equipment in the node set and positioned in the same rack as the client; or the target node equipment is the node equipment with the same network address as the client in the node set; the data format of the content data acquired from the client is a first data format.
In one embodiment, the sending unit is further configured to send the content data in the first data format to the target disk; the storage unit is further configured to write the content data into the target disk after each of the other node devices in the node set except the target node device sends the content data in the first data format to the target disk; wherein the content data in the first data format in the other node device is transmitted by the target node device.
In one embodiment, the data format of the attribute data is converted by adopting a first data format to obtain the attribute data of the first data format; the storage unit is further configured to write the attribute data in the first data format to the target disk using the target protocol.
In one embodiment, when the attribute data in the first data format is written to the target disk by using the target protocol, the number of times of copying the attribute data in the first data format is less than a preset number threshold.
In one embodiment, the data storage device further comprises:
a judging unit configured to judge whether or not the content data is completely written to the target disk;
if the content data are written to the target disk completely, the obtaining unit is used for obtaining attribute data of the data to be stored; the storage unit is used for writing the attribute data into a target disk when the target node equipment is the main node equipment in the node set; the sending unit is used for sending the attribute data to the master node device if the target node device is not the master node device in the node set, so that the master node device writes the attribute data into the target disk.
In one embodiment, content data of one data to be stored is divided into one or more data blocks; after writing the last data block of the data to be stored to the target disk, performing disk brushing processing on the target disk, and detecting whether each disk brushing operation in the target disk is successful; and if each disk refreshing operation aiming at the target disk is successful, determining that the content data is completely written into the target disk.
In one aspect, an embodiment of the present invention provides a target node device, including:
a processor adapted to implement one or more instructions, an
A computer storage medium storing one or more instructions adapted to be loaded and executed by the processor to:
responding to a storage triggering operation aiming at the data to be stored, acquiring content data of the data to be stored from a client, and writing the content data into a target disk;
acquiring attribute data of data to be stored, and writing the attribute data into a target disk when target node equipment is main node equipment in a node set;
and if the target node equipment is not the main node equipment in the node set, sending the attribute data to the main node equipment so that the main node equipment writes the attribute information into the target disk.
In one aspect, an embodiment of the present invention provides a computer-readable storage medium, where program instructions are stored in the computer-readable storage medium, and when executed by a processor, the program instructions are configured to perform:
responding to storage triggering operation aiming at the data to be stored, acquiring content data of the data to be stored from a client, and writing the content data into a target disk;
acquiring attribute data of data to be stored, and writing the attribute data into a target disk when target node equipment is main node equipment in a node set;
and if the target node equipment is not the master node equipment in the node set, sending the attribute data to the master node equipment so that the master node equipment writes the attribute information into the target disk.
In one aspect, an embodiment of the present invention provides a computer program product or a computer program, where the computer program product includes a computer program, and the computer program is stored in a computer storage medium; a processor of the server reads the computer instructions from the computer storage medium, the processor performing:
responding to a storage triggering operation aiming at the data to be stored, acquiring content data of the data to be stored from a client, and writing the content data into a target disk;
acquiring attribute data of data to be stored, and writing the attribute data into a target disk when target node equipment is main node equipment in a node set;
and if the target node equipment is not the main node equipment in the node set, sending the attribute data to the main node equipment so that the main node equipment writes the attribute information into the target disk.
In the embodiment of the invention, when the client needs to store the data to be stored in the target disk, the client can transmit the data to be stored based on the target node device, wherein the target node device can firstly acquire the content data of the data to be stored from the client, the content data can be divided into one or more data blocks, the data format of the data block is a first data format, and the target node device acquires one data block from the client each time and writes the data block into the target disk. When the total amount of data blocks which are written into a target disk and are not subjected to disk refreshing operation reaches a set threshold value, the target node device performs disk refreshing operation, and compared with the case that the target node device acquires data blocks in a non-first data format, and then writes the data blocks in the non-first data format into the target disk, once disk refreshing operation is performed every 4MB, the number of times of disk refreshing of the target node device for storing the content data into the target disk can be reduced.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a block diagram of a data storage system according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a data storage method according to an embodiment of the present invention;
fig. 3 is a schematic diagram of copying when content data in a first data format is transmitted according to an embodiment of the present invention;
fig. 4 is a schematic diagram illustrating a transmission flow of content data according to an embodiment of the present invention;
FIG. 5 is a schematic diagram illustrating a transmission flow of attribute data according to an embodiment of the present invention;
fig. 6 is a schematic diagram illustrating copying performed when attribute data in a non-first data format is transmitted according to an embodiment of the present invention;
FIG. 7 is a schematic flow chart diagram of another data storage method provided by an embodiment of the invention;
FIG. 8 is a schematic flow chart of a data storage method applied to a data storage system according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of an encoding for performing a first data format conversion according to an embodiment of the present invention;
fig. 10 is a schematic code diagram for parsing data to be stored in a first data format according to an embodiment of the present invention;
FIG. 11 is a schematic flow chart diagram illustrating an alternative data storage method as applied to a data storage system according to an embodiment of the present invention;
FIG. 12 is a schematic structural diagram of a data storage device according to an embodiment of the present invention;
fig. 13 is a schematic structural diagram of a target node device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The embodiment of the invention provides a data storage method, which can enable a client to write data to be stored into a target disk for storage through any node device (such as a target node device) in a node device set, and does not need to store the data through a main node device in a node device set, so that the problem of redundant data copy caused when the data to be stored is written into the target disk can be avoided, meanwhile, the forced disk refreshing times of the target node device when the data to be stored is written into the target disk can be reduced, and the efficiency of the target node device in data storage is improved. The data storage method can be executed by a 0-zone system (next generation unified data lake object storage system following a distributed file system) and also can be executed by a data storage system as shown in fig. 1, wherein the data storage system comprises: a client 10, a node set 11, and a target disk 12, where the client 10 may be in a terminal shown in fig. 1, and the terminal may be a smart phone, a tablet computer, a notebook computer, etc., but is not limited thereto, and the node set 11 may include one or more node devices, where the node devices included in the node set 11 may be, for example, a node device 110, a node device 111, and a node device 112 shown in fig. 1. In one embodiment, communication connections are established between node devices included in the node set 11, that is, the node devices in the node set 11 can communicate with each other, and each node device in the node set 11 also establishes a communication connection with the client 10, so that the client 10 can also implement communication with any node device in the node set 11.
In an embodiment, when the client 10 needs to store the data to be stored in the target disk 12, the data to be stored may be stored in the target disk 12 through the node set 11, specifically, the client 10 may first send content data of the data to be stored to any node device in the node set, if the any node device is a target node device, the target node device may be, for example, a node device as denoted by 110 in fig. 1, and after receiving the content data sent by the client 10, the target node device 110 may synchronize the content data to other node devices in the node set, the other node devices may be, for example, node devices as denoted by 111 or 112 in fig. 1, and after each node device in the node set acquires the content data of the data to be stored, the content data may be written in the target disk 12, it may be understood that, writing the content data of the data to be stored in the target disk 12 based on each node device, backup of the content data to be stored in the target disk 12 may be achieved.
After the client 10 writes the content data of the data to be stored in the target disk 12, further, the client 10 may also store the attribute data of the data to be stored in the target disk 12, in a specific implementation, the client 10 may write the attribute data into any node device in the node set 11 first, if the node device written by the client 10 into the node set 11 is a target node device marked by 110, after the target node device 110 receives the attribute data sent by the client 10, the target node device 110 first determines whether the target node device 110 is a master node device, and when it is determined that the target node device 110 is a master node device, the attribute data may be directly written into the target disk 12 to store the attribute information in the target disk 12. If the target node device 110 is not the master node device, the target node device 110 may send the attribute data to the master node device based on the communication connections established between the node devices in the node set. After the master node device receives the attribute data from the target node device 110, the master node device may write the attribute data into the target disk for storage, so as to store the attribute data of the data to be stored in the target disk. In order to implement backup storage of the data to be stored in the target disk, in an embodiment, after the client sends the content data (or attribute data) of the data to be stored to the target node device, the content data may be further synchronized to other node devices, so that the other node devices write the content data into the target disk, thereby completing backup storage of the data to be stored in the target disk.
In an embodiment, the client and the node device in the node set may both be node devices in a blockchain network, and then the client and each node device in the node set may perform data transmission based on the blockchain network. The Blockchain (Blockchain) is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. The block chain, which is essentially a decentralized database, is a string of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, which is used for verifying the validity (anti-counterfeiting) of the information and generating a next block. The blockchain may include a blockchain underlying platform, a platform product services layer, and an application services layer.
Referring to fig. 2, which is a schematic flowchart of a data storage method according to an embodiment of the present invention, where the data storage method is executed by a target node device in the node set, as shown in fig. 2, the method may include:
s201, responding to the storage triggering operation aiming at the data to be stored, acquiring the content data of the data to be stored from the client, and writing the content data into the target disk.
The data to be stored may be file data, document data, or video data, and it is understood that the content data of the data to be stored may be data content of the file data, such as a document or a picture included in the file data, or data content of the document data, such as a text or a symbol in the document data, or data content of the video data, such as an image frame or audio included in the video data. In an embodiment, when a client needs to store data to be stored in a target disk, the client may first write the data to be stored into a target node device (or referred to as a Primary node device) of a node set, and after the target node device receives the data to be stored from the client, it is determined that a storage trigger operation for the data to be stored is received, where the node set includes one or more node devices, and when the node set includes at least two node devices, the node device for receiving the data to be stored written by the client is the target node device, it may be understood that node devices other than the target node device in the node set are other node devices, and the other node devices are sibling (Peer) node devices of the target node device.
In an embodiment, a data format of content data acquired by a target node device from a client is a first data format, when the data format of the content data acquired by the target node device from the client is the first data format, a section of user space memory of the target node device is mapped to a kernel space in the process of transmitting the content data by the target node device, after the mapping is successful, modification of the user space data by the target node device can be directly reflected to the kernel space, and similarly, modification of the kernel space data is also directly reflected to the user space, and the mapping relation enables the target node device not to copy data in the user space and the kernel space when transmitting the content data. The first data format may be a DirectByteBuffer type (one data type) data format in a Netty architecture (one data architecture), so that it can be understood that the data format of the content data is converted into the first data format, and when other node devices in the node set except the target node device transmit the content data, the user spaces and kernel spaces of the other node devices also have the mapping relationship, so that the copy times of the content data transmitted by the other node devices can be reduced.
Referring to fig. 3, fig. 3 is a schematic diagram illustrating that copying is performed when content data in a first data format is transmitted according to an embodiment of the present invention, and as shown in fig. 3, when content data in the first data format is transmitted, node devices in a node set relate to data interaction between an off-heap memory and a network card, and between the off-heap memory and a target disk. The method includes the steps that an off-heap memory allocates memory objects outside a heap of a Java (one-door-oriented object programming language) virtual machine, taking data copy when target node equipment in a node set transmits content data in a first data format as an example, the target node equipment acquires the content data from a client, the content data in the first data format needs to be read from the client through a network, when the target node equipment reads the content data in the first data format through the network, the target node equipment can directly read the content data in the first data format into the off-heap memory through the network, and similarly, when the target node equipment transmits the content data in the first data format to other node equipment in the node set, the target node equipment can write the content data in the first data format into the other node equipment from the off-heap memory through the network, wherein the network described here is a communication network between the node equipment; similarly, when the target node device writes the content data in the first data format to the target disk, the target node device may write the content data in the first data format from the off-heap memory to the target disk through the network.
In one embodiment, the target node device sends the content data in the first data format to the target disk, and at the same time, the target node device sends the content data in the first data format to other node devices in the node set except the target node device, and when all the other node devices send the content data in the first data format to the target disk, the target node device writes the content data in the target disk. After each Peer node device in the node set receives content data in a first data format sent by a target node device and sends the content data in the first data format to a target disk, each Peer node device sends a feedback message to the target node device, the target node device receives the feedback message sent by each Peer node device in the node set, and the feedback message is a message that the Peer node devices in the node set successfully write the content data in the first data format to the target disk. And after the target node equipment receives the feedback message sent by each Peer node equipment and successfully writes the content data to the target disk, the target node equipment sends the feedback message to the client, wherein the feedback message is a message that all the node equipment in the node set successfully writes the content data in the first data format to the target disk.
Referring to fig. 4, fig. 4 is a schematic diagram illustrating a transmission flow of content data according to an embodiment of the present invention. As shown in fig. 4, if the client and the node apparatus 1 and the node apparatus 2 are in the same rack (i.e., rack 1), and the node apparatus 3 and the node apparatus 1 and the node apparatus 2 are in different racks, it is assumed that the rack in which the node apparatus 3 is located is rack 2, and if the target node apparatus is node apparatus 1 in the same rack as the client, then node apparatus 2 and node apparatus 3 are Peer node apparatuses. When the client stores the data to be stored in the disk, the client may first send the content data of the data to be stored to the node device 1, after the node device 1 acquires the content data, the content data may be written into the target disk, meanwhile, the node device 1 may also send the content data to the Peer node device (e.g., node device 2 and node device 3 in fig. 4), after receiving the content data, the Peer node device may write the content data into the target disk, and based on that each node device in the node set writes the content data into the target disk, the content data backup of the data to be stored may be stored in the target disk.
S202, acquiring attribute data of the data to be stored, and writing the attribute data into the target disk when the target node device is a main node device in the node set.
Attribute data may be understood as information describing attributes of data, which attribute data is used to reflect certain aspects of the data, such as length, type, size, etc. of the data, wherein attribute data may also be referred to as metadata. In an embodiment, a target node device converts a data format of attribute data by using a first data format to obtain attribute data in the first data format, and then the target node device may write the attribute data in the first data format into a target disk by using a target protocol, where the target protocol may be a Raft protocol (a distributed consistency protocol), all operation requests in the Raft protocol are forwarded to a master node device (Leader node device), and then the Leader node device forwards the operation requests to each slave node device (Follower node device), and meanwhile, the Leader node device sends a log to each Follower node device for synchronization. When the target node device is the master node device, the target node device may write the attribute data in the first data format to the target disk, and the target node device sends the attribute data in the first data format to other node devices in the node set by using the target protocol. And the Follower node equipment in the node set writes the attribute data in the first data format to a target disk and forcibly refreshes the disk, and simultaneously writes a log raft log to the target disk and forcibly refreshes the disk.
If the target node device is a master node device, as shown in fig. 4, when the node device 1 is a master node device, the node device 2 and the node device 3 are taken as Follower node devices, and at this time, a schematic diagram of a transmission flow direction of the attribute data may also refer to fig. 4. The client sends the attribute data to the node device 1 (master node device), the node device 1 writes the attribute data to the target disk after receiving the attribute data sent by the client, the node device 1 adopts a target protocol to send the attribute data to the node device 2 (Follower node device) and the node device 3 (Follower node device), and the two Follower node devices write the attribute data to the target disk and then forcibly write the disk. The transmission of the attribute data from the client to the node device 1 and the transmission of the attribute data from the node device 1 to the node device 2 are both carried out in the same rack, so that the network delay can be reduced.
And S203, if the target node equipment is not the main node equipment in the node set, sending the attribute data to the main node equipment so that the main node equipment writes the attribute information into the target disk.
In one embodiment, when the target node device is not the master node device, the target node device sends attribute data in the first data format to the master node device, the master node device writes the attribute data in the first data format to a target disk, and the master node device sends the attribute data in the first data format to a Follower node device in the node set by using a target protocol. And the Follower node equipment in the node set writes the attribute data in the first data format to the target disk and forcibly writes the attribute data in the first data format to the target disk, and simultaneously writes the log raft log to the target disk and forcibly writes the log raft log to the target disk.
For example, if the target node device is not the master node device, please refer to fig. 5, and fig. 5 is a schematic diagram illustrating a transmission flow of attribute data according to an embodiment of the present invention. As shown in fig. 5, the client is on the same rack (i.e., rack 1) as node device 1 and node device 2, and node device 3 is on rack 2. The node device 1 in the same rack as the client is a target node device, and the node device 2 is a master node device. The client sends the attribute data to the node device 1, because the node device 1 is not the master node device, the node device 1 sends the attribute data to the node device 2 after receiving the attribute data sent by the client, and because the node device 2 is the master node device, the node device 2 can write the attribute data to the target disk after receiving the attribute data.
In one embodiment, when the attribute data in the first data format is written to the target disk by using the target protocol, the number of times of copying the attribute data in the first data format is less than a threshold value of preset times. The threshold for the predetermined number of times is the number of copies that would be made if the attribute data were not transmitted in the first data format in the data storage system. Referring to fig. 3, a copy process of the attribute data converted by the node device in the first data format (e.g., a data transmission format under a Netty data architecture) may be referred to, and a copy process of the attribute data converted by the node device in the non-first data format (e.g., a data transmission format under a ProtoBuf architecture) may be referred to fig. 6, which is shown in fig. 6 and is a schematic diagram of copying when the attribute data in the non-first data format is transmitted according to an embodiment of the present invention, where the non-first data format may be a data format obtained after being packaged by using ProtoBuf (a tool library).
As shown in fig. 6, taking a copy process when the target node device in the node set transmits attribute data in a non-first data format as an example, the target node device in the node set acquires the attribute data from the client, and needs to read the attribute data from the network, where the attribute data is not attribute data in the first data format. When the target node equipment reads the attribute data from the network, the target node equipment needs to copy the attribute data from the network card to the off-heap memory, then copy the attribute data from the off-heap memory to the small transit byte array, and then copy the attribute data from the small transit byte array to the big byte array; when the target node device sends the attribute data to other node devices in the node set, the target node device needs to copy the attribute data from the large byte array to the off-heap memory, then copies the attribute data from the off-heap memory to the network card, and then the target node device sends the attribute data to other node devices in the node set through the network; when the target node device writes the attribute data to the target disk, the target node device needs to copy the attribute data from the large byte array to the off-heap memory, and then write the attribute data from the off-heap memory to the target disk. As can be seen from fig. 3 and 6, when the target node device obtains the attribute data from the client, it needs to read the attribute data from the network, and when the target node device reads the attribute data in the first data format from the network, the target node device does not need to read the attribute data from the network and then cache the attribute data in the memory, that is, the target node device does not need to copy the attribute data from the off-heap memory to the small transit byte array, and then copy the attribute data from the small transit byte array to the large byte array, and the target node device can directly read the attribute data from the network to the off-heap memory, thereby reducing two data copies; when the target node device writes the attribute data in the first data format into the network and sends the attribute data to other node devices in the node set, the target node device does not need to copy the attribute data from the in-heap memory to the out-heap memory, and the target node device can directly write the attribute data into the network from the out-heap memory, so that one-time data copying can be reduced. When the target node device writes the attribute data in the first data format into the target disk, the target node device does not need to copy the attribute data from the in-heap memory to the out-heap memory, and can directly write the attribute data from the out-heap memory into the target disk, so that one-time data copying can be reduced. Therefore, the number of times of copying the attribute data in the first data format is smaller than the threshold value of the preset number of times.
In the embodiment of the invention, when the client needs to store the data to be stored in the target disk, the data to be stored can be transmitted based on the target node device, wherein the target node device can firstly acquire the content data of the data to be stored from the client and store the content data in the target disk, in addition, the target node device can also acquire the attribute data of the data to be stored from the client, because the attribute data can only be stored by the master node device in the node set, after the target node device receives the attribute data, whether the target node device is the master node device can be firstly judged, and when the target node device is determined to be the master node device, the attribute data can be directly stored, so that the storage of the data to be stored is completed, and when the target node device is not the master node device, the target node device can further forward the attribute data to the master node device, and the master node device stores the attribute data, so that the target node device can realize the storage of the data to be stored in the target disk when the master node device is not the master node device, and the content data and the attribute data in the target disk are in the first data format adopted to be transmitted, thereby improving the data storage performance.
Referring to fig. 7, which is a schematic flow chart of another data storage method according to an embodiment of the present invention, where the data storage method is executed by a target node device in the node set, as shown in fig. 7, the method may include:
s701, responding to the storage triggering operation aiming at the data to be stored, acquiring the content data of the data to be stored from the client, and writing the content data into the target disk.
S702, judging whether the content data is completely written to the target disk.
In one embodiment, content data of data to be stored received by a target node device from a client is divided into one or more data blocks, and if the content data has 128MB, the 128MB content data can be divided into 32 data blocks (chunk), and each data block has 4MB content data. When the target node device obtains content data from the client, a data block can be obtained from the client in an asynchronous mode each time, the data format of the data block is the first data format, the target node device receives the data block in the first data format sent by the client, then the data block is written into a target disk, and meanwhile the target node device sends the data block to Peer node devices in a node set. After each Peer node device in the node set receives a data block in a first data format sent by a target node device and sends the data block to a target disk, each Peer node device sends a message of successful writing of the data block to the target disk to the target node device, the target node device receives a message of successful writing of the data block to the target disk sent by each Peer node device in the node set, and after the target node device checks that the data block written by the target node device is successful to the target disk, the message is sent to a client, and the message is a message of successful writing of the data block to the target disk by all the node devices in the node set.
The client may not wait for the target node device to send a message that the data block is successfully written to the target disk, that is, after the client sends a data block to the target node device, the client may immediately send the next data block to the target node device, when all the node devices in the node set successfully write the last data block to the target disk, the target node device performs a disk-flushing process on the target disk, and if each disk-flushing operation in the target disk is successful, it is determined that the content data has been completely written to the target disk. When a data block is written into the target disk, the data volume of the target disk which is not subjected to the disk refreshing operation reaches a set threshold value, and at this time, the data block is the last data block written into the target disk. Each successful disk-swiping operation means that after the total data volume of the data blocks successfully written to the target disk reaches a set threshold, the target node device performs the disk-swiping operation, each successful disk-swiping operation is performed, and the threshold can be set according to the situation. For example, content data of one data to be stored is divided into 32 data blocks of 4MB, the set threshold may be 32MB, the eighth data block is the last data block when the first disk-flushing operation is performed, when the total amount of data of the data blocks successfully written into the target disk and not subjected to the disk-flushing operation reaches 32MB, the target node device can perform one disk-flushing operation, then the target node device will perform four disk-flushing operations, and when each disk-flushing operation (i.e., four disk-flushing operations) is successful, it is determined that the content data has been completely written into the target disk. When the data block of the non-first data format is transmitted, the target node equipment needs to perform the disk refreshing operation once every time the target node equipment writes the data block of 4MB into the target disk, so that the disk refreshing time can be reduced when the data block of the first data format is transmitted compared with the data block of the non-first data format.
And S703, if yes, acquiring attribute data of the data to be stored.
S704, when the target node device is the main node device in the node set, the attribute data is written into the target disk.
S705, if the target node device is not the master node device in the node set, sending the attribute data to the master node device, so that the master node device writes the attribute information in the target disk.
In steps S703 to S705, please refer to fig. 8, and fig. 8 is a schematic flowchart illustrating a data storage method applied to a data storage system according to an embodiment of the present invention. As shown in fig. 8, content data of data to be stored in the client is divided into 32 data blocks, and the client may send each data block to the target node device in an asynchronous manner, where each data block is included in the message request, and a format of each data block is a first data format, where the first data format may be a data format converted by using Netty. As shown in fig. 9, the user request byte buffer includes a message header (request) and a message body (request body), and both are data of a DirectByteBuffer type, which can prevent the target node device from copying the data block from a Java Virtual Machine (JVM) heap memory to an out-heap memory. Netty does not need to copy data in multiple small buffers (directbytebuffers) into one large buffer (DirectByteBuffer), as in the code shown in fig. 9, each request. It is also necessary to write the length of the request header (request header for remaining ()) to the header lenbuf (a parameter defined), and put the header lenbuf into the List, so as to distinguish the request header from the request body.
As shown in fig. 8, the target node device writes the received data block in the first data format to the target disk after Netty analysis, and sends the data block in the first data format to the two Peer node devices, the two Peer node devices receive the data block in the first data format, analyze the data block in the first data format using Netty, and then write the data block to the target disk. Referring to fig. 10, when parsing a data block in a first data format, as shown in fig. 10, by asserting a data accumulator (COMPOSITE _ accumulator), the Netty can combine the received contents of the data block instead of copying all the contents of the data block to a large buffer. In addition, during compiling, the application program obtains the message body request.body, namely the data block, through a slice operation (a group operation), namely bytebuf.slice (), and the bytebuf.slice () does not relate to memory copy but only refers to the memory copy, so that the memory copy can be avoided.
As shown in fig. 8, after all the node devices in the node set successfully write the data block reaching the set threshold to the target disk, the target node device performs a forced disk-flushing operation through a sync command, when all the content data have been successfully flushed, the client sends the attribute data in the first data format to the target node device, the encoding for performing the first data format conversion on the attribute data may refer to fig. 9, the target node device receives the attribute data in the first data format sent by the client, when the target node device is not the master node device, the target node device sends the attribute data in the first data format to the master node device, the master node device writes the attribute data in the first data format to the target disk after Netty analysis, the analyzed code may refer to fig. 10, and then the master node device sends the attribute data in the first data format to the slave node device. The slave node equipment receives the attribute data in the first data format sent by the master node equipment, analyzes the attribute data in the first data format by using Netty and writes the analyzed attribute data in the first data format into a target disk, and performs forced disk refreshing operation through a command sync, and simultaneously writes a log Raft log in a Raft request into the target disk and performs forced disk refreshing operation through the command sync.
For another example, please refer to fig. 11, fig. 11 is a schematic flowchart illustrating another data storage method applied to a data storage system according to an embodiment of the present invention. The schematic flowchart is a schematic flowchart of transmitting attribute data in a non-first data format when the target node device is a master node device. As shown in fig. 11, the client packages the attribute data into a non-first data format, which may be a data format converted by using a ProtoBuf, and sends the non-first data format to the target node device. The master node equipment receives the attribute data in the non-first data format sent by the client side, and sends the attribute data in the non-first data format to the slave node equipment through a target protocol Raft protocol. The method comprises the steps that the slave node equipment receives attribute data in a non-first data format, utilizes the ProtoBuf to analyze the attribute data in the non-first data format, writes the attribute data to a target disk, and conducts forced disk refreshing operation through a sync command, and meanwhile, the slave node equipment writes a log Raft log in a Raft request to the target disk, and conducts forced disk refreshing operation through the sync command.
In the embodiment of the invention, when the client needs to store the data to be stored in the target disk, the client can transmit the data to be stored based on the target node device, wherein the target node device can firstly acquire the content data of the data to be stored from the client, the content data can be divided into one or more data blocks, the data format of the data block is a first data format, and the target node device acquires one data block from the client each time and writes the data block into the target disk. When the total amount of data blocks which are written into a target disk and are not subjected to disk refreshing operation reaches a set threshold value, the target node device performs disk refreshing operation, and compared with the case that the target node device acquires data blocks in a non-first data format, and then writes the data blocks in the non-first data format into the target disk, once disk refreshing operation is performed every 4MB, the number of times of disk refreshing of the target node device for storing the content data into the target disk can be reduced.
Based on the data storage method, the embodiment of the invention provides a data storage device. Fig. 12 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present invention. The data processing apparatus described in fig. 12 may operate as follows:
an obtaining unit 1201, configured to obtain content data of data to be stored from a client in response to a storage trigger operation for the data to be stored;
a storage unit 1202 configured to write the content data to a target disk;
an obtaining unit 1201, further configured to obtain attribute data of the data to be stored;
a storage unit 1202, further configured to write the attribute data to the target disk when the target node device is a master node device in the node set;
a sending unit 1203, configured to send the attribute data to the master node device if the target node device is not the master node device in the node set, so that the master node device writes the attribute information in the target disk.
In one embodiment, the target node device is any node device in the node set and in the same rack as the client; or the target node device is a node device in the node set, which has the same network address as the client.
In one embodiment, the data format of the content data acquired by the acquisition unit 1201 from the client is a first data format.
In one embodiment, the sending unit 1203 sends the content data in the first data format to the target disk; the storage unit 1202 writes the content data to the target disk after each of the other node devices in the node set, except for the target node device, sends the content data in the first data format to the target disk. Wherein the content data in the first data format in the other node device is transmitted by the target node device.
In one embodiment, the data format of the attribute data is converted by adopting a first data format to obtain the attribute data of the first data format; the storage unit 1202 writes the attribute data of the first data format to the target disk using the target protocol.
In one embodiment, when the attribute data in the first data format is written to the target disk by using the target protocol, the number of times of copying the attribute data in the first data format is less than a preset number threshold.
In one embodiment, the data storage device further comprises:
a judgment unit 1204 that judges whether or not the content data is completely written to the target disk;
dividing content data of data to be stored into one or more data blocks; after writing the last data block of the data to be stored to the target disk, performing disk brushing processing on the target disk, and detecting whether each disk brushing operation in the target disk is successful; and if each disk refreshing operation aiming at the target disk is successful, determining that the content data is completely written into the target disk.
According to an embodiment of the present invention, the steps involved in the data storage method shown in fig. 2 may be performed by the units in the data storage device shown in fig. 12. For example, S201 shown in fig. 2 may be executed by the acquiring unit 1201 and the storing unit 1202 in the data processing apparatus shown in fig. 12, S202 may be executed by the acquiring unit 1201 and the storing unit 1202 in the data storage apparatus shown in fig. 12, and S203 may be executed by the transmitting unit 1203 in the data storage apparatus shown in fig. 12.
According to an embodiment of the present invention, the steps involved in the data storage method shown in fig. 7 may be performed by the units in the data storage device shown in fig. 12. For example, S701 described in fig. 7 may be executed by the acquisition unit 1201 and the storage unit 1202 in the data processing apparatus shown in fig. 12, S702 may be executed by the determination unit 1204 in the data storage apparatus shown in fig. 12, S703 may be executed by the acquisition unit 1201 in the data storage apparatus shown in fig. 12, S704 may be executed by the storage unit 1202 in the data storage apparatus shown in fig. 12, and S705 may be executed by the transmission unit 1203 in the data storage apparatus shown in fig. 12.
According to another embodiment of the present invention, the units in the data storage device shown in fig. 12 may be respectively or entirely combined into one or several other units to form one or several other units, or some unit(s) may be further split into multiple units with smaller functions to form the same operation, without affecting the achievement of the technical effect of the embodiment of the present invention. The units are divided based on logic functions, and in practical applications, the functions of one unit can also be implemented by a plurality of units, or the functions of a plurality of units can also be implemented by one unit. In other embodiments of the present invention, the data storage device may also include other units, and in practical applications, these functions may also be implemented by the assistance of other units, and may be implemented by cooperation of a plurality of units.
According to another embodiment of the present invention, the data processing apparatus as shown in fig. 12 may be constructed by running a computer program (including program codes) capable of executing the steps involved in the respective methods shown in fig. 2 or fig. 7 described above on a general-purpose computing device such as a computer including a Central Processing Unit (CPU), a random access storage medium (RAM), a read-only storage medium (ROM), and the like, and a storage element, and the data storage method of the embodiment of the present invention may be implemented. The computer program may be embodied on a computer-readable storage medium, for example, and loaded into and executed by the above-described computing apparatus via the computer-readable storage medium.
In this embodiment of the present invention, when a client needs to store data to be stored in a target disk, the client may transmit the data to be stored based on a data storage device, where the obtaining unit 1201 may obtain content data of the data to be stored from the client first, and then the storage unit 1202 stores the content data in the target disk, and furthermore, the obtaining unit 1201 may also obtain attribute data of the data to be stored from the client, because the attribute data may only be stored by a master node device in a node set, after the obtaining unit 1201 receives the attribute data, the determining unit 1204 may determine whether the target node device is the master node device first, and when the target node device is determined to be the master node device, the storage unit 1202 directly stores the attribute data, and completes storage of the data to be stored, and when the target node device is not the master node device, the sending unit 1203 may further forward the attribute data to the device, and the master node device stores the attribute data, so that the target node device may store the data to be stored in the target disk when the target node device is not the master node device, and since both the content data and the attribute data in the target disk are in a first data format, the first data, the data transmission performance of the data may be reduced, and the data may be transmitted.
Based on the embodiments of the data processing method and the data processing apparatus, the embodiments of the present invention provide a target node device, where the target node device may be a server or a terminal. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as cloud service, a cloud database, cloud computing, cloud functions, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and a big data and artificial intelligence platform. The terminal may be, but is not limited to, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and the like. Fig. 13 is a schematic structural diagram of a target node device according to an embodiment of the present invention. The target node device depicted in fig. 13 may include at least a processor 1301, an input interface 1302, an output interface 1303, and a computer storage medium 1304. The processor 1301, the input interface 1302, the output interface 1303, and the computer storage medium 1304 may be connected by a bus or other means.
A computer storage medium 1304 may be stored in the memory of the target node device, the computer storage medium 1304 being for storing a computer program comprising program instructions, the processor 1301 being for executing the program instructions stored by the computer storage medium 1304. Processor 1301 (or CPU) is a computing core and a control core of a server, which is adapted to implement one or more instructions for implementing the steps of the corresponding methods in fig. 2 and 7.
In one embodiment, the processor 1301 is configured to invoke the program instructions to perform:
responding to storage triggering operation aiming at the data to be stored, acquiring content data of the data to be stored from a client, and writing the content data into a target disk;
acquiring attribute data of data to be stored, and writing the attribute data into a target disk when target node equipment is main node equipment in a node set;
and if the target node equipment is not the main node equipment in the node set, sending the attribute data to the main node equipment so that the main node equipment writes the attribute information into the target disk.
In one embodiment, the data format of the content data obtained from the client is a first data format, and the processor 1301 is configured to call the program instructions to perform:
and sending the content data in the first data format to a target disk, and writing the content data into the target disk after each other node device except the target node device in the node set sends the content data in the first data format to the target disk.
Wherein the content data in the first data format in the other node device is transmitted by the target node device.
In one embodiment, the processor 1301 is configured to invoke the program instructions to perform:
converting the data format of the attribute data by adopting a first data format to obtain attribute data of the first data format;
and writing the attribute data in the first data format into the target disk by adopting a target protocol.
When the attribute data in the first data format is written to the target disk by adopting the target protocol, the copying times of the attribute data in the first data format is less than a preset time threshold value.
In one embodiment, before obtaining the attribute data of the data to be stored, the processor 1301 is configured to call the program instruction to perform:
judging whether the content data is completely written to the target disk;
if yes, triggering and executing the step of obtaining the attribute data of the data to be stored.
In one embodiment, content data of one data to be stored is divided into one or more data blocks; determining whether the content data is written to the target disk, where the processor 1301 is configured to call the program instruction to perform:
and after writing the last data block of the data to be stored to the target disk, performing disk brushing processing on the target disk, and detecting whether each disk brushing operation in the target disk is successful.
Embodiments of the present invention provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method embodiments as shown in fig. 2 or fig. 7. The computer-readable storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
While the invention has been described with reference to a particular embodiment, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (8)
1. A data storage method performed by a target node device in a node set, the method comprising:
responding to a storage triggering operation aiming at data to be stored, acquiring content data of the data to be stored from a client, sending the content data to a target disk, writing the content data into the target disk after each other node device except the target node device in the node set sends the content data to the target disk, wherein the data format of the content data is a first data format, the disk refreshing time of the data in the first data format when being written into the disk is smaller than a preset disk refreshing time threshold value, and the preset disk refreshing time threshold value is the disk refreshing time of the data in a non-first data format when being written into the disk; wherein the content data in the first data format in the other node device is transmitted by the target node device;
acquiring attribute data of the data to be stored, converting the data format of the attribute data by adopting the first data format to obtain attribute data of the first data format, and writing the attribute data of the first data format into the target disk by adopting a target protocol when the target node equipment is a main node equipment in the node set;
if the target node device is not the master node device in the node set, the attribute data is sent to the master node device, so that the master node device writes the attribute information in the first data format into the target disk by adopting a target protocol.
2. The method of claim 1, wherein the target node device is any node device in the node set that is in the same chassis as the client; alternatively, the first and second electrodes may be,
the target node device is a node device in the node set, which has the same network address with the client.
3. The method according to claim 1, wherein when the target protocol is used to write the attribute data in the first data format to the target disk, the number of times of copying the attribute data in the first data format is less than a preset number threshold.
4. The method according to claim 1, wherein before the obtaining the attribute data of the data to be stored, the method further comprises:
judging whether the content data is completely written to the target disk;
and if so, triggering and executing the step of acquiring the attribute data of the data to be stored.
5. The method according to claim 4, wherein the content data of one of the data to be stored is divided into one or more data blocks; the determining whether the content data is completely written to the target disk includes:
after writing the last data block of the data to be stored to the target disk, performing disk brushing processing on the target disk, and detecting whether each disk brushing operation in the target disk is successful;
and if each disk refreshing operation aiming at the target disk is successful, determining that the content data is completely written into the target disk.
6. A data storage device, characterized in that the device comprises:
the device comprises an acquisition unit, a storage unit and a processing unit, wherein the acquisition unit is used for responding to storage triggering operation aiming at data to be stored and acquiring content data of the data to be stored from a client;
the storage unit is used for sending the content data to a target disk, writing the content data into the target disk after each other node device except the target node device in a node set sends the content data to the target disk, wherein the data format of the content data is a first data format, the disk refreshing time of the data in the first data format when being written into the disk is smaller than a preset disk refreshing time threshold value, and the preset disk refreshing time threshold value is the disk refreshing time of the data in a non-first data format when being written into the disk; wherein the content data in the first data format in the other node device is transmitted by the target node device;
the acquiring unit is further configured to acquire attribute data of the data to be stored, and convert the data format of the attribute data by using the first data format to obtain attribute data of the first data format;
the storage unit is further configured to, when the target node device is a master node device in the node set, write attribute data in the first data format to the target disk by using a target protocol;
a sending unit, configured to send the attribute data to the master node device if the target node device is not the master node device in the node set, so that the master node device writes the attribute information in the first data format in the target disk by using a target protocol.
7. A target node device, comprising:
a processor adapted to implement one or more instructions, an
A computer storage medium storing one or more instructions adapted to be loaded by the processor and to perform a data storage method as claimed in any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that it stores a computer program comprising program instructions that are loaded and executed by one or more processors to perform the method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110059716.9A CN113010103B (en) | 2021-01-15 | 2021-01-15 | Data storage method and device, related equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110059716.9A CN113010103B (en) | 2021-01-15 | 2021-01-15 | Data storage method and device, related equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113010103A CN113010103A (en) | 2021-06-22 |
| CN113010103B true CN113010103B (en) | 2023-03-21 |
Family
ID=76384249
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110059716.9A Active CN113010103B (en) | 2021-01-15 | 2021-01-15 | Data storage method and device, related equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113010103B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114726867B (en) * | 2022-02-28 | 2023-09-26 | 重庆趣链数字科技有限公司 | Hot standby multi-main method based on Lift |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110691062A (en) * | 2018-07-06 | 2020-01-14 | 浙江大学 | Data writing method, device and equipment |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104272274B (en) * | 2013-12-31 | 2017-06-09 | 华为技术有限公司 | Data processing method and equipment in a kind of distributed file storage system |
| US10929311B2 (en) * | 2019-06-26 | 2021-02-23 | EMC IP Holding Company LLC | Stream level uninterrupted backup operation using data probe |
| CN110928496B (en) * | 2019-11-12 | 2022-04-22 | 杭州宏杉科技股份有限公司 | Data processing method and device on multi-control storage system |
| CN111309262B (en) * | 2020-02-16 | 2021-01-29 | 西安奥卡云数据科技有限公司 | Distributed storage cache reading and writing method |
| CN111638980A (en) * | 2020-05-25 | 2020-09-08 | 北京青云科技股份有限公司 | Message processing method, device and system based on memory mapping and storage medium |
-
2021
- 2021-01-15 CN CN202110059716.9A patent/CN113010103B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110691062A (en) * | 2018-07-06 | 2020-01-14 | 浙江大学 | Data writing method, device and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113010103A (en) | 2021-06-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10356156B2 (en) | Method and device for loading webpage | |
| US10693816B2 (en) | Communication methods and systems, electronic devices, and computer clusters | |
| CN107153644B (en) | Data synchronization method and device | |
| CN114201421B (en) | Data stream processing method, storage control node and readable storage medium | |
| CN109101662B (en) | Block generation method, device, equipment and storage medium | |
| CN110019873B (en) | Face data processing method, device and equipment | |
| CN112416654B (en) | Database log replay method, device, equipment and storage medium | |
| CN109213828B (en) | Block generation method, device, equipment and storage medium | |
| US10970236B2 (en) | System and method for optimized input/output to an object storage system | |
| CN111338688B (en) | Data long-acting caching method and device, computer system and readable storage medium | |
| CN113703672A (en) | Super-fusion system, IO request issuing method thereof and physical server | |
| CN113010103B (en) | Data storage method and device, related equipment and storage medium | |
| CN115470156A (en) | RDMA-based memory use method, system, electronic device and storage medium | |
| CN111966653A (en) | Data processing method, device, server and storage medium for micro-service call link | |
| CN111435323B (en) | Information transmission method, device, terminal, server and storage medium | |
| CN112202862A (en) | Kafka-based cluster data and file synchronization method and device | |
| CN113010113B (en) | Data processing method, device and equipment | |
| CN114925078A (en) | Data updating method, system, electronic device and storage medium | |
| CN112486413B (en) | Data reading method, device, medium and computing equipment | |
| CN114490540A (en) | Data storage method, medium, device and computing equipment | |
| CN113868249A (en) | Data storage method and device, computer equipment and storage medium | |
| CN115129779A (en) | Database synchronization method, device and readable medium | |
| CN109241362B (en) | Block generation method, device, equipment and storage medium | |
| US10671505B2 (en) | System and method for monitoring service | |
| CN113612869A (en) | Short network address generation method, terminal and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40046485 Country of ref document: HK |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |