WO2021238798A1 - 一种数据的存储方法、装置及计算机系统 - Google Patents

一种数据的存储方法、装置及计算机系统 Download PDF

Info

Publication number
WO2021238798A1
WO2021238798A1 PCT/CN2021/095198 CN2021095198W WO2021238798A1 WO 2021238798 A1 WO2021238798 A1 WO 2021238798A1 CN 2021095198 W CN2021095198 W CN 2021095198W WO 2021238798 A1 WO2021238798 A1 WO 2021238798A1
Authority
WO
WIPO (PCT)
Prior art keywords
serial number
written
copy
data
preset condition
Prior art date
Application number
PCT/CN2021/095198
Other languages
English (en)
French (fr)
Inventor
荆伟
陈玉强
何抗洪
韩盛中
Original Assignee
苏宁易购集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 苏宁易购集团股份有限公司 filed Critical 苏宁易购集团股份有限公司
Publication of WO2021238798A1 publication Critical patent/WO2021238798A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0673—Single storage device
    • G06F3/0674—Disk device
    • G06F3/0676—Magnetic disk device

Definitions

  • the present invention relates to the field of data storage, in particular to a data storage method, device and computer system.
  • Disk is an unstable storage medium, and it is often damaged. When the disk is damaged, the data stored in it will be lost.
  • One way to solve the problem of easy loss of data stored on disk is to back up a data block, store it in multiple copies at the same time, and store each copy on a different storage medium, which greatly avoids the storage medium. A situation where data is lost due to damage.
  • the main purpose of the present invention is to provide a data storage method, device and computer system to ensure the data consistency of each copy of the data block.
  • the present invention provides a data storage method, the method includes:
  • the method includes:
  • the receiving a write request of the target data block includes:
  • the storage method includes:
  • the current serial number corresponding to the target data block is updated according to the serial number to be written.
  • updating the current serial number corresponding to the target data block according to the serial number to be written includes:
  • the current serial number corresponding to the target data block is updated according to the serial number to be written.
  • the method includes:
  • the preset condition includes that the difference between the serial number to be written and the maximum serial number does not exceed one.
  • the present application discloses a data storage device, the device includes:
  • a receiving module configured to receive a write request of a target data block, the write request including a serial number to be written and data corresponding to the serial number to be written;
  • the processing module is configured to traverse all the copies corresponding to the target data block, and determine whether there is the copy with the difference between the stored maximum serial number and the serial number to be written that satisfies a preset condition, and the copy is used for storage Data and the serial number corresponding to each said data;
  • the writing module is configured to write the serial number to be written and the data to be written into the copy that meets the preset condition when there is the copy that meets the preset condition.
  • the processing module may also be used for storing the copy according to the serial number to be written and the copy that does not meet the preset condition when there is the copy that does not meet the preset condition.
  • the maximum serial number determines the missing serial number of the copy that does not meet the preset condition; obtains the data corresponding to the missing serial number from the copy that meets the preset condition, and writes it in the missing serial number. The copy that meets the preset condition.
  • the receiving module may also be used to obtain the current serial number corresponding to the target data block
  • the processing module may also be used to update the target data block according to the serial number to be written when the write is successful The corresponding current serial number.
  • this application discloses a computer system, which includes:
  • One or more processors are One or more processors;
  • a memory associated with the one or more processors where the memory is used to store program instructions, and when the program instructions are read and executed by the one or more processors, perform the following operations:
  • the write request including the data to be written and the serial number to be written corresponding to the data to be written;
  • the present application provides a data storage method, the method includes: receiving a write request of a target data block, the write request including data to be written and a serial number to be written corresponding to the data to be written; Traverse all the copies corresponding to the target data block, and determine whether there is the copy in which the difference between the stored maximum serial number and the serial number to be written meets a preset condition, and the copy is used to store the corresponding serial number When there is the copy that meets the preset condition, the data to be written and the serial number to be written are written into the copy that meets the preset condition, so as to realize that according to the serial number Manage the writing of each copy of the data block to avoid the loss of a certain data block or a copy after the previous writing fails and the new write will be stored, causing the data of the copy to be inconsistent. Integrity, affecting the reliability of data storage;
  • this application also proposes that for a copy that does not meet the preset conditions, the data corresponding to the missing serial number of the copy is obtained from the copy that meets the preset conditions and stored in the copy, so that when the copy needs to be repaired, the serial number can be found.
  • the missing data is removed and incremental repair is performed, which avoids the full data repair of the copy, saves the time required for repair, and avoids the risk of damage to other data blocks caused by long-term repair;
  • This application proposes that when the number of successfully written copies exceeds a preset threshold, the current sequence number of the corresponding target data block is updated according to the sequence number to be written to ensure that the number of successfully written copies can satisfy the data The need for backup to avoid data loss when the copy is damaged due to too few successfully written copies.
  • FIG. 1 is a schematic diagram of a multi-copy sequence architecture provided by an embodiment of the present application
  • FIG. 2 is a schematic diagram of a process of writing multi-copy sequence data provided by an embodiment of the present application
  • Figure 3 is a flowchart of a method provided by an embodiment of the present application.
  • Figure 4 is a structural diagram of an apparatus provided by an embodiment of the present application.
  • Fig. 5 is a structural diagram of a computer system provided by an embodiment of the present application.
  • this application provides a data storage method, which assigns a corresponding serial number to each write to the data block, and records each data The parent serial number of the block, each time the copy is read or written, it is checked whether the written serial number and the serial number of the copy meet the preset conditions. For the copies that do not meet the preset conditions, the missing data is obtained from other copies and written to ensure the data consistency of each copy.
  • Step 1 Receive a data write request sent by the requesting end, where the request includes the data to be written and the address to be written;
  • the server calculates the block number of the target data block to be stored according to the preset size of each data block and the address to be written, and generates a write request and adds it to the request queue corresponding to the target data block.
  • the generated write request includes the data to be written.
  • the write request When there is an unprocessed task in the corresponding request queue, the write request is processed after waiting for the completion of the unprocessed task. When there is no unprocessed task, the write request is sent to the serial number management module.
  • Step 2 The serial number management module obtains the current serial number of the target data block, and generates the corresponding serial number to be written according to the current serial number;
  • the value of the serial number to be written should be the current serial number+1.
  • the serial number management module may also calculate the next serial number corresponding to the current serial number according to other calculation methods or formulas, and use the next serial number as the serial number to be written corresponding to the current serial number.
  • the serial number management module generates a write request and sends it to all copies of the target data block according to the data to be written and the generated serial number to be written.
  • Each data block has a preset number of copies.
  • the copy is used to back up the data block.
  • the data written by each write request is stored in each copy according to the sequence contained in the request. You can find and obtain the corresponding data from the copy.
  • the serial number and the data corresponding to each serial number are stored.
  • the copy also records the serial number corresponding to the latest data stored in the copy.
  • the serial number corresponding to the latest data stored in the copy is the copy
  • the serial number with the largest value among the serial numbers corresponding to the stored data is the largest serial number.
  • Step 3 The copy receives the write request, and confirms whether the serial number to be written contained in the write request and the maximum serial number of the copy meet the preset conditions;
  • the preset condition includes that the difference between the serial number to be written and the maximum serial number of the copy does not exceed 1.
  • Step 4 The copy that meets the preset condition is written according to the data included in the write request and the serial number to be written;
  • the copy When the copy finishes writing the data, it sends a write success signal to the corresponding target data block.
  • the target data block determines that the data contained in the write request is successfully written, and updates the local current serial number to the serial number to be written .
  • the target data block may determine that the data is written successfully, and update the local current serial number to the serial number to be written.
  • Copy 3 sends a data pull request to copy 1 or copy 2, and stores the data corresponding to serial number 2 and serial number 3 obtained from copy 1 or copy 2 locally, and then stores the received serial number 4 to be written The corresponding data is written, and a successful write signal is sent to the target data block after the write is successful.
  • the method further includes:
  • Step 5 Determine the block number of the target data block to be read according to the address of the data to be read contained in the read request and the preset data block size, and send the read request of the target data block to the serial number Management module;
  • Step 6 The serial number management module obtains the current serial number of the target data block and the maximum serial number of all copies of the target data block, matches and obtains the target copy with the same maximum serial number as the current serial number;
  • the serial number management module sends a data read request to the target copy, and the target copy returns corresponding data to the requesting end that sent the request according to the read request.
  • this application provides a data storage method. As shown in FIG. 3, the method includes:
  • the method includes:
  • the receiving the write request of the target data block includes:
  • the storage method includes:
  • updating the current serial number corresponding to the target data block according to the serial number to be written includes:
  • the method includes:
  • the preset condition includes that the difference between the serial number to be written and the maximum serial number does not exceed one.
  • this application provides a data storage method.
  • the device includes:
  • the receiving module 410 is configured to receive a write request of a target data block, where the write request includes the data to be written and the serial number to be written corresponding to the data to be written;
  • the processing module 420 is configured to traverse all the copies corresponding to the target data block and determine whether there is the copy in which the difference between the stored maximum sequence number and the sequence number to be written meets a preset condition, and the copy is used for Store the data corresponding to each serial number;
  • the writing module 430 is configured to write the serial number to be written and the data to be written into the copy that meets the preset condition when there is the copy that meets the preset condition.
  • the processing module 420 may be further configured to, when there is the copy that does not meet the preset condition, store the stored copy according to the serial number to be written and the copy that does not meet the preset condition.
  • the largest serial number is used to determine the missing serial number of the copy that does not meet the preset condition; the writing module 430 can also be used to obtain the missing serial number corresponding to the copy that meets the preset condition And write the copy that does not meet the preset conditions.
  • the receiving module 410 may also be used to obtain the current serial number corresponding to the target data block; according to a preset calculation method, determine the serial number to be written corresponding to the current serial number; the processing module 420 can also be used to update the current serial number corresponding to the target data block according to the serial number to be written when the writing is successful.
  • the processing module 420 may also be configured to update the current serial number corresponding to the target data block according to the serial number to be written when the number of the successfully written copies exceeds a preset threshold.
  • the receiving module 410 may also be configured to receive a read request for a target data block sent by the requesting end; the processing module 420 queries whether the target data block has the same maximum serial number as the current serial number The copy; when it exists, return the data stored in the copy with the maximum serial number the same as the current serial number to the requesting end.
  • the preset condition includes that the difference between the serial number to be written and the maximum serial number does not exceed one.
  • the fourth embodiment of the present application provides a computer system, including: one or more processors; and a memory associated with the one or more processors, the memory being used to store program instructions When the program instructions are read and executed by the one or more processors, the following operations are performed:
  • FIG. 5 exemplarily shows the architecture of the computer system, which may specifically include a processor 1510, a video display adapter 1511, a disk drive 1512, an input/output interface 1513, a network interface 1514, and a memory 1520.
  • the processor 1510, the video display adapter 1511, the disk drive 1512, the input/output interface 1513, the network interface 1514, and the memory 1520 may be communicatively connected through the communication bus 1530.
  • the processor 1510 may be implemented in a general-purpose CPU (Central Processing Unit, central processing unit), microprocessor, application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits. Perform relevant procedures to realize the technical solutions provided in this application.
  • a general-purpose CPU Central Processing Unit, central processing unit
  • microprocessor microprocessor
  • ASIC Application Specific Integrated Circuit
  • the memory 1520 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc.
  • the memory 1520 may store an operating system 1521 for controlling the operation of the computer system 1500, and a basic input output system (BIOS) for controlling the low-level operation of the computer system 1500.
  • BIOS basic input output system
  • a web browser 1523, a data storage management system 1524, and an icon font processing system 1525 can also be stored.
  • the foregoing icon font processing system 1525 may be an application program that specifically implements the foregoing steps in the embodiment of the present application. In short, when the technical solution provided by the present application is implemented through software or firmware, the relevant program code is stored in the memory 1520 and is called and executed by the processor 1510.
  • the input/output interface 1513 is used to connect input/output modules to realize information input and output.
  • the input/output/module can be configured in the device as a component (not shown in the figure), or it can be connected to the device to provide corresponding functions.
  • the input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and an output device may include a display, a speaker, a vibrator, an indicator light, and the like.
  • the network interface 1514 is used to connect to a communication module (not shown in the figure) to implement communication interaction between the device and other devices.
  • the communication module can realize communication through wired means (such as USB, network cable, etc.), or through wireless means (such as mobile network, WIFI, Bluetooth, etc.).
  • the bus 1530 includes a path to transmit information between various components of the device (for example, the processor 1510, the video display adapter 1511, the disk drive 1512, the input/output interface 1513, the network interface 1514, and the memory 1520).
  • various components of the device for example, the processor 1510, the video display adapter 1511, the disk drive 1512, the input/output interface 1513, the network interface 1514, and the memory 1520.
  • the computer system 1500 can also obtain information about specific receiving conditions from the virtual resource object receiving condition information database 1541 for condition judgment, and so on.
  • the above device only shows the processor 1510, the video display adapter 1511, the disk drive 1512, the input/output interface 1513, the network interface 1514, the memory 1520, the bus 1530, etc., in the specific implementation process, the The equipment may also include other components necessary for normal operation.
  • the above-mentioned device may also include only the components necessary to implement the solution of the present application, and not necessarily include all the components shown in the figure.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Techniques For Improving Reliability Of Storages (AREA)
  • Debugging And Monitoring (AREA)

Abstract

一种数据的存储方法、装置及计算机系统，其中所述方法包括：接收目标数据块的写入请求，所述写入请求包括待写入数据及所述待写入数据对应的待写入序列号（310）；遍历所有所述目标数据块对应的副本，判断是否存在存储的最大序列号与所述待写入序列号的差值满足预设条件的所述副本（320），所述副本用于存储数据及每一所述数据对应的序列号；当存在满足所述预设条件的所述副本时，将所述待写入序列号及所述待写入数据写入满足所述预设条件的所述副本（330），避免了出现数据块丢失某一次的写入或是某一副本在之前的写入失败后即存入新一次的写入、造成该副本的数据不完整，影响数据存储的可靠性的情况。

Description

一种数据的存储方法、装置及计算机系统 技术领域
本发明涉及数据存储领域，尤其涉及一种数据的存储方法、装置及计算机系统。
 背景技术
磁盘是一种不稳定存储介质，常常会发生损坏，而当磁盘损坏时存储于其中的数据就会发生丢失。一种解决存储在磁盘的数据容易丢失问题的方法是将一个数据块进行备份，将其同时存储于多个副本中，并将每个副本存储在不同的存储介质上，大大避免了由于存储介质损坏导致数据丢失的情况。
 技术问题
由于数据块常常会发生多次的存储或读取，而存储的过程中可能会发生错误或是丢失。当出现某一数据块发生一次或多次的存储错误或丢失时，可能会出现数据块的多个副本存储的数据不一致、导致读取时可能从不完整的副本读取数据，影响读取的准确性。
 技术解决方案
为了解决现有技术的不足，本发明的主要目的在于提供一种数据的存储方法、装置及计算机系统，以保证数据块的每一副本的数据一致性。
为了达到上述目的，第一方面本发明提供了一种数据的存储方法，所述方法包括：
接收目标数据块的写入请求，所述写入请求包括待
数据及所述待写入数据对应的待写入序列号；
遍历所述目标数据块对应的所有副本，判断是否存在存储的最大序列号与所述待写入序列号的差值满足预设条件的所述副本，所述副本用于存储数据及每一所述数据对应的序列号；
当存在满足所述预设条件的所述副本时，将所述待写入序列号及所述待写入数据写入满足所述预设条件的所述副本。
在一些实施例中，所述方法包括：
当存在不满足所述预设条件的所述副本时，根据所述待写入序列号及不满足所述预设条件的所述副本存储的所述最大序列号，确定不满足所述预设条件的所述副本的缺失的序列号；
从满足所述预设条件的所述副本获取所述缺失的序列号对应的数据，并写入不满足所述预设条件的所述副本。
在一些实施例中，所述接收目标数据块的写入请求包括：
获取所述目标数据块对应的当前序列号；
按照预设的计算方法，确定所述当前序列号对应的所述待写入序列号；
所述存储方法包括：
当写入成功时，根据所述待写入序列号更新所述目标数据块对应的当前序列号。
在一些实施例中，所述当写入成功时，根据所述待写入序列号更新所述目标数据块对应的当前序列号包括：
当写入成功的所述副本的数量超过预设阈值时，根据所述待写入序列号更新所述目标数据块对应的当前序列号。
在一些实施例中，所述方法包括：
接收请求端发出的目标数据块的读取请求；
查询所述目标数据块是否存在所述最大序列号与所述当前序列号相同的所述副本；
当存在时，将所述最大序列号与所述当前序列号相同的所述副本存储的数据返回至所述请求端。
在一些实施例中，所述预设条件包括所述待写入序列号与所述最大序列号的差值不超过1。
第二方面，本申请公开了一种数据的存储装置，所述装置包括：
接收模块，用于接收目标数据块的写入请求，所述写入请求包括待写入序列号及所述待写入序列号对应的数据；
处理模块，用于遍历所述目标数据块对应的所有副本，判断是否存在存储的最大序列号与所述待写入序列号的差值满足预设条件的所述副本，所述副本用于存储数据及每一所述数据对应的序列号；
写入模块，用于当存在满足所述预设条件的所述副本时，将所述待写入序列号及所述待写入数据写入满足所述预设条件的所述副本。
在一些实施例中，所述处理模块还可用于当存在不满足所述预设条件的所述副本时，根据所述待写入序列号及不满足所述预设条件的所述副本存储的所述最大序列号，确定不满足所述预设条件的所述副本的缺失的序列号；从满足所述预设条件的所述副本获取所述缺失的序列号对应的数据，并写入不满足所述预设条件的所述副本。
在一些实施例中，所述接收模块还可用于获取所述目标数据块对应的当前序列号；
按照预设的计算方法，确定所述当前序列号对应的所述待写入序列号；所述处理模块还可用于当写入成功时，根据所述待写入序列号更新所述目标数据块对应的当前序列号。
第三方面，本申请公开了一种计算机系统，所述系统包括：
一个或多个处理器；
以及与所述一个或多个处理器关联的存储器，所述存储器用于存储程序指令,所述程序指令在被所述一个或多个处理器读取执行时，执行如下操作：
接收目标数据块的写入请求，所述写入请求包括待写入数据及所述待写入数据对应的待写入序列号；
遍历所述目标数据块对应的所有副本，判断是否存在存储的最大序列号与所述待写入序列号的差值满足预设条件的所述副本，所述副本用于存储数据及每一所述数据对应的序列号；
当存在满足所述预设条件的所述副本时，将所述待写入序列号及所述待写入数据写入满足所述预设条件的所述副本。
 有益效果
本发明实现的有益效果为：
本申请提供了一种数据的存储方法，所述方法包括：接收目标数据块的写入请求，所述写入请求包括待写入数据及所述待写入数据对应的待写入序列号；遍历所有所述目标数据块对应的副本，判断是否存在存储的最大序列号与所述待写入序列号的差值满足预设条件的所述副本，所述副本用于存储每一序列号对应的数据；当存在满足所述预设条件的所述副本时，将所述待写入数据及所述待写入序列号写入满足所述预设条件的所述副本，实现了根据序列号对数据块的每个副本的写入进行管理，避免出现数据块丢失某一次的写入或是某一副本在之前的写入失败后即存入新一次的写入，造成该副本的数据不完整，影响数据存储的可靠性；
进一步地，本申请还提出了对不满足预设条件的副本，从满足预设条件的副本获取该副本缺失的序列号对应的数据并存入该副本，使得副本需要修复时可以根据序列号找出缺失的数据并进行增量修复，避免了对副本进行全量数据的修复，节省了修复所需的时间，避免了长时间的修复对其他数据块造成损坏的风险；
本申请提出了当写入成功的所述副本的数量超过预设阈值时，根据所述待写入序列号更新对应的目标数据块的当前序列号，保证成功写入的副本的数量能够满足数据的备份需求，避免了写入成功的副本过少导致副本出现损坏时数据发生丢失。
本发明所有产品并不需要具备上述所有效果。
 附图说明
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的多副本序列架构示意图；
图2是本申请实施例提供的多副本序列数据写入过程示意图；
图3是本申请实施例提供的方法流程图；
图4是本申请实施例提供的装置结构图；
图5是本申请实施例提供的计算机系统结构图。
 本发明的实施方式
为使本发明的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
如背景技术所述，为了保证数据块的每一副本的一致性，本申请提供了一种数据的存储方法，将对数据块的每次写入都赋予对应的序列号，并记录每一数据块的父序列号，在每次对副本进行读写时，对写入的序列号及副本的序列号是否满足预设条件进行检验。对不满足预设条件的副本从其他副本获取缺少的数据并写入，以保证每个副本的数据一致性。
实施例一
具体的，上述方案可通过如下步骤实现：
步骤一、接收请求端发出的数据写入请求，所述请求中包括待写入数据及待写入地址；
服务端根据预设的每一数据块的大小及所述待写入地址，计算得到待存入的目标数据块的块号，生成写入请求加入至目标数据块对应的请求队列中。
具体的，生成的写入请求中包括待写入数据。
当对应的请求队列中存在未处理任务时，即等待未处理任务完成后处理写入请求，当不存在未处理任务时，即将所述写入请求送至序列号管理模块。
步骤二、序列号管理模块获取目标数据块的当前序列号，根据当前序列号生成对应的待写入序列号；
具体的，待写入序列号的值应当为当前序列号+1。
具体的，序列号管理模块也可根据其他的计算方法或公式计算获得当前序列号对应的下一序列号，并将该下一序列号作为该当前序列号对应的待写入序列号。
序列号管理模块根据待写入数据及生成的待写入序列号，生成写入请求并发送至目标数据块的所有副本。
每一数据块具有预设个数的副本，副本用于对数据块进行备份，如图1所示，在每一副本中存储着每次写入请求写入的数据，根据请求中包含的序列号可从副本中查找并获取对应的数据。
在每一副本中，存储着序列号及每一序列号对应的数据。为了便于对接收的写入请求进行校验，副本中还记录了该副本最新存储的数据对应的序列号，当序列号以升序的顺序生成时，副本最新存储的数据对应的序列号为该副本存储的数据对应的序列号中的数值最大的序列号即最大序列号。
步骤三、副本接收到写入请求，并确认写入请求中包含的待写入序列号及所述副本的最大序列号是否满足预设条件；
具体的，当待写入序列号为当前序列号+1时，所述预设条件包括待写入序列号与该副本的最大序列号的差值不超过1。
步骤四、满足所述预设条件的副本根据所述写入请求包含的数据及待写入序列号进行写入；
当副本完成数据的写入后，即向对应的目标数据块发送写入成功信号。当目标数据块接收的写入成功信号的数量超过预设阈值时，目标数据块即判断该写入请求包含的数据写入成功，并将本地的当前序列号更新为所述待写入序列号。
具体的，目标数据块可在接收的写入成功信号的数量超过副本总数的一半时，判断数据写入成功，并将本地的当前序列号更新为所述待写入序列号。
对不满足预设条件的副本，首先根据待写入序列号与该副本的最大序列号，确定该副本缺失的数据对应的序列号。具体的，如图2所示，副本3在接收到待写入序列号为4的写入请求时，判断待写入序列号4与本地的最大序列号1的差值为3超过了1，副本3即认为发生了数据的缺失，并根据待写入序列号4及最大序列号1判断本地缺失了序列号2及序列号3对应的数据。副本3即向副本1或副本2发出数据的拉取请求，将从副本1或副本2获取的序列号2及序列号3对应的数据存储至本地，然后再将接收的待写入序列号4对应的数据进行写入，并在写入成功后向目标数据块发出写入成功的信号。
当服务端接收到请求端发出的数据读取请求后，所述方法还包括：
步骤五、根据读取请求中包含的待读取数据的地址及预设的数据块的大小，确定需要读取的目标数据块的块号，并将目标数据块的读取请求发送至序列号管理模块；
步骤六、序列号管理模块获取该目标数据块的当前序列号及该目标数据块所有副本的最大序列号，匹配并获取最大序列号与所述当前序列号相同的目标副本；
在确定了目标副本后，序列号管理模块向目标副本发送数据的读取请求，目标副本根据读取请求向发送请求的请求端返回对应的数据。
实施例二
对应上述实施例，本申请提供了一种数据的存储方法，如图3所示，所述方法包括：
310、接收目标数据块的写入请求，所述写入请求包括待写入数据及所述待写入数据对应的待写入序列号；
320、遍历所有所述目标数据块对应的副本，判断是否存在存储的最大序列号与所述待写入序列号的差值满足预设条件的所述副本，所述副本用于存储每一序列号对应的数据；
330、当存在满足所述预设条件的所述副本时，将所述待写入序列号及所述待写入数据写入满足所述预设条件的所述副本。
优选的，所述方法包括：
340、当存在不满足所述预设条件的所述副本时，根据所述待写入序列号及不满足所述预设条件的所述副本存储的所述最大序列号，确定不满足所述预设条件的所述副本的缺失的序列号；
341、从满足所述预设条件的所述副本获取所述缺失的序列号对应的数据，并写入不满足所述预设条件的所述副本。
优选的，所述接收目标数据块的写入请求包括：
351、获取所述目标数据块对应的当前序列号；
352、按照预设的计算方法，确定所述当前序列号对应的所述待写入序列号；
所述存储方法包括：
353、当写入成功时，根据所述待写入序列号更新所述目标数据块对应的当前序列号。
优选的，所述当写入成功时，根据所述待写入序列号更新所述目标数据块对应的当前序列号包括：
354、当写入成功的所述副本的数量超过预设阈值时，根据所述待写入序列号更新所述目标数据块对应的当前序列号。
优选的，所述方法包括：
360、接收请求端发出的目标数据块的读取请求；
361、查询所述目标数据块是否存在所述最大序列号与所述当前序列号相同的所述副本；
362、当存在时，将所述最大序列号与所述当前序列号相同的所述副本存储的数据返回至所述请求端。
优选的，所述预设条件包括所述待写入序列号与所述最大序列号的差值不超过1。
实施例三
对应所述方法实施例，本申请提供了一种数据的存储方法，如图4所示，所述装置包括：
接收模块410，用于接收目标数据块的写入请求，所述写入请求包括待写入数据及所述待写入数据对应的待写入序列号；
处理模块420，用于遍历所有所述目标数据块对应的副本，判断是否存在存储的最大序列号与所述待写入序列号的差值满足预设条件的所述副本，所述副本用于存储每一序列号对应的数据；
写入模块430，用于当存在满足所述预设条件的所述副本时，将所述待写入序列号及所述待写入数据写入满足所述预设条件的所述副本。
优选的，所述处理模块420还可用于当存在不满足所述预设条件的所述副本时，根据所述待写入序列号及不满足所述预设条件的所述副本存储的所述最大序列号，确定不满足所述预设条件的所述副本的缺失的序列号；所述写入模块430还可用于从满足所述预设条件的所述副本获取所述缺失的序列号对应的数据，并写入不满足所述预设条件的所述副本。
优选的，所述接收模块410还可用于获取所述目标数据块对应的当前序列号；按照预设的计算方法，确定所述当前序列号对应的所述待写入序列号；所述处理模块420还可用于当写入成功时，根据所述待写入序列号更新所述目标数据块对应的当前序列号。
优选的，所述处理模块420还可用于当写入成功的所述副本的数量超过预设阈值时，根据所述待写入序列号更新所述目标数据块对应的当前序列号。
优选的，所述接收模块410还可用于接收请求端发出的目标数据块的读取请求；所述处理模块420查询所述目标数据块是否存在所述最大序列号与所述当前序列号相同的所述副本；当存在时，将所述最大序列号与所述当前序列号相同的所述副本存储的数据返回至所述请求端。
优选的，所述预设条件包括所述待写入序列号与所述最大序列号的差值不超过1。
实施例四
对应上述方法、设备及系统，本申请实施例四提供一种计算机系统，包括： 一个或多个处理器；以及与所述一个或多个处理器关联的存储器，所述存储器用于存储程序指令,所述程序指令在被所述一个或多个处理器读取执行时，执行如下操作：
接收目标数据块的写入请求，所述写入请求包括待写入序列号及所述待写入序列号对应的数据；
遍历所有所述目标数据块对应的副本，判断是否存在存储的最大序列号与所述待写入序列号的差值满足预设条件的所述副本，所述副本用于存储每一序列号对应的数据；
当存在满足所述预设条件的所述副本时，将所述待写入序列号及所述待写入数据写入满足所述预设条件的所述副本。
其中，图5示例性的展示出了计算机系统的架构，具体可以包括处理器1510，视频显示适配器1511，磁盘驱动器1512，输入/输出接口1513，网络接口1514，以及存储器1520。上述处理器1510、视频显示适配器1511、磁盘驱动器1512、输入/输出接口1513、网络接口1514，与存储器1520之间可以通过通信总线1530进行通信连接。
其中，处理器1510可以采用通用的CPU（Central Processing Unit，中央处理器）、微处理器、应用专用集成电路（Application Specific Integrated Circuit，ASIC）、或者一个或多个集成电路等方式实现，用于执行相关程序，以实现本申请所提供的技术方案。
存储器1520可以采用ROM（Read Only Memory，只读存储器）、RAM（Random Access Memory，随机存取存储器）、静态存储设备，动态存储设备等形式实现。存储器1520可以存储用于控制计算机系统1500运行的操作系统1521，用于控制计算机系统1500的低级别操作的基本输入输出系统(BIOS)。另外，还可以存储网页浏览器1523，数据存储管理系统1524，以及图标字体处理系统1525等等。上述图标字体处理系统1525就可以是本申请实施例中具体实现前述各步骤操作的应用程序。总之，在通过软件或者固件来实现本申请所提供的技术方案时，相关的程序代码保存在存储器1520中，并由处理器1510来调用执行。 输入/输出接口1513用于连接输入/输出模块，以实现信息输入及输出。输入输出/模块可以作为组件配置在设备中（图中未示出），也可以外接于设备以提供相应功能。其中输入设备可以包括键盘、鼠标、触摸屏、麦克风、各类传感器等，输出设备可以包括显示器、扬声器、振动器、指示灯等。
网络接口1514用于连接通信模块（图中未示出），以实现本设备与其他设备的通信交互。其中通信模块可以通过有线方式（例如USB、网线等）实现通信，也可以通过无线方式（例如移动网络、WIFI、蓝牙等）实现通信。
总线1530包括一通路，在设备的各个组件（例如处理器1510、视频显示适配器1511、磁盘驱动器1512、输入/输出接口1513、网络接口1514，与存储器1520）之间传输信息。
另外，该计算机系统1500还可以从虚拟资源对象领取条件信息数据库1541中获得具体领取条件的信息，以用于进行条件判断，等等。
需要说明的是，尽管上述设备仅示出了处理器1510、视频显示适配器1511、磁盘驱动器1512、输入/输出接口1513、网络接口1514，存储器1520，总线1530等，但是在具体实施过程中，该设备还可以包括实现正常运行所必需的其他组件。此外，本领域的技术人员可以理解的是，上述设备中也可以仅包含实现本申请方案所必需的组件，而不必包含图中所示的全部组件。
通过以上的实施方式的描述可知，本领域的技术人员可以清楚地了解到本申请可借助软件加必需的通用硬件平台的方式来实现。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备（可以是个人计算机，云服务器，或者网络设备等）执行本申请各个实施例或者实施例的某些部分所述的方法。
本说明书中的各个实施例均采用递进的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于系统或系统实施例而言，由于其基本相似于方法实施例，所以描述得比较简单，相关之处参见方法实施例的部分说明即可。以上所描述的系统及系统实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性劳动的情况下，即可以理解并实施。
以上所述仅为本发明的较佳实施例，并不用以限制本发明，凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。

Claims (10)

  1. 一种数据的存储方法，其特征在于，所述方法包括：
    接收目标数据块的写入请求，所述写入请求包括待写入数据及所述待写入数据对应的待写入序列号；
    遍历所述目标数据块对应的所有副本，判断是否存在存储的最大序列号与所述待写入序列号的差值满足预设条件的所述副本，所述副本用于存储数据及每一所述数据对应的序列号；
    当存在满足所述预设条件的所述副本时，将所述待写入序列号及所述待写入数据写入满足所述预设条件的所述副本。
  2. 根据权利要求1所述的方法，其特征在于，所述方法包括：
    当存在不满足所述预设条件的所述副本时，根据所述待写入序列号及不满足所述预设条件的所述副本存储的所述最大序列号，确定不满足所述预设条件的所述副本的缺失的序列号；
    从满足所述预设条件的所述副本获取所述缺失的序列号对应的数据，并写入不满足所述预设条件的所述副本。
  3. 根据权利要求1或2所述的方法，其特征在于，所述接收目标数据块的写入请求包括：
    获取所述目标数据块对应的当前序列号；
    按照预设的计算方法，确定所述当前序列号对应的所述待写入序列号；
    所述方法包括：
    当写入成功时，根据所述待写入序列号更新所述目标数据块对应的当前序列号。
  4. 根据权利要求3所述的方法，其特征在于，所述当写入成功时，根据所述待写入序列号更新所述目标数据块对应的当前序列号包括：
    当写入成功的所述副本的数量超过预设阈值时，根据所述待写入序列号更新所述目标数据块对应的当前序列号。
  5. 根据权利要求3所述的方法，其特征在于，所述方法包括：
    接收请求端发出的目标数据块的读取请求；
    查询所述目标数据块是否存在所述最大序列号与所述当前序列号相同的所述副本；
    当存在时，将所述最大序列号与所述当前序列号相同的所述副本存储的数据返回至所述请求端。
  6. 根据权利要求1或2所述的方法，其特征在于，所述预设条件包括所述待写入序列号与所述最大序列号的差值不超过1。
  7. 一种数据的存储装置，其特征在于，所述装置包括：
    接收模块，用于接收目标数据块的写入请求，所述写入请求包括待写入数据及所述待写入数据对应的待写入序列号；
    处理模块，用于遍历所述目标数据块对应的所有副本，判断是否存在存储的最大序列号与所述待写入序列号的差值满足预设条件的所述副本，所述副本用于存储数据及每一所述数据对应的序列号；
    写入模块，用于当存在满足所述预设条件的所述副本时，将所述待写入序列号及所述待写入数据写入满足所述预设条件的所述副本。
  8. 根据权利要求7所述的存储装置，其特征在于，所述处理模块还可用于当存在不满足所述预设条件的所述副本时，根据所述待写入序列号及不满足所述预设条件的所述副本存储的所述最大序列号，确定不满足所述预设条件的所述副本的缺失的序列号；所述写入模块还可用于从满足所述预设条件的所述副本获取所述缺失的序列号对应的数据，并写入不满足所述预设条件的所述副本。
  9. 根据权利要求7或8所述的存储装置，其特征在于，所述接收模块还可用于获取所述目标数据块对应的当前序列号；按照预设的计算方法，确定所述当前序列号对应的所述待写入序列号；所述处理模块还可用于当写入成功时，根据所述待写入序列号更新所述目标数据块对应的当前序列号。
  10. 一种计算机系统，其特征在于，所述系统包括：
    一个或多个处理器；
    以及与所述一个或多个处理器关联的存储器，所述存储器用于存储程序指令,所述程序指令在被所述一个或多个处理器读取执行时，执行如下操作：
    接收目标数据块的写入请求，所述写入请求包括待写入数据及所述待写入数据对应的待写入序列号；
    遍历所述目标数据块对应的所有副本，判断是否存在存储的最大序列号与所述待写入序列号的差值满足预设条件的所述副本，所述副本用于存储数据及每一所述数据对应的序列号；
    当存在满足所述预设条件的所述副本时，将所述待写入序列号及所述待写入数据写入满足所述预设条件的所述副本。
PCT/CN2021/095198 2020-05-29 2021-05-21 一种数据的存储方法、装置及计算机系统 WO2021238798A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010476545.5A CN111796766A (zh) 2020-05-29 2020-05-29 一种数据的存储方法、装置及计算机系统
CN202010476545.5 2020-05-29

Publications (1)

Publication Number Publication Date
WO2021238798A1 true WO2021238798A1 (zh) 2021-12-02

Family

ID=72805974

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/095198 WO2021238798A1 (zh) 2020-05-29 2021-05-21 一种数据的存储方法、装置及计算机系统

Country Status (2)

Country Link
CN (1) CN111796766A (zh)
WO (1) WO2021238798A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111796766A (zh) * 2020-05-29 2020-10-20 苏宁云计算有限公司 一种数据的存储方法、装置及计算机系统
CN112395260B (zh) * 2020-11-11 2022-11-15 深圳市云网万店科技有限公司 一种数据存储方法及介质
CN114780021B (zh) * 2022-03-25 2022-11-29 北京百度网讯科技有限公司 副本修复方法、装置、电子设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104283956A (zh) * 2014-09-30 2015-01-14 腾讯科技（深圳）有限公司 强一致性分布式数据存储方法、装置及系统
CN105892954A (zh) * 2016-04-25 2016-08-24 乐视控股（北京）有限公司 基于多副本的数据存储方法和装置
CN106201788A (zh) * 2016-07-26 2016-12-07 乐视控股（北京）有限公司 用于分布式存储集群的副本修复方法及系统
US20190187921A1 (en) * 2017-12-14 2019-06-20 International Business Machines Corporation Copy source to target management in a data storage system
CN111796766A (zh) * 2020-05-29 2020-10-20 苏宁云计算有限公司 一种数据的存储方法、装置及计算机系统

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7103884B2 (en) * 2002-03-27 2006-09-05 Lucent Technologies Inc. Method for maintaining consistency and performing recovery in a replicated data storage system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104283956A (zh) * 2014-09-30 2015-01-14 腾讯科技（深圳）有限公司 强一致性分布式数据存储方法、装置及系统
CN105892954A (zh) * 2016-04-25 2016-08-24 乐视控股（北京）有限公司 基于多副本的数据存储方法和装置
CN106201788A (zh) * 2016-07-26 2016-12-07 乐视控股（北京）有限公司 用于分布式存储集群的副本修复方法及系统
US20190187921A1 (en) * 2017-12-14 2019-06-20 International Business Machines Corporation Copy source to target management in a data storage system
CN111796766A (zh) * 2020-05-29 2020-10-20 苏宁云计算有限公司 一种数据的存储方法、装置及计算机系统

Also Published As

Publication number Publication date
CN111796766A (zh) 2020-10-20

Similar Documents

Publication Publication Date Title
WO2021238798A1 (zh) 一种数据的存储方法、装置及计算机系统
JP5309043B2 (ja) ストレージシステム及びストレージシステムでの重複データ削除のための方法
JP2020091835A (ja) 情報を処理するための方法及び装置
JP2005301497A (ja) ストレージ管理装置、リストア方法及びそのプログラム
US9948520B2 (en) Efficiently determining network topology
CN111708755A (zh) 数据迁移方法、装置、系统、电子设备以及可读存储介质
CN111818145B (zh) 一种文件传输方法、装置、系统、设备及存储介质
WO2020019965A1 (zh) 数据监控方法、装置、电子设备及计算机可读存储介质
US11429311B1 (en) Method and system for managing requests in a distributed system
CN113535087A (zh) 数据迁移过程中的数据处理方法、服务器及存储系统
JP2006085539A (ja) データベース管理プログラム、方法、装置及び記録媒体
CN116540938A (zh) 数据读取方法、装置、分布式存储系统、设备和存储介质
US10198375B2 (en) Synchronization processing unit, device, and system
US11068406B2 (en) Maintaining a single copy of data within a read cache
CN112286909B (zh) 执行状态确定方法、装置、服务器及存储介质
CN115114086A (zh) 基于磁盘阵列的阵列卷恢复方法、系统、设备及存储介质
CA3150581A1 (en) Method, apparatus, and system for incremental update of graphs
US8935471B2 (en) Conditional write processing for a cache structure of a coupling facility
JP2013029911A (ja) 制御装置、制御方法およびストレージ装置
US10936429B2 (en) System and method for a fast backup operation remediation during a network disruption using a helper virtual machine
US9304699B1 (en) Managing token states across multiple storage processors of a data storage array
US9460013B2 (en) Method and system for removal of a cache agent
CN113836114B (zh) 数据迁移方法、系统、设备及存储介质
CN112579384B (zh) 一种sas域的节点监控方法、装置、系统及节点
US8074100B2 (en) Method and apparatus for restore management

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21814606

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21814606

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 21814606

Country of ref document: EP

Kind code of ref document: A1