WO2013044726A1 - Data index failure recovery method and device - Google Patents

Data index failure recovery method and device Download PDF

Info

Publication number
WO2013044726A1
WO2013044726A1 PCT/CN2012/081176 CN2012081176W WO2013044726A1 WO 2013044726 A1 WO2013044726 A1 WO 2013044726A1 CN 2012081176 W CN2012081176 W CN 2012081176W WO 2013044726 A1 WO2013044726 A1 WO 2013044726A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
index
backup
block space
stored
Prior art date
Application number
PCT/CN2012/081176
Other languages
French (fr)
Chinese (zh)
Inventor
陈晓亮
石玉清
钟超
李占军
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2013044726A1 publication Critical patent/WO2013044726A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14Error detection or correction of the data by redundancy in operation
    • G06F11/1402Saving, restoring, recovering or retrying
    • G06F11/1415Saving, restoring, recovering or retrying at system level
    • G06F11/1435Saving, restoring, recovering or retrying at system level using file system or storage system metadata

Definitions

  • the present invention relates to communication technologies, and in particular, to a data recovery fault recovery method and apparatus. Background technique
  • data is typically stored on disk and the corresponding data index is stored to make it easier to find the data it needs from disk.
  • the video data is stored in the data disk, and is specifically stored in a plurality of block spaces on the data disk; the data index corresponding to the video data is stored in a system disk on which the operating system is installed; the video monitoring system is generally 7 X 24 hours of continuous operation, the system disk will be frequently read and written for a long time, which will easily cause system disk failure, resulting in data index loss.
  • the data index is lost, the user will not be able to find the recorded data on the data disk, and the recorded data on the data disk will be lost, which will cause serious system failure and cause huge loss to the user.
  • the currently used remedy for system disk failure is to take a video surveillance system as an example, read the video data on the data disk, and gradually analyze and reverse the data index according to the video data, so that the new data is replaced. After the system disk, you can restore the data index to the new system disk and continue to use it.
  • this method must traverse all the recorded data on all data disks to fully recover the data index corresponding to the data; when the amount of data on the data disk is extremely large, the recovery method may take a lot of time. Long, at the same time, during the data index recovery process, the video surveillance service cannot be started, and the service will be interrupted for a long time. Summary of the invention
  • the present invention provides a data index fault recovery method, the data index is used to identify storage location information of stored data in at least one data disk, and the data disk includes at least one block space for placing the stored data;
  • the fault recovery method includes: scanning a head reserved area of the block space, and acquiring a backup index stored in the head reserved area, where the backup index includes at least a data identifier of the stored data in the block space. And storage address information of the stored data in the block space; generating a data index, the data index including at least the backup index.
  • the present invention provides a data indexing fault recovery apparatus, including: a backup obtaining unit, configured to scan a head reserved area of a block space in a data disk, and obtain a backup index stored in the head reserved area,
  • the backup index includes at least a data identifier of the storage data in the block space and storage address information of the storage data in the block space; wherein the storage data is stored on at least one data disk, and each The data disk includes at least one block space for placing the storage data;
  • an index recovery unit configured to generate a data index that identifies storage location information of the storage data in the at least one data disk, the data index including at least the Backup index.
  • the data recovery fault recovery method and device of the present invention can quickly restore the data index according to the backup index by storing the backup index in the block space of the data disk, thereby solving the data index recovery time. Longer problems have greatly improved the speed of data index recovery. DRAWINGS
  • FIG. 1 is a schematic flowchart of an embodiment of a method for recovering a data index according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram of an application scenario of another method for recovering a data index according to another embodiment of the present invention
  • FIG. 3 is a schematic flowchart of another method for recovering a data index according to another embodiment of the present invention
  • FIG. 5 is a schematic structural diagram of an embodiment of a data recovery fault recovery apparatus according to the present invention.
  • the data index is used to identify the storage location of the storage data on the data disk.
  • Information where the data on the data disk can be called storage data
  • the data disk includes at least one block space for placing the stored data, and the block space is a fixed-size space pre-allocated on the data disk. Users can get the storage location of the data according to the data index, so that the required data can be quickly found.
  • the video surveillance system is taken as an example in the embodiment of the present invention, but it should be understood by those skilled in the art that the embodiments of the present invention are equally applicable to other storage systems.
  • FIG. 1 is a schematic flowchart of a method for recovering a data index according to an embodiment of the present invention. As shown in FIG. 1, the method may include:
  • each data disk and the block space may be scanned one by one in order.
  • the block space is stored in the block space, and the head reserved area is usually set in the block space, but the current utilization rate is still low, and there is still excess space; for example, the size of the block space is generally 1M. Actually, only about 20 bytes of space are actually used for placing information indicating the camera to which the block space belongs.
  • a backup index is placed in the head reserved area.
  • the backup index includes at least a data identifier of the stored data placed in the block space and storage address information of the stored data in the block space.
  • the stored data is recorded data, and the data identifier of the stored data is used to distinguish the stored data from other stored data, for example, in a block space of the data disk.
  • the backup index is generated according to the data index after storing the data and generating the data index corresponding to the data. For example, if 12 to 3 minutes and 5 seconds of video data is stored in the 11th to 28th bytes in the block space a in the data disk A, the data index corresponding to the video data generated thereby may include 12:3:5, data disk A, block space a, 1st byte (start address), 28th byte (end address), etc., and the data index indicates storage location information of video data.
  • the backup index and the data index may be the same or different; for example, the data index may be directly used as backup data; or only part of the information in the data index may be extracted as a backup index, but At least 12:3:5, the 11th byte (starting address), the 28th byte (end address) should be included; and the information of the data disk and the block space a can also be in the process of scanning the block space of the data disk. Get it in.
  • the data index can be obtained according to the backup index.
  • the backup index includes only 12 points, 3 minutes, 5 seconds, 11th byte (starting address), and 28th byte. (End address); when recovering the data index according to the recovery, in particular, in the process of scanning and obtaining the backup index, the block space identifier (block space a) of the block space where the backup index is located, and the block space can be obtained.
  • the data disk identifier of the data disk (data disk A), which belongs to the attribute information of the data disk, and is available when scanning the data disk.
  • the backup index can be directly used as the data index in the 102.
  • the data index may further include other information such as an index identification item, which is used to identify whether the index is valid, but when the 102 is executed, the index is generally considered to be valid.
  • the storage address information of the data in the block space can be obtained only by scanning all the stored data in the block space.
  • the backup index includes the data identifier of the stored data and the storage address information of the stored data in the block space, so that when the data index is restored, it is not necessary to scan all the stored data in the entire block space, only
  • the above information can be obtained by scanning the reserved area of the head where the backup cable is located, thereby greatly reducing the scanning time and improving the efficiency of data index recovery.
  • the head reserved area occupied by the backup data in the block space usually has a small space, and the data disk only pays a small space cost, but in exchange for a significant reduction in the failure recovery time, making the time efficiency in the video monitoring system And the space efficiency is well balanced.
  • this embodiment distributes the backup index in each block space of the data disk, and the backup index in the block space corresponds to the data stored in the block space, so that even if the data disk is partially damaged, If the data in a block space and its corresponding index are not damaged, the data in the block space can be restored, thereby greatly improving the security of data storage relative to the manner of centrally storing all the indexes of the entire data disk.
  • the backup index is stored in the head of each block space in a distributed manner, and the data is more secure than the manner in which the data is stored in the data disk, for example, a dedicated folder is placed to place the files. Because after a storage area in the backup index set, if there is damage or data loss, all data may not be recovered. In addition, the way of centralized storage, if you need to regularly prepare these indexes The gap may be relatively large, for example, it is difficult to perform real-time backup by backing up every hour. However, the distributed storage method of the embodiment of the present invention can greatly reduce the time gap.
  • the data index can be quickly restored according to the backup index, thereby solving the problem that the data index recovery time is long, and the data index is greatly improved. Recovery speed.
  • the data recovery of the video surveillance system is taken as an example to describe the fault recovery method of the present invention.
  • the fault recovery method of the present invention is not limited to the field of video surveillance, and can be applied to all applications requiring special protection of critical data, which can improve data security and fault recovery speed.
  • a disk ⁇ disk N represents a data disk for storing video data; an operating system and a database are installed on the system disk. And the system disk further stores a data index for searching the recorded data on the data disk; the storage server can receive the data stream of the recorded data, and store the data through the system disk to the data disk.
  • the method of this embodiment may be performed by the storage server.
  • FIG. 3 is a schematic flowchart of another embodiment of a method for recovering a data index according to the present invention. As shown in FIG. 3, the method may include:
  • the storage server puts the stored data into a block space of the data disk, and generates a data index corresponding to the stored data.
  • the stored data refers to the recorded data; the stored data is stored in blocks on the data disk, and the data disk may be configured with a plurality of block spaces, each of which has a size of 1 GB. When the recording data of a block space is full, it will automatically switch to another free block space to continue storage.
  • the data of the data is usually generated when the data is stored. index.
  • the storage server in the embodiment may simultaneously generate a corresponding data index in the system disk, and the data index may include a data identifier, storage location information of the data in the data disk, and the like.
  • FIG. 4 is a schematic diagram of another embodiment of a method for recovering data index of the present invention.
  • data disk A is provided with n block spaces, where n is stored in block space a.
  • the recording data of the second; the data index initially generated by the n seconds of recording data includes n indexes corresponding to the recording data of each second, for example, the data index of the first second recording data shown in FIG. 4,
  • the data index of the second second recorded data, etc., the n data indexes have the same format, including multiple index items, and may include, for example, time, data disk number, block space number, start address, end address, and index identifier. (i sD ⁇ my ), reserved fields, etc.
  • the time is the recording time of the recorded data, such as 12:3:5 on January 1, 2011, as the data identifier of the recorded data;
  • the storage location information may include the data disk number, the block space number, the start address, and the end address.
  • the data disk number is a data disk identifier
  • the block space number is a block space identifier on the data disk
  • the start address and the end address are physical storage addresses of the data in the block space
  • the storage location information may be It is the 11th byte to the 28th byte of the block space a on the data disk A.
  • the index identification item is an identifier for identifying whether the index is valid. For example, "1" can be set to be valid, that is, the corresponding recording data can be found by using the index, and "0" means invalid; the reserved field is used for future expansion. In the case of no use, you can first fill in 0.
  • the data disk may be stored once at a time; for example, after receiving the recorded data, the storage server may first store the data in the cache, and then pass through, for example, a buffer of 4 seconds. After that, the data in this period is uniformly stored in the data disk, and a data index is generated, and the data index may be corresponding to the data per second, for example, the data index of the first second recorded data, and the second second recorded data. Data indexing, etc.
  • the storage server obtains a backup index according to the data index, and stores the backup index in a reserved area of the head space of the block space;
  • the storage index may be obtained according to the data index, and the backup index is stored in the head reserved area of the block space.
  • the backup index is to back up the data index, and the data generated in 301 can be directly used as a backup index, or only part of the content can be selected as a backup cable); whether all backups or partial backups can be selected specifically According to the actual situation, for example, according to the code stream speed, the lower the code stream speed, the longer the time required to fill a block space. If it is necessary to record the data index corresponding to the recording data per second, then the number of data indexes. The more the block space area occupied by the data index is, and the head reserved area of the block space is certain. If the data is still backed up by the entire contents of the data index, it is possible that the block space is pre-predicted.
  • the data index in the backup index pair 301 of the embodiment is simplified, and part of the content is selected for backup; the backup index includes at least a data identifier of the stored data in the block space, and storage of the stored data in the block space.
  • Address information the storage address information including a start address and an end address stored in the block space.
  • At least the above information is included because it takes a long time to recover the data index in the prior art because the block space of the data disk needs to be completely Scanning to determine the specific physical storage address of the data in the block space, which is the main reason for time-consuming. Therefore, it is necessary to save the storage address information of the data in the block space during backup, and it is no longer needed when index recovery is performed. For the full scan of the block space; for the data disk number and the block space number, this is some attribute information recorded by the data disk itself, which can be easily obtained when scanning the data disk.
  • an index identifier item may also be included.
  • the role of the index identifier item is as follows. Assume that there is a video recording for one hour, one record is stored every second, and a total of 3,600 records, for example, 1800 of the 3,600 records are valid. Recording, if another 1800 is invalid recording, the index identification item corresponding to the valid record can be set to 1, and the index identification item corresponding to the invalid record can be set to 0; when performing backup, only the valid record, that is, the above 1800, can be backed up. At this time, the index identification item may or may not be backed up.
  • the index identification item must be restored to distinguish between the valid record and the invalid record, because If the recovery is to be restored to 3600 records, the records that are restored from the backup are valid. You need to set the index identification item for identification. In addition, the restored records are invalid, and the index identification items are also required for identification. Reserved fields are not required.
  • the occupied space of the index item "time” is also reduced.
  • “Time” occupies 8 bytes in the prior art, which belongs to a redundant design, and actually only needs 4 bytes, so
  • the occupied space of "time” is designed to be 4 bytes; the start address and the end address are consistent with the original, each occupying 4 bytes.
  • the backup index of this embodiment has a size of only 12 bytes after the above-mentioned simplification, and the occupied space is reduced by 50% compared with the prior art 24-byte data index, which greatly saves the occupied block space.
  • This embodiment stores the backup index in the header reserved area of the block space.
  • a small portion of space is reserved in the header area of each block space, which is called a head reserved area; corresponding to the block space
  • the backup index of the stored data in the storage area can be stored in the header reserved area.
  • the recorded data from the first second to the nth second is stored, and in the header reserved area of the block space a, the recorded data corresponding to the above from the first second to the nth second is stored.
  • Backup index including the first second backup index, the second second backup index ... until the nth second.
  • the space occupied by the backup index is very small. For example, according to the minimum 256Kb code stream of standard definition video, the backup index space required for 1 GB of video data is only 384 KB, which accounts for about 4/4 of the total storage capacity of the block space.
  • the backup index corresponding to the stored data is stored in the reserved area of the head space of the block space.
  • the storage server may store the data once in the block space every 4 seconds, and generate a data index and a backup index corresponding to the stored data.
  • the backup index corresponding to the data stored in the block space may be used. It is placed in the head reserved area of the block space. This can reduce the frequent operation of the data disk and extend the life of the data disk.
  • the storage server scans a head reserved area of the block space, and obtains a backup index stored in the reserved area of the header.
  • the storage server scans the header reserved area of the block space, and obtains a backup index stored in the reserved area of the header.
  • the backup index includes at least a time, a start address, and an end address.
  • the data disk number and the block space number are also required.
  • the above two index items may be obtained during the scanning process;
  • the block space number is the sequence number of the block space currently being restored, and may be from the data disk attribute information. Obtained in the block space file name;
  • the data disk number is the disk serial number currently being processed, and is incremented according to the scanning order of the disk.
  • the index identifier i sDummy can be considered to be valid when index recovery is performed, and is set to a valid identifier such as "1"; the reserved field for future expansion can be 3 bytes, which is temporarily useless. You can fill in all 0 when recovering. At this point, the backup index has been restored to the original data index, and the storage server writes the restored data index to the new system disk, which can be applied to subsequent data lookups.
  • the backup index by storing the backup index in the block space, when the index recovery is performed, only the head reserved area of all the block spaces needs to be scanned once, and the data index can be restored to the system disk, and there is no need to traverse all the data.
  • Disk data, and the backup index occupies a small amount of block space, usually only a few hundred KB, compared to the prior art full scan recovery method, the recovery time can be shortened from the previous 2 days to a few minutes;
  • the backup index takes up less space, and the scan recovery time is further shortened, which greatly saves the time required to restore the data index, reduces service interruption time, and improves data security and integrity. Add extra protection for security and disaster tolerance.
  • the backup index may be stored in a data storage space outside the reserved area of the head space of the block space, where the data storage space is The storage data corresponding to the backup index is stored, and the backup index and the corresponding storage data are encapsulated and stored.
  • the area outside the reserved area of the head is a data storage space, and the video storage space stores video data of multiple frames.
  • the embodiment is encapsulated, and the encapsulation refers to adding recovery information for restoring the part of the original data on the basis of the original data, and the original data and the recovery information are combined.
  • the storage server can scan each frame of data in the block space and obtain a backup index other than the original data in each frame, according to which the original data index can be restored.
  • frame header information, frame tail information, and data length and check information may also be set in the frame, and the above information is used to check whether the frame data is damaged; for example, it is inevitable to add a data length from the beginning of the frame. The end of the frame is obtained, otherwise the frame data can be invalidated.
  • the security of the data protection is further greatly improved, so that even if the reserved area of the block space header is damaged, as long as the stored data is not damaged, for example, the above-mentioned frame data is not damaged. , you can restore the frame data.
  • the data index in the system disk is further provided with the database shown in FIG. 2, where the database stores a statistical index for the data index, and the statistical index includes The data identification range of the stored data, and the storage location information of the data identification range in at least one data disk.
  • the recorded data of the first second to the tenth second is stored, and the first second recorded data index and the second second recorded data index are recorded in the data index of the system disk.
  • the 10 second recording is stored separately every second; however, it is recorded in the database that the recording data of the first second to the tenth second is stored in the block space a of the data disk A, and is statistical information.
  • the user searches for the recorded data of the data disk, for example, the target data searched by the user is the recorded data of the 6th second, it can first search the database, and know that the recorded data of the first second to the tenth second is stored in the block of the data disk A.
  • the data index, recorded in the data index is a detailed record of each second of the recorded data stored in the block space, from which the user can find the 6th second of the recorded data is stored in the 11th byte of the block space a to The location of the 20th byte, so the user can get the target data accordingly.
  • the statistical index data in the above database is also lost.
  • the database information can be recovered together, and the data index obtained by the recovery is counted to obtain a statistical index. Information, and store the statistical index in the database.
  • the data index can be quickly restored according to the backup index, thereby solving the problem that the data index recovery time is long, and the data index is greatly improved. Recovery speed.
  • FIG. 5 is a schematic structural diagram of an embodiment of a data recovery fault recovery apparatus according to the present invention.
  • the apparatus may be the storage server shown in FIG. 2 or one of the functional units provided in the storage server.
  • the apparatus may perform any of the present invention.
  • the apparatus may include a backup acquisition unit 51 and an index recovery unit 52.
  • the backup obtaining unit 51 is configured to scan a head reserved area of the block space in the data disk, and obtain a backup index stored in the head reserved area, where the backup index includes at least the block space.
  • the index restoring unit 52 is configured to generate a data index that identifies storage location information of the stored data in the at least one data disk, where the data index includes at least the backup index.
  • the device may further include an index generating unit 53 and an index backup unit 54.
  • the index generating unit 53 is configured to put the stored data into the block space of the data disk, and generate the data index corresponding to the stored data.
  • the index backup unit 54 is configured to obtain a backup according to the data index. Indexing, and storing the backup index in a header reserved area of the block space.
  • the index backup unit 54 is configured to store, in the header reserved area of the block space, a backup index corresponding to the stored data when the storage data is placed in each of the block spaces.
  • the index backup unit 54 is specifically configured to extract, from the data index, a data identifier corresponding to the storage data in the block space, and storage address information of the storage data in the block space as the backup index;
  • the index recovery unit 52 is configured to: obtain, by scanning a block space in the data disk, a block identifier of a block space where the backup index is located, and a data disk identifier of a data disk where the block space is located; At least the data index of the backup index, the block space identifier, and the data disk identifier is included.
  • the index backup unit 55 of the embodiment is further configured to store the backup index in a data storage outside the head reserved area of the block space after obtaining the backup index according to the data index.
  • a storage area corresponding to the backup index is stored in the data storage space; and the backup index and the corresponding storage data are encapsulated and stored.
  • the data recovery fault recovery device of the embodiment can quickly restore the data index according to the backup index by storing the backup index in the block space of the data disk, thereby solving the problem that the data index recovery time is long, and the data index is greatly improved. Recovery speed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
  • the components displayed by the unit may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium.
  • the technical solution of the present invention may contribute to the prior art or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium.
  • a number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes: a USB flash drive, a removable hard disk, a read-only memory (ROM), A medium that can store program codes, such as random access memory (RAM, Random Acce ss Memory), disk, or optical disk.
  • the foregoing program may be stored in a computer readable storage medium, and the program is executed when executed.
  • the steps of the foregoing method embodiments are included; and the foregoing storage medium includes: a medium that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Library & Information Science (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Signal Processing For Digital Recording And Reproducing (AREA)
  • Management Or Editing Of Information On Record Carriers (AREA)

Abstract

Provided are a data index failure recovery method and device, the method comprising: scanning the overhead reserved region of a block space to obtain a backup index stored in the overhead reserved region, the backup index comprising at least a data identifier of the data stored in the block space and the storage address information of the data stored in the block space; and generating a data index, the data index comprising at least the backup index. By storing the backup index in the block space of a data disk, the data index failure recovery method and device of the present invention can quickly recover the data index according to the backup index, and solves the problem of long data index recovery time, thus greatly improving data index recovery speed.

Description

数据索引的故障恢复方法和装置  Method and device for recovering data index
本申请要求于 2011 年 9 月 28 日提交中国专利局、 申请号为 201110297834. X,发明名称为 "数据索引的故障恢复方法和装置,,的中国 专利申请的优先权, 其全部内容通过引用结合在本申请中。 This application claims to be filed on September 28, 2011, the Chinese Patent Office, Application No. 201110297834. X, the invention titled "Data Indexing Fault Recovery Method and Apparatus," the priority of the Chinese patent application, the entire contents of which are incorporated by reference. In this application.
技术领域 Technical field
本发明涉及通信技术, 特别涉及一种数据索引的故障恢复方法和装置。 背景技术  The present invention relates to communication technologies, and in particular, to a data recovery fault recovery method and apparatus. Background technique
在数据存储中,通常是将数据存储在磁盘上, 并存储对应的数据索引以方 便从磁盘上查找需要的数据。 例如, 在视频监控系统中, 录像数据存放在数据 盘, 具体存放在该数据盘上的多个块空间中; 录像数据对应的数据索引存放在 安装操作系统的系统盘; 视频监控系统一般都是 7 X 24小时不间断运行,会对 系统盘长时间频繁读写, 极易造成系统盘故障, 进而导致数据索引丟失。 由此 造成的后果是, 一旦数据索引丟失, 用户将无法查找数据盘上的录像数据, 等 同于数据盘上的录像数据丟失,从而会引起系统严重故障,给用户造成巨大损 失。  In data storage, data is typically stored on disk and the corresponding data index is stored to make it easier to find the data it needs from disk. For example, in the video surveillance system, the video data is stored in the data disk, and is specifically stored in a plurality of block spaces on the data disk; the data index corresponding to the video data is stored in a system disk on which the operating system is installed; the video monitoring system is generally 7 X 24 hours of continuous operation, the system disk will be frequently read and written for a long time, which will easily cause system disk failure, resulting in data index loss. As a result, once the data index is lost, the user will not be able to find the recorded data on the data disk, and the recorded data on the data disk will be lost, which will cause serious system failure and cause huge loss to the user.
现有技术中, 目前常用的针对系统盘故障的补救方法是, 以视频监控系统 为例,读取数据盘上的录像数据,根据该录像数据逐步分析并反推出数据索引, 以使得在更换新的系统盘后, 可以将数据索引恢复到新的系统盘上继续使用。 但是, 这种方法必须将所有数据盘上的录像数据全部遍历一次, 才能完整恢复 出对应该数据的数据索引; 当数据盘上的数据量极大时, 恢复方法可能耗时很 长, 同时, 在数据索引的恢复过程中, 视频监控服务不能启动, 业务会长时间 中断。 发明内容 In the prior art, the currently used remedy for system disk failure is to take a video surveillance system as an example, read the video data on the data disk, and gradually analyze and reverse the data index according to the video data, so that the new data is replaced. After the system disk, you can restore the data index to the new system disk and continue to use it. However, this method must traverse all the recorded data on all data disks to fully recover the data index corresponding to the data; when the amount of data on the data disk is extremely large, the recovery method may take a lot of time. Long, at the same time, during the data index recovery process, the video surveillance service cannot be started, and the service will be interrupted for a long time. Summary of the invention
本发明的目的是提供一种数据索引的故障恢复方法和装置,以实现快速恢 复系统盘上的数据索引。 本发明提供一种数据索引的故障恢复方法,所述数据索引用于标识存储数 据在至少一个数据盘的存储位置信息,所述数据盘包括至少一个用于放置所述 存储数据的块空间; 所述故障恢复方法包括: 扫描所述块空间的头部预留区域,获取存储在所述头部预留区域中的备份 索引, 所述备份索引至少包括所述块空间中的存储数据的数据标识、 以及所述 存储数据在所述块空间中的存储地址信息; 生成数据索引, 所述数据索引至少包括所述备份索引。 本发明提供一种数据索引的故障恢复装置, 包括: 备份获取单元, 用于扫描数据盘中的块空间的头部预留区域, 并获取存储 在所述头部预留区域中的备份索引,所述备份索引至少包括所述块空间中的存 储数据的数据标识、以及所述存储数据在所述块空间中的存储地址信息;其中, 所述存储数据存储在至少一个数据盘上,且每一所述数据盘包括至少一个用于 放置所述存储数据的块空间; 索引恢复单元,用于生成标识存储数据在至少一个数据盘的存储位置信息 的数据索引, 所述数据索引至少包括所述备份索引。 本发明的数据索引的故障恢复方法和装置,通过在数据盘的块空间中存储 备份索引, 可以根据该备份索引快速恢复数据索引,解决了数据索引恢复时间 较长的问题, 大大提高了数据索引的恢复速度。 附图说明 It is an object of the present invention to provide a data recovery fault recovery method and apparatus for enabling fast recovery of data indexing on a system disk. The present invention provides a data index fault recovery method, the data index is used to identify storage location information of stored data in at least one data disk, and the data disk includes at least one block space for placing the stored data; The fault recovery method includes: scanning a head reserved area of the block space, and acquiring a backup index stored in the head reserved area, where the backup index includes at least a data identifier of the stored data in the block space. And storage address information of the stored data in the block space; generating a data index, the data index including at least the backup index. The present invention provides a data indexing fault recovery apparatus, including: a backup obtaining unit, configured to scan a head reserved area of a block space in a data disk, and obtain a backup index stored in the head reserved area, The backup index includes at least a data identifier of the storage data in the block space and storage address information of the storage data in the block space; wherein the storage data is stored on at least one data disk, and each The data disk includes at least one block space for placing the storage data; an index recovery unit, configured to generate a data index that identifies storage location information of the storage data in the at least one data disk, the data index including at least the Backup index. The data recovery fault recovery method and device of the present invention can quickly restore the data index according to the backup index by storing the backup index in the block space of the data disk, thereby solving the data index recovery time. Longer problems have greatly improved the speed of data index recovery. DRAWINGS
为了更清楚地说明本发明实施例或现有技术中的技术方案, 下面将对 实施例或现有技术描述中所需要使用的附图作一简单地介绍,显而易见地, 下 面描述中的附图是本发明的一些实施例,对于本领域普通技术人员来讲, 在不 付出创造性劳动的前提下, 还可以根据这些附图获得其他的附图。  In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, a brief description of the drawings used in the embodiments or the prior art description will be briefly made. Obviously, the drawings in the following description It is a certain embodiment of the present invention, and other drawings can be obtained from those skilled in the art without any creative work.
图 1为本发明数据索引的故障恢复方法一实施例的流程示意图;  1 is a schematic flowchart of an embodiment of a method for recovering a data index according to an embodiment of the present invention;
图 2为本发明数据索引的故障恢复方法另一实施例的应用场景示意图; 图 3为本发明数据索引的故障恢复方法另一实施例的流程示意图; 图 4为本发明数据索引的故障恢复方法另一实施例的原理示意图; 图 5为本发明数据索引的故障恢复装置实施例的结构示意图。 具体实施方式  2 is a schematic diagram of an application scenario of another method for recovering a data index according to another embodiment of the present invention; FIG. 3 is a schematic flowchart of another method for recovering a data index according to another embodiment of the present invention; A schematic diagram of another embodiment of the present invention; FIG. 5 is a schematic structural diagram of an embodiment of a data recovery fault recovery apparatus according to the present invention. detailed description
为使本发明的目的、技术方案和优点更加清楚, 下面将结合本发明实施例 中的附图, 对本发明实施例中的技术方案进行清楚、 完整地描述, 显然, 所描 述的实施例是本发明一部分实施例, 而不是全部的实施例。基于本发明中的实 施例,本领域普通技术人员在没有做出创造性劳动的前提下所获得的所有其他 实施例, 都属于本发明保护的范围。  The technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Some embodiments, rather than all of the embodiments, are invented. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.
为使得下面的本发明实施例对数据索引故障恢复方法的说明更加清楚,首 先对实施例中所提到的几个概念进行说明如下:数据索引是用于标识存储数据 在数据盘上的存储位置信息的, 其中, 数据盘上的数据可以称为存储数据, 所 述的数据盘包括至少一个用于放置所述存储数据的块空间 ,块空间是在数据盘 上预先分配好的固定大小的空间。用户可以根据数据索引,得到数据的存储位 置, 从而可以快速查找到所需的数据。 In order to make the description of the data index failure recovery method clearer in the following embodiments of the present invention, firstly, several concepts mentioned in the embodiments are described as follows: The data index is used to identify the storage location of the storage data on the data disk. Information, where the data on the data disk can be called storage data, The data disk includes at least one block space for placing the stored data, and the block space is a fixed-size space pre-allocated on the data disk. Users can get the storage location of the data according to the data index, so that the required data can be quickly found.
为了方便说明, 本发明实施例以视频监控系统为例,但本领域人员应该知 道, 本发明实施例同样可以应用于其他存储系统。  For convenience of description, the video surveillance system is taken as an example in the embodiment of the present invention, but it should be understood by those skilled in the art that the embodiments of the present invention are equally applicable to other storage systems.
实施例一  Embodiment 1
图 1为本发明数据索引的故障恢复方法一实施例的流程示意图, 如图 1所 示, 该方法可以包括:  FIG. 1 is a schematic flowchart of a method for recovering a data index according to an embodiment of the present invention. As shown in FIG. 1, the method may include:
101、 扫描数据盘上的块空间的头部预留区域, 获取存储在头部预留区域 中的备份索引;  101. Scan a head reserved area of a block space on the data disk, and obtain a backup index stored in a reserved area of the header;
其中, 需要对放置有存储数据的各个数据盘的块空间进行扫描; 具体实施 中, 可以按序逐个扫描每个数据盘、 以及块空间。  Wherein, the block space of each data disk on which the stored data is stored needs to be scanned; in the specific implementation, each data disk and the block space may be scanned one by one in order.
所述的块空间中存储有头部预留区域,该头部预留区域通常在块空间中都 会设置, 只是目前利用率还较低, 尚有多余空间; 例如, 块空间的大小一般为 1M, 实际只利用了其中的大约 20字节的空间, 用于放置表示该块空间所属的摄 像机等信息。本实施例在该头部预留区域中放置了备份索引。该备份索引至少 包括该块空间中放置的存储数据的数据标识、以及存储数据在块空间中的存储 地址信息。通过将备份索引设置在块空间的头部区域, 可以在对数据盘进行扫 描时快速获取备份索引, 提高索引恢复的效率。  The block space is stored in the block space, and the head reserved area is usually set in the block space, but the current utilization rate is still low, and there is still excess space; for example, the size of the block space is generally 1M. Actually, only about 20 bytes of space are actually used for placing information indicating the camera to which the block space belongs. In this embodiment, a backup index is placed in the head reserved area. The backup index includes at least a data identifier of the stored data placed in the block space and storage address information of the stored data in the block space. By setting the backup index in the header area of the block space, you can quickly obtain the backup index when scanning the data disk, improving the efficiency of index recovery.
例如, 以视频监控系统为例, 所述的存储数据为录像数据, 所述的存储数 据的数据标识是用于将该条存储数据与其他存储数据区分开的, 例如,数据盘 的块空间中存储有某 1个小时内的所有录像视频, 并且是存储有对应于其中每 一秒的录像视频, 共 3600条存储数据, 则为了区分所有数据, 可以定义数据标 识为录像视频所对应的时间(如 2011年 1月 1日 12点 3分 5秒), 则可以将所述的 3600条存储数据区分开;所述的存储数据在块空间中的存储地址信息指的是在 块空间中的物理存储地址(如设置在块空间的第 1 1字节至第 28字节) 。 For example, in the video surveillance system, the stored data is recorded data, and the data identifier of the stored data is used to distinguish the stored data from other stored data, for example, in a block space of the data disk. Stores all recorded videos within a certain hour, and is stored with each corresponding to each One second of video recording, a total of 3,600 pieces of stored data, in order to distinguish all data, you can define the time corresponding to the data identification as the video (such as January 1, 2011, 12:3:5), then you can The 3600 pieces of storage data are separated; the storage address information of the stored data in the block space refers to a physical storage address in the block space (such as the 1st to 28th bytes set in the block space) .
其中, 该备份索引是在存储数据并生成数据对应的数据索引之后,根据数 据索引生成的。例如,将 12点 3分 5秒的视频数据存储在了数据盘 A中的块空间 a 中的第 11字节至第 28字节,则据此生成的与该视频数据对应的数据索引可以包 括 12点 3分 5秒、 数据盘 A、 块空间 a、 第 1 1字节(起始地址) 、 第 28字节(结束 地址)等, 该数据索引表示视频数据的存储位置信息。  The backup index is generated according to the data index after storing the data and generating the data index corresponding to the data. For example, if 12 to 3 minutes and 5 seconds of video data is stored in the 11th to 28th bytes in the block space a in the data disk A, the data index corresponding to the video data generated thereby may include 12:3:5, data disk A, block space a, 1st byte (start address), 28th byte (end address), etc., and the data index indicates storage location information of video data.
在根据上述数据索引生成备份索引时,备份索引与数据索引可以相同也可 以不同; 例如, 可以直接将所述的数据索引作为备份数据; 也可以仅提取数据 索引中的部分信息作为备份索引, 但至少应包括 12点 3分 5秒、 第 11字节(起始 地址) 、 第 28字节(结束地址); 而数据盘 、 块空间 a这些信息也可以在上述 扫描数据盘的块空间的过程中获取得到。  When the backup index is generated according to the foregoing data index, the backup index and the data index may be the same or different; for example, the data index may be directly used as backup data; or only part of the information in the data index may be extracted as a backup index, but At least 12:3:5, the 11th byte (starting address), the 28th byte (end address) should be included; and the information of the data disk and the block space a can also be in the process of scanning the block space of the data disk. Get it in.
102、 生成数据索引, 所述数据索引至少包括所述备份索引;  102. Generate a data index, where the data index includes at least the backup index.
其中, 在 1 01中扫描获取到备份索引后, 可以根据该备份索引得到数据索 引。 例如, 若仅提取数据索引中的部分信息作为备份索引时, 如 101中的举例 所述, 备份索引中仅包括 12点 3分 5秒、 第 11字节(起始地址)、 第 28字节(结 束地址); 则在据此恢复生成数据索引时, 具体是在扫描获取备份索引的过程 中, 就可以得到该备份索引所在的块空间的块空间标识(块空间 a ) 、 以及该 块空间所在的数据盘的数据盘标识(数据盘 A ) , 这些信息属于数据盘的属性 信息,在对数据盘进行扫描时是可以获取到的。可以将扫描获取的上述备份索 引、 块空间标识、 数据盘标识等信息组合而成数据索引。 After the scan index is obtained in 01, the data index can be obtained according to the backup index. For example, if only part of the information in the data index is extracted as a backup index, as described in the example in 101, the backup index includes only 12 points, 3 minutes, 5 seconds, 11th byte (starting address), and 28th byte. (End address); when recovering the data index according to the recovery, in particular, in the process of scanning and obtaining the backup index, the block space identifier (block space a) of the block space where the backup index is located, and the block space can be obtained. The data disk identifier of the data disk (data disk A), which belongs to the attribute information of the data disk, and is available when scanning the data disk. The above backups that can be obtained by scanning Information such as reference, block space identifier, and data disk identifier are combined to form a data index.
例如, 若直接将数据索引作为备份数据, 则在该 1 02中可以直接将备份索 引作为数据索引。 该数据索引中还可以包括索引标识项等其他信息, 该索引标 识项用于标识索引是否有效, 不过在执行该 102时, 通常认为索引有效。  For example, if the data index is directly used as backup data, the backup index can be directly used as the data index in the 102. The data index may further include other information such as an index identification item, which is used to identify whether the index is valid, but when the 102 is executed, the index is generally considered to be valid.
与现有技术相比, 现有技术中只有通过扫描块空间中的所有存储数据, 才 可以得到数据在块空间中的存储地址信息, 而本实施例中, 通过在块空间中存 储备份索引, 并且该备份索引中包括了存储数据的数据标识、 以及存储数据在 块空间中的存储地址信息,使得在进行数据索引的恢复时, 可以不需要再扫描 整个块空间中的所有的存储数据,仅仅扫描该备份索弓 )所在的头部预留区域即 可得到上述信息, 从而大大减少了扫描时间, 提高了数据索引恢复的效率。 而 且,备份数据在块空间中所占用的头部预留区域通常空间很小,数据盘仅付出 了极少的空间代价, 却换取了故障恢复时间的大幅减少,使得视频监控系统中 的时间效率和空间效率得到很好的平衡。  Compared with the prior art, in the prior art, the storage address information of the data in the block space can be obtained only by scanning all the stored data in the block space. In this embodiment, by storing the backup index in the block space, And the backup index includes the data identifier of the stored data and the storage address information of the stored data in the block space, so that when the data index is restored, it is not necessary to scan all the stored data in the entire block space, only The above information can be obtained by scanning the reserved area of the head where the backup cable is located, thereby greatly reducing the scanning time and improving the efficiency of data index recovery. Moreover, the head reserved area occupied by the backup data in the block space usually has a small space, and the data disk only pays a small space cost, but in exchange for a significant reduction in the failure recovery time, making the time efficiency in the video monitoring system And the space efficiency is well balanced.
此外, 本实施例通过将备份索引分散存储在数据盘的各个块空间中, 并且 所述块空间中的备份索引是与该块空间中存储的数据对应的,使得即使数据盘 发生部分损坏, 只要某个块空间中的数据及其对应的索引未损坏, 就可以恢复 该块空间中的数据,从而相对于集中存放整个数据盘的所有索引的方式, 大大 提高了数据存储的安全性。  In addition, this embodiment distributes the backup index in each block space of the data disk, and the backup index in the block space corresponds to the data stored in the block space, so that even if the data disk is partially damaged, If the data in a block space and its corresponding index are not damaged, the data in the block space can be restored, thereby greatly improving the security of data storage relative to the manner of centrally storing all the indexes of the entire data disk.
本发明实施例中,备份索引分散地存储于各个块空间的头部,相较于集中 存放在数据磁盘中的方式, 例如专门开一个文件夹来放置这些文件, 其数据安 全性更高。 因为备份索引集中某一存储区域后, 一旦出现损坏或数据丟失, 可 能造成所有数据无法恢复。 另外, 集中存储的方式如果需要定期将这些索引备 份, 间隙会比较大, 比如每小时备份一次, 难以做到实时备份, 而本发明实施 例分散式存储方法, 可以大大缩小这个时间间隙。 In the embodiment of the present invention, the backup index is stored in the head of each block space in a distributed manner, and the data is more secure than the manner in which the data is stored in the data disk, for example, a dedicated folder is placed to place the files. Because after a storage area in the backup index set, if there is damage or data loss, all data may not be recovered. In addition, the way of centralized storage, if you need to regularly prepare these indexes The gap may be relatively large, for example, it is difficult to perform real-time backup by backing up every hour. However, the distributed storage method of the embodiment of the present invention can greatly reduce the time gap.
本实施例的数据索引的故障恢复方法,通过在数据盘的块空间中存储备份 索引, 可以根据该备份索引快速恢复数据索引,解决了数据索引恢复时间较长 的问题, 大大提高了数据索引的恢复速度。  In the data recovery fault recovery method of the embodiment, by storing the backup index in the block space of the data disk, the data index can be quickly restored according to the backup index, thereby solving the problem that the data index recovery time is long, and the data index is greatly improved. Recovery speed.
实施例二  Embodiment 2
本实施例以视频监控系统的数据索引恢复为例,对本发明的故障恢复方法 进行说明。但是本发明的故障恢复方法不局限于视频监控领域, 可以适用于所 有需要对关键数据进行特殊保护的应用场合,可以提高数据安全性和故障恢复 速度。  In this embodiment, the data recovery of the video surveillance system is taken as an example to describe the fault recovery method of the present invention. However, the fault recovery method of the present invention is not limited to the field of video surveillance, and can be applied to all applications requiring special protection of critical data, which can improve data security and fault recovery speed.
图 2为本发明数据索引的故障恢复方法另一实施例的应用场景示意图, 如 图 2所示, 磁盘 Γ磁盘 N代表数据盘, 用于保存录像数据; 操作系统、 数据库均 安装在系统盘上 ,并且该系统盘上还存储有用于查找数据盘上的录像数据的数 据索引; 存储服务器可以接收录像数据的数据流, 并将数据流经系统盘存储至 数据盘中。 本实施例的方法可以是该存储服务器所执行。  2 is a schematic diagram of an application scenario of another method for recovering a data index according to the present invention. As shown in FIG. 2, a disk Γ disk N represents a data disk for storing video data; an operating system and a database are installed on the system disk. And the system disk further stores a data index for searching the recorded data on the data disk; the storage server can receive the data stream of the recorded data, and store the data through the system disk to the data disk. The method of this embodiment may be performed by the storage server.
图 3为本发明数据索引的故障恢复方法另一实施例的流程示意图, 如图 3 所示, 该方法可以包括:  FIG. 3 is a schematic flowchart of another embodiment of a method for recovering a data index according to the present invention. As shown in FIG. 3, the method may include:
301、 存储服务器将存储数据放入数据盘的块空间, 并生成存储数据对应 的数据索引;  301. The storage server puts the stored data into a block space of the data disk, and generates a data index corresponding to the stored data.
其中, 所述的存储数据指的是录像数据; 存储数据在数据盘上是按块存放 的, 数据盘上例如可以设置多个块空间, 每个块空间的大小为 1 GB。 当某个块 空间的录像数据存满时, 将自动切换到另外一个空闲的块空间继续存储。 此外, 由于存储数据一般是放置在多个数据盘上, 而且数据盘上的块空间 也较多, 为方便后续对数据的查找获取,通常会在对数据进行存储时也相应生 成该数据的数据索引。例如, 本实施例中的存储服务器在向数据盘中存储录像 数据时, 可以同时在系统盘中生成对应的数据索引, 该数据索引可以包括数据 标识、 数据在数据盘的存储位置信息等。 The stored data refers to the recorded data; the stored data is stored in blocks on the data disk, and the data disk may be configured with a plurality of block spaces, each of which has a size of 1 GB. When the recording data of a block space is full, it will automatically switch to another free block space to continue storage. In addition, since the stored data is generally placed on multiple data disks, and the block space on the data disk is also large, in order to facilitate the subsequent search and acquisition of data, the data of the data is usually generated when the data is stored. index. For example, when storing the recorded data in the data disk, the storage server in the embodiment may simultaneously generate a corresponding data index in the system disk, and the data index may include a data identifier, storage location information of the data in the data disk, and the like.
具体的,图 4为本发明数据索引的故障恢复方法另一实施例的原理示意图, 如图 4所示, 例如, 数据盘 A上设置有 n个块空间, 其中在块空间 a中存储了 n秒 的录像数据;这 n秒的录像数据在存储时所初始生成的数据索引中共包括 n个分 别对应每一秒录像数据的索引, 例如图 4中所示的第 1秒录像数据的数据索引、 第 2秒录像数据的数据索引等, 该 n个数据索引具有相同的格式, 其中包括多个 索引项, 例如可以包括时间、 数据盘号、 块空间号、 起始地址、 结束地址、 索 引标识项 (i sD丽 my ) 、 保留字段等。  Specifically, FIG. 4 is a schematic diagram of another embodiment of a method for recovering data index of the present invention. As shown in FIG. 4, for example, data disk A is provided with n block spaces, where n is stored in block space a. The recording data of the second; the data index initially generated by the n seconds of recording data includes n indexes corresponding to the recording data of each second, for example, the data index of the first second recording data shown in FIG. 4, The data index of the second second recorded data, etc., the n data indexes have the same format, including multiple index items, and may include, for example, time, data disk number, block space number, start address, end address, and index identifier. (i sD 丽 my ), reserved fields, etc.
其中, 时间为录像数据的录像时间, 如 2011年 1月 1日 12点 3分 5秒, 作为录 像数据的数据标识; 存储位置信息可以包括数据盘号、 块空间号、 起始地址、 结束地址; 所述的数据盘号为数据盘标识, 所述的块空间号为数据盘上的块空 间标识, 起始地址、 结束地址为数据在块空间中的物理存储地址, 例如, 存储 位置信息可以为数据盘 A上的块空间 a的第 11字节至第 28字节。索引标识项是用 于标识索引是否有效的标识, 例如可以设置 "1" 表示有效, 即可以利用该索 引查找到对应的录像数据, "0" 表示无效; 保留字段是用于以后扩展用的, 在暂无用到的情况下可以先填为 0。  The time is the recording time of the recorded data, such as 12:3:5 on January 1, 2011, as the data identifier of the recorded data; the storage location information may include the data disk number, the block space number, the start address, and the end address. The data disk number is a data disk identifier, the block space number is a block space identifier on the data disk, and the start address and the end address are physical storage addresses of the data in the block space, for example, the storage location information may be It is the 11th byte to the 28th byte of the block space a on the data disk A. The index identification item is an identifier for identifying whether the index is valid. For example, "1" can be set to be valid, that is, the corresponding recording data can be found by using the index, and "0" means invalid; the reserved field is used for future expansion. In the case of no use, you can first fill in 0.
具体实施中, 可以间隔一段时间向数据盘存储一次; 例如存储服务器在接 收到录像数据后, 可以先将数据存储在緩存中, 然后经过例如 4秒钟的緩存时 间后再将这段时间内的数据统一存入数据盘中, 并且生成数据索引, 该数据索 引可以是与每秒的数据对应的, 例如第 1秒录像数据的数据索引、 第 2秒录像数 据的数据索引等。 In a specific implementation, the data disk may be stored once at a time; for example, after receiving the recorded data, the storage server may first store the data in the cache, and then pass through, for example, a buffer of 4 seconds. After that, the data in this period is uniformly stored in the data disk, and a data index is generated, and the data index may be corresponding to the data per second, for example, the data index of the first second recorded data, and the second second recorded data. Data indexing, etc.
302、 存储服务器根据数据索引得到备份索引, 并将备份索引存储在块空 间的头部预留区域内;  302. The storage server obtains a backup index according to the data index, and stores the backup index in a reserved area of the head space of the block space;
其中, 本实施例的存储服务器在 301中生成数据索引后, 可以根据该数据 索引得到备份索引, 并将备份索引存储至块空间的头部预留区域。  After the storage server of the embodiment generates the data index in 301, the storage index may be obtained according to the data index, and the backup index is stored in the head reserved area of the block space.
具体的, 该备份索引是对数据索引进行备份的, 可以将 301中生成的数据 索 )直接作为备份索引, 或者也可以只选择其中的一部分内容组成备份索 ); 具体选择全部备份还是部分备份可以根据实际情况进行选择,例如根据码流速 度, 码流速度越低, 则填充完一个块空间所需要的时间就越长, 若需要记录对 应每一秒录像数据的数据索引, 那么数据索引的数量就越多,数据索引占用的 块空间区域就越大, 而块空间的头部预留区域是一定的, 则此时若仍然以数据 索引的全部内容进行备份,有可能块空间的头部预留区域无法放置, 则可以选 择部分备份; 相反的, 码流速度越高, 则填充完一个块空间所需要的时间就越 短, 数据索引占用的块空间区域就越小, 此时可以选择全部备份。 在具体实施 中, 对备份方式不做严格限制。  Specifically, the backup index is to back up the data index, and the data generated in 301 can be directly used as a backup index, or only part of the content can be selected as a backup cable); whether all backups or partial backups can be selected specifically According to the actual situation, for example, according to the code stream speed, the lower the code stream speed, the longer the time required to fill a block space. If it is necessary to record the data index corresponding to the recording data per second, then the number of data indexes. The more the block space area occupied by the data index is, and the head reserved area of the block space is certain. If the data is still backed up by the entire contents of the data index, it is possible that the block space is pre-predicted. If the reserved area cannot be placed, you can select a partial backup. Conversely, the higher the code stream speed, the shorter the time required to fill a block space, and the smaller the block space area occupied by the data index. Backup. In the specific implementation, there is no strict restriction on the backup method.
例如, 本实施例的备份索引对 301中的数据索引进行了精简, 从中选择部 分内容进行备份; 该备份索引中至少包括块空间中的存储数据的数据标识、 以 及存储数据在块空间中的存储地址信息,该存储地址信息包括在块空间中存储 的起始地址和结束地址。 至少包括上述信息是因为,现有技术中在对数据索引 进行故障恢复时之所以要耗费很长时间 ,是因为需要对数据盘的块空间进行全 部扫描, 以确定数据在块空间的具体物理存储地址, 这是耗时的主要原因, 因 此,在备份时需要将数据在块空间的存储地址信息保存下来,在进行索引恢复 时就不再需要进行块空间的全部扫描; 而对于数据盘号和块空间号, 这是数据 盘本身所记录的一些属性信息, 可以在扫描数据盘时很容易的获取。 For example, the data index in the backup index pair 301 of the embodiment is simplified, and part of the content is selected for backup; the backup index includes at least a data identifier of the stored data in the block space, and storage of the stored data in the block space. Address information, the storage address information including a start address and an end address stored in the block space. At least the above information is included because it takes a long time to recover the data index in the prior art because the block space of the data disk needs to be completely Scanning to determine the specific physical storage address of the data in the block space, which is the main reason for time-consuming. Therefore, it is necessary to save the storage address information of the data in the block space during backup, and it is no longer needed when index recovery is performed. For the full scan of the block space; for the data disk number and the block space number, this is some attribute information recorded by the data disk itself, which can be easily obtained when scanning the data disk.
此外, 还可以包括索引标识项, 该索引标识项的作用举例说明如下, 假设 有摄像机录像 1小时, 每一秒存储一条记录, 共 3600条记录, 该 3600条记录中 例如其中的 1800条为有效录像, 另外 1800条为无效录像, 则对应有效记录的索 引标识项可以设置为 1 ,对应无效记录的索引标识项可以设置为 0; 在进行备份 时, 可以只备份有效记录即上述的 1800条, 此时, 索引标识项可以备份也可以 不备份, 如果备份则设置为有效标识 1 ; 但是, 这种情况下在后续进行索引恢 复时, 必须恢复索引标识项对有效记录和无效记录进行区分, 因为在恢复时是 需要恢复为 3600条记录的,从备份恢复的记录为有效, 需要设置索引标识项进 行标识, 另外恢复的记录为无效, 也需要索引标识项进行标识。 保留字段不是 必须的。  In addition, an index identifier item may also be included. The role of the index identifier item is as follows. Assume that there is a video recording for one hour, one record is stored every second, and a total of 3,600 records, for example, 1800 of the 3,600 records are valid. Recording, if another 1800 is invalid recording, the index identification item corresponding to the valid record can be set to 1, and the index identification item corresponding to the invalid record can be set to 0; when performing backup, only the valid record, that is, the above 1800, can be backed up. At this time, the index identification item may or may not be backed up. If the backup is set to a valid identifier 1 , in this case, in the subsequent index recovery, the index identification item must be restored to distinguish between the valid record and the invalid record, because If the recovery is to be restored to 3600 records, the records that are restored from the backup are valid. You need to set the index identification item for identification. In addition, the restored records are invalid, and the index identification items are also required for identification. Reserved fields are not required.
此外, 本实施例还对索引项 "时间" 的占用空间进行了缩减, "时间" 在 现有技术中占用 8个字节, 属于冗余设计, 而实际只需要 4个字节即可, 因此本 实施例在设计备份索引时, 将 "时间" 的占用空间设计为 4个字节; 起始地址 和结束地址与原来保持一致, 各占用 4个字节。 本实施例的备份索引, 在进行 上述的精简后, 大小只有 12个字节, 相对于现有技术的 24字节的数据索引, 占 用空间减少了 50% , 大大节省了所占用的块空间。  In addition, in this embodiment, the occupied space of the index item "time" is also reduced. "Time" occupies 8 bytes in the prior art, which belongs to a redundant design, and actually only needs 4 bytes, so In this embodiment, when designing the backup index, the occupied space of "time" is designed to be 4 bytes; the start address and the end address are consistent with the original, each occupying 4 bytes. The backup index of this embodiment has a size of only 12 bytes after the above-mentioned simplification, and the occupied space is reduced by 50% compared with the prior art 24-byte data index, which greatly saves the occupied block space.
本实施例将备份索引存储在块空间的头部预留区域中。在每一个块空间的 头部区域都预留有一小部分空间, 该空间称为头部预留区域; 对应于该块空间 中的存储数据的备份索引都可以存储在该头部预留区域中。 例如, 块空间 a中 存储了从第 1秒至第 n秒的录像数据, 则在该块空间 a的头部预留区域中, 存储 了对应于上述的从第 1秒至第 n秒录像数据的备份索引,包括第 1秒的备份索引、 第 2秒的备份索引 ... ...直至第 n秒。但是备份索引所占用的空间是很小的,例如, 按照标清视频最低 256Kb码流计算, 1GB录像数据所需的备份索引空间仅为 384KB, 约占块空间总存储容量的万分之四。 This embodiment stores the backup index in the header reserved area of the block space. A small portion of space is reserved in the header area of each block space, which is called a head reserved area; corresponding to the block space The backup index of the stored data in the storage area can be stored in the header reserved area. For example, in the block space a, the recorded data from the first second to the nth second is stored, and in the header reserved area of the block space a, the recorded data corresponding to the above from the first second to the nth second is stored. Backup index, including the first second backup index, the second second backup index ... until the nth second. However, the space occupied by the backup index is very small. For example, according to the minimum 256Kb code stream of standard definition video, the backup index space required for 1 GB of video data is only 384 KB, which accounts for about 4/4 of the total storage capacity of the block space.
在具体实施中, 可以在每一个块空间中放置存储数据结束时,再在块空间 的头部预留区域内存储对应存储数据的备份索引。 例如,存储服务器可以每隔 4秒钟向块空间存储一次数据, 并生成对应存储数据的数据索引和备份索引, 在块空间存储满时,可以将对应该块空间中所存储的数据的备份索引一并放入 该块空间的头部预留区域中。这样可以减少对数据盘的频繁操作,延长数据盘 的寿命。  In a specific implementation, when the storage data is placed in each block space, the backup index corresponding to the stored data is stored in the reserved area of the head space of the block space. For example, the storage server may store the data once in the block space every 4 seconds, and generate a data index and a backup index corresponding to the stored data. When the block space is full, the backup index corresponding to the data stored in the block space may be used. It is placed in the head reserved area of the block space. This can reduce the frequent operation of the data disk and extend the life of the data disk.
303、 存储服务器扫描块空间的头部预留区域, 获取存储在该头部预留区 域中的备份索引;  303. The storage server scans a head reserved area of the block space, and obtains a backup index stored in the reserved area of the header.
其中, 当系统盘损坏后, 首先要使用新的系统盘重新安装操作系统和视频 监控软件; 在安装完成后进行数据索引的恢复。存储服务器扫描块空间的头部 预留区域, 可以获取存储在该头部预留区域中的备份索引, 该备份索引中至少 包括时间、 起始地址、 结束地址。  After the system disk is damaged, first use the new system disk to reinstall the operating system and video monitoring software; restore the data index after the installation is complete. The storage server scans the header reserved area of the block space, and obtains a backup index stored in the reserved area of the header. The backup index includes at least a time, a start address, and an end address.
304、 生成数据索引, 所述数据索引至少包括所述备份索引;  304. Generate a data index, where the data index includes at least the backup index.
其中, 在 303中获取到时间、 起始地址、 结束地址之后, 要恢复原始的数 据索引, 还需要数据盘号、 块空间号。 本实施例可以在扫描过程中得到上述二 个索引项; 块空间号即当前正在恢复的块空间的序号, 可以从数据盘属性信息 中的块空间文件名中获取; 数据盘号即当前正在处理的磁盘序号,按照磁盘的 扫描顺序依次递增。 After obtaining the time, the start address, and the end address in 303, to restore the original data index, the data disk number and the block space number are also required. In this embodiment, the above two index items may be obtained during the scanning process; the block space number is the sequence number of the block space currently being restored, and may be from the data disk attribute information. Obtained in the block space file name; the data disk number is the disk serial number currently being processed, and is incremented according to the scanning order of the disk.
此外, 索引标识项 i sDummy可以在进行索引恢复时, 认为该索引项一定有 效,将其设置为有效标识例如 "1 "; 用于以后扩展的保留字段可以为 3个字节, 暂无用到, 可以在恢复时全部填 0。 至此, 已经将备份索引恢复为了原始的数 据索引,存储服务器将该恢复后的数据索引写入新的系统盘中, 即可应用于后 续的数据查找。  In addition, the index identifier i sDummy can be considered to be valid when index recovery is performed, and is set to a valid identifier such as "1"; the reserved field for future expansion can be 3 bytes, which is temporarily useless. You can fill in all 0 when recovering. At this point, the backup index has been restored to the original data index, and the storage server writes the restored data index to the new system disk, which can be applied to subsequent data lookups.
本实施例中, 通过在块空间存储备份索引, 在进行索引恢复时, 只需要将 所有块空间的头部预留区域扫描一遍, 即可将数据索引恢复到系统盘, 不需要 再去遍历所有磁盘数据,而且备份索引所占用的块空间很小,通常仅为几百 KB, 相比现有技术的全盘扫描的恢复方式, 恢复时间可以从之前的 2天缩短到几分 钟内; 当将数据索引进行精简后再备份时, 备份索引所占用的空间将更小, 扫 描恢复时间也将进一步缩短, 大幅节省恢复数据索引所需的时间, 减少业务中 断时间, 提高数据的安全性和完整性, 为安全性和容灾性添加额外的保障。  In this embodiment, by storing the backup index in the block space, when the index recovery is performed, only the head reserved area of all the block spaces needs to be scanned once, and the data index can be restored to the system disk, and there is no need to traverse all the data. Disk data, and the backup index occupies a small amount of block space, usually only a few hundred KB, compared to the prior art full scan recovery method, the recovery time can be shortened from the previous 2 days to a few minutes; When the index is streamlined and then backed up, the backup index takes up less space, and the scan recovery time is further shortened, which greatly saves the time required to restore the data index, reduces service interruption time, and improves data security and integrity. Add extra protection for security and disaster tolerance.
进一步的, 本实施例中, 在 302中根据数据索引得到备份索引之后, 还可 以将所述的备份索引存储在块空间的头部预留区域之外的数据存储空间,所述 数据存储空间中存储有所述备份索引对应的存储数据,并将所述备份索引与对 应的所述存储数据封装存储。  Further, in this embodiment, after the backup index is obtained according to the data index in 302, the backup index may be stored in a data storage space outside the reserved area of the head space of the block space, where the data storage space is The storage data corresponding to the backup index is stored, and the backup index and the corresponding storage data are encapsulated and stored.
具体的, 以视频监控应用为例, 在视频监控的数据盘中, 头部预留区域之 外的区域即为数据存储空间, 该数据存储空间中存储有多个帧的视频数据。对 每一帧数据, 本实施例均进行了封装, 所述的封装指的是, 在原始数据的基础 上增加用于恢复该部分原始数据的恢复信息,该原始数据和恢复信息共同构成 新的一帧数据; 所述的恢复信息即为备份索引。 在进行索引恢复时, 存储服务 器可以扫描块空间中的每一帧数据, 并获取每帧中的原始数据之外的备份索 引, 根据该备份索引就可以恢复原始的数据索引。 此外, 还可以在该帧中设置 帧头信息、 帧尾信息以及数据长度和校验信息, 上述的信息是用于检验该帧数 据是否被损坏的; 例如, 从帧头开始加上数据长度必然得到帧尾信息, 否则就 可以认定该帧数据无效。 Specifically, in the video surveillance application, in the data disk of the video surveillance, the area outside the reserved area of the head is a data storage space, and the video storage space stores video data of multiple frames. For each frame of data, the embodiment is encapsulated, and the encapsulation refers to adding recovery information for restoring the part of the original data on the basis of the original data, and the original data and the recovery information are combined. A new frame of data; the recovery information is a backup index. During index recovery, the storage server can scan each frame of data in the block space and obtain a backup index other than the original data in each frame, according to which the original data index can be restored. In addition, frame header information, frame tail information, and data length and check information may also be set in the frame, and the above information is used to check whether the frame data is damaged; for example, it is inevitable to add a data length from the beginning of the frame. The end of the frame is obtained, otherwise the frame data can be invalidated.
通过将备份索引与存储数据进行混合封装存储,进一步大大提高了对数据 保护的安全性, 使得即使块空间头部预留区域也损坏, 只要存储数据未损坏, 例如上述的某一帧数据未损坏, 就可以恢复该帧数据。  By mixing and storing the backup index and the stored data, the security of the data protection is further greatly improved, so that even if the reserved area of the block space header is damaged, as long as the stored data is not damaged, for example, the above-mentioned frame data is not damaged. , you can restore the frame data.
305、 对恢复后的数据索引进行统计得到统计索引, 将所述统计索引存储 至数据库中。  305. Perform statistics on the restored data index to obtain a statistical index, and store the statistical index into the database.
其中, 通常情况下为了方便存储数据的查找, 系统盘中在数据索引之夕卜, 还设置有图 2中所示的数据库,该数据库中存储的是针对数据索引的统计索引, 该统计索引包括存储数据的数据标识范围,以及数据标识范围在至少一个数据 盘的存储位置信息。  Generally, in order to facilitate the search of the stored data, the data index in the system disk is further provided with the database shown in FIG. 2, where the database stores a statistical index for the data index, and the statistical index includes The data identification range of the stored data, and the storage location information of the data identification range in at least one data disk.
例如, 数据盘 A的块空间 a中, 存储着第 1秒至第 1 0秒的录像数据, 则在系 统盘的数据索引中记录的是第 1秒录像数据索引、第 2秒录像数据索引 即 这 10秒的录像是每一秒分别存储的; 但是在数据库中的记载是, 第 1秒至第 1 0 秒的录像数据存储在数据盘 A的块空间 a中,是统计信息。 当用户查找数据盘的 录像数据时, 例如用户查找的目标数据是第 6秒的录像数据, 则其可以先查找 数据库,得知第 1秒至第 10秒的录像数据存储在数据盘 A的块空间 a中,但是第 6 秒的录像数据存储在块空间 a的什么位置尚未得知; 用户将继续查找系统盘上 的数据索引,该数据索引中记录的是详细的每一秒的录像数据存储在块空间的 具体位置,用户可以从中查找到第 6秒的录像数据是存储在块空间 a的第 11字节 至第 20字节的位置, 如此用户就可以据此得到目标数据。 For example, in the block space a of the data disk A, the recorded data of the first second to the tenth second is stored, and the first second recorded data index and the second second recorded data index are recorded in the data index of the system disk. The 10 second recording is stored separately every second; however, it is recorded in the database that the recording data of the first second to the tenth second is stored in the block space a of the data disk A, and is statistical information. When the user searches for the recorded data of the data disk, for example, the target data searched by the user is the recorded data of the 6th second, it can first search the database, and know that the recorded data of the first second to the tenth second is stored in the block of the data disk A. In space a, but the 6th second of the recorded data is stored in the block space a where is not known; the user will continue to find the system disk The data index, recorded in the data index is a detailed record of each second of the recorded data stored in the block space, from which the user can find the 6th second of the recorded data is stored in the 11th byte of the block space a to The location of the 20th byte, so the user can get the target data accordingly.
在系统盘发生故障时, 上述的数据库中的统计索引数据也会丟失; 在进行 本实施例的数据索引的恢复时, 可以一并恢复数据库信息, 即将恢复得到的数 据索引进行统计, 得到统计索引信息, 并将该统计索引存储至数据库中。  When the system disk fails, the statistical index data in the above database is also lost. When the data index is restored in this embodiment, the database information can be recovered together, and the data index obtained by the recovery is counted to obtain a statistical index. Information, and store the statistical index in the database.
本实施例的数据索引的故障恢复方法,通过在数据盘的块空间中存储备份 索引, 可以根据该备份索引快速恢复数据索引,解决了数据索引恢复时间较长 的问题, 大大提高了数据索引的恢复速度。  In the data recovery fault recovery method of the embodiment, by storing the backup index in the block space of the data disk, the data index can be quickly restored according to the backup index, thereby solving the problem that the data index recovery time is long, and the data index is greatly improved. Recovery speed.
实施例三  Embodiment 3
图 5为本发明数据索引的故障恢复装置实施例的结构示意图, 该装置可以 为图 2中所示的存储服务器, 或者为存储服务器中所设置的其中一功能单元; 该装置可以执行本发明任意实施例所述的恢复方法。本实施例对装置的结构进 行简单说明, 其具体的工作原理可以结合参见方法实施例所述。  5 is a schematic structural diagram of an embodiment of a data recovery fault recovery apparatus according to the present invention. The apparatus may be the storage server shown in FIG. 2 or one of the functional units provided in the storage server. The apparatus may perform any of the present invention. The recovery method described in the examples. This embodiment briefly describes the structure of the device. The specific working principle can be combined with the method embodiment.
如图 5所示, 该装置可以包括备份获取单元 51和索引恢复单元 52。 其中, 备份获取单元 51 , 用于扫描数据盘中的块空间的头部预留区域, 并获取存储在 所述头部预留区域中的备份索引,所述备份索引至少包括所述块空间中的存储 数据的数据标识、 以及所述存储数据在所述块空间中的存储地址信息; 其中, 所述存储数据存储在至少一个数据盘上,且每一所述数据盘包括至少一个用于 放置所述存储数据的块空间; 索引恢复单元 52 , 用于生成标识存储数据在至少 一个数据盘的存储位置信息的数据索引, 所述数据索引至少包括所述备份索 引。 进一步的, 该装置还可以包括索引生成单元 53、 索引备份单元 54。 其中, 索引生成单元 53 , 用于将存储数据放入所述数据盘的块空间中, 并生成所述存 储数据对应的所述数据索引; 索引备份单元 54 , 用于根据所述数据索引得到备 份索引, 并将所述备份索引存储在所述块空间的头部预留区域内。 As shown in FIG. 5, the apparatus may include a backup acquisition unit 51 and an index recovery unit 52. The backup obtaining unit 51 is configured to scan a head reserved area of the block space in the data disk, and obtain a backup index stored in the head reserved area, where the backup index includes at least the block space. Data identification of stored data, and storage address information of the storage data in the block space; wherein the storage data is stored on at least one data disk, and each of the data disks includes at least one for placement The block space for storing data; the index restoring unit 52 is configured to generate a data index that identifies storage location information of the stored data in the at least one data disk, where the data index includes at least the backup index. Further, the device may further include an index generating unit 53 and an index backup unit 54. The index generating unit 53 is configured to put the stored data into the block space of the data disk, and generate the data index corresponding to the stored data. The index backup unit 54 is configured to obtain a backup according to the data index. Indexing, and storing the backup index in a header reserved area of the block space.
进一步的, 索引备份单元 54 , 具体用于在每一个所述块空间中放置存储数 据结束时, 在块空间的头部预留区域内存储对应所述存储数据的备份索引。  Further, the index backup unit 54 is configured to store, in the header reserved area of the block space, a backup index corresponding to the stored data when the storage data is placed in each of the block spaces.
进一步的, 索引备份单元 54 , 具体用于从数据索引中提取对应所述块空间 中的存储数据的数据标识、以及所述存储数据在所述块空间中的存储地址信息 作为所述备份索引; 相应的, 索引恢复单元 52 , 具体用于通过扫描所述数据盘 中的块空间, 获取所述备份索引所在的块空间的块标识、 所述块空间所在的数 据盘的数据盘标识; 并生成至少包括所述备份索引、所述块空间标识和数据盘 标识的数据索引。  Further, the index backup unit 54 is specifically configured to extract, from the data index, a data identifier corresponding to the storage data in the block space, and storage address information of the storage data in the block space as the backup index; Correspondingly, the index recovery unit 52 is configured to: obtain, by scanning a block space in the data disk, a block identifier of a block space where the backup index is located, and a data disk identifier of a data disk where the block space is located; At least the data index of the backup index, the block space identifier, and the data disk identifier is included.
进一步的, 本实施例的索引备份单元 55 ,还用于在根据所述数据索引得到 所述备份索引之后,将所述备份索引存储在所述块空间的头部预留区域之外的 数据存储空间, 所述数据存储空间中存储有所述备份索引对应的存储数据; 并 将所述备份索引与对应的所述存储数据封装存储。  Further, the index backup unit 55 of the embodiment is further configured to store the backup index in a data storage outside the head reserved area of the block space after obtaining the backup index according to the data index. a storage area corresponding to the backup index is stored in the data storage space; and the backup index and the corresponding storage data are encapsulated and stored.
本实施例的数据索引的故障恢复装置,通过在数据盘的块空间中存储备份 索引, 可以根据该备份索引快速恢复数据索引,解决了数据索引恢复时间较长 的问题, 大大提高了数据索引的恢复速度。  The data recovery fault recovery device of the embodiment can quickly restore the data index according to the backup index by storing the backup index in the block space of the data disk, thereby solving the problem that the data index recovery time is long, and the data index is greatly improved. Recovery speed.
所属领域的技术人员可以清楚地了解到, 为描述的方便和简洁, 上述描述 的系统 ,装置和单元的具体工作过程 ,可以参考前述方法实施例中的对应过程 , 在此不再赘述。 在本申请所提供的几个实施例中, 应该理解到, 所揭露的系统, 装置和方 法, 可以通过其它的方式实现。 例如, 以上所描述的装置实施例仅仅是示意性 的, 例如, 所述单元的划分, 仅仅为一种逻辑功能划分, 实际实现时可以有另 外的划分方式, 例如多个单元或组件可以结合或者可以集成到另一个系统, 或 一些特征可以忽略, 或不执行。 另一点, 所显示或讨论的相互之间的耦合或直 接耦合或通信连接可以是通过一些接口, 装置或单元的间接耦合或通信连接, 可以是电性, 机械或其它的形式。 单元显示的部件可以是或者也可以不是物理单元, 即可以位于一个地方, 或者 也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部 单元来实现本实施例方案的目的。 A person skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can be referred to the corresponding process in the foregoing method embodiment, and details are not described herein again. In the several embodiments provided by the present application, it should be understood that the disclosed system, apparatus, and method may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form. The components displayed by the unit may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
另外, 在本发明各个实施例中的各功能单元可以集成在一个处理单元中, 也可以是各个单元单独物理存在 ,也可以两个或两个以上单元集成在一个单元 中。上述集成的单元既可以釆用硬件的形式实现, 也可以釆用软件功能单元的 形式实现。  In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售 或使用时, 可以存储在一个计算机可读取存储介质中。 基于这样的理解, 本发 明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全 部或部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储 介质中, 包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器, 或者网络设备等 )执行本发明各个实施例所述方法的全部或部分步骤。 而前述 的存储介质包括: U盘、 移动硬盘、 只读存储器(ROM, Read-Only Memory ) 、 随机存取存储器 ( RAM , Random Acce s s Memory ) 、 磁碟或者光盘等各种可以 存储程序代码的介质。 The integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may contribute to the prior art or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium. A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: a USB flash drive, a removable hard disk, a read-only memory (ROM), A medium that can store program codes, such as random access memory (RAM, Random Acce ss Memory), disk, or optical disk.
本领域普通技术人员可以理解:实现上述方法实施例的全部或部分步骤可 以通过程序指令相关的硬件来完成,前述的程序可以存储于一计算机可读取存 储介质中, 该程序在执行时, 执行包括上述方法实施例的步骤; 而前述存储介 质包括: R0M、 RAM, 磁碟或者光盘等各种可以存储程序代码的介质。  A person skilled in the art can understand that all or part of the steps of implementing the above method embodiments may be completed by using hardware related to the program instructions. The foregoing program may be stored in a computer readable storage medium, and the program is executed when executed. The steps of the foregoing method embodiments are included; and the foregoing storage medium includes: a medium that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.
最后应说明的是: 以上实施例仅用以说明本发明的技术方案, 而非对其限 制; 尽管参照前述实施例对本发明进行了详细的说明, 本领域的普通技术人员 应当理解: 其依然可以对前述各实施例所记载的技术方案进行修改, 或者对其 中部分技术特征进行等同替换; 而这些修改或者替换, 并不使相应技术方案的 本质脱离本发明各实施例技术方案的精神和范围。  It should be noted that the above embodiments are only for explaining the technical solutions of the present invention, and are not intended to be limiting; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those skilled in the art that: The technical solutions described in the foregoing embodiments are modified, or some of the technical features are equivalently replaced. The modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

权 利 要 求 Rights request
1、 一种数据索引的故障恢复方法, 其特征在于, 所述数据索引用于标识 存储数据在至少一个数据盘的存储位置信息,所述数据盘包括至少一个用于放 置所述存储数据的块空间; 所述故障恢复方法包括: A data index fault recovery method, wherein the data index is used to identify storage location information of storage data in at least one data disk, and the data disk includes at least one block for placing the storage data Space; the fault recovery method includes:
扫描所述块空间的头部预留区域,获取存储在所述头部预留区域中的备份 索引,所述备份索引至少包括所述块空间中的存储数据的数据标识、 以及所述 存储数据在所述块空间中的存储地址信息;  Scanning a head reserved area of the block space, and acquiring a backup index stored in the head reserved area, where the backup index includes at least a data identifier of the stored data in the block space, and the stored data Storage address information in the block space;
生成数据索引, 所述数据索引至少包括所述备份索引。  Generating a data index, the data index including at least the backup index.
2、 根据权利要求 1所述的数据索引的故障恢复方法, 其特征在于, 在扫描 所述块空间的头部预留区域之前, 还包括:  The method for recovering data index according to claim 1, wherein before the header reserved area of the block space is scanned, the method further includes:
将所述存储数据放入所述数据盘的块空间中,并生成所述存储数据对应的 所述数据索引;  Putting the stored data into a block space of the data disk, and generating the data index corresponding to the storage data;
根据所述数据索弓 I得到所述备份索弓 I,并将所述备份索弓 I存储在所述块空 间的头部预留区域内。  The backup cable I is obtained based on the data, and the backup cable I is stored in the head reserved area of the block space.
3、 根据权利要求 2所述的数据索引的故障恢复方法, 其特征在于, 所述将 备份索引存储在所述块空间的头部预留区域内, 包括:  The data recovery fault recovery method according to claim 2, wherein the storing the backup index in the head reserved area of the block space comprises:
在每一个所述块空间中放置存储数据结束时,在所述块空间的头部预留区 域内存储对应所述存储数据的备份索弓 I。  When the storage data is placed in each of the block spaces, a backup cable corresponding to the stored data is stored in the header reserved area of the block space.
4、 根据权利要求 2所述的数据索引的故障恢复方法, 其特征在于, 所述根 据所述数据索引得到备份索引, 包括:  The data recovery fault recovery method according to claim 2, wherein the obtaining the backup index according to the data index comprises:
从所述数据索引中提取对应所述块空间中的存储数据的数据标识、以及所 述存储数据在所述块空间中的存储地址信息作为所述备份索弓 I;  Extracting, from the data index, a data identifier corresponding to the stored data in the block space, and storage address information of the stored data in the block space as the backup cable;
相应的, 所述生成数据索引, 包括:  Correspondingly, the generating data index includes:
通过扫描所述数据盘中的块空间,获取所述备份索引所在的块空间的块空 间标识、所述块空间所在的数据盘的数据盘标识; 并生成至少包括所述备份索 弓 1、 所述块空间标识和数据盘标识的数据索引。  Obtaining, by scanning a block space in the data disk, a block space identifier of a block space where the backup index is located, and a data disk identifier of a data disk where the block space is located; and generating at least the backup cable 1 The data index of the block space identifier and the data disk identifier.
5、 根据权利要求 2所述的数据索引的故障恢复方法, 其特征在于, 在根据 所述数据索引得到所述备份索引之后, 还包括: 将所述备份索弓 I存储在所述块空间的头部预留区域之外的数据存储空间, 所述数据存储空间中存储有所述备份索弓 I对应的存储数据;并将所述备份索弓 I 与对应的所述存储数据封装存储。 The method for recovering the data index according to claim 2, further comprising: after obtaining the backup index according to the data index, further comprising: Storing the backup cable I in a data storage space outside the reserved area of the header of the block space, where the data storage space stores the storage data corresponding to the backup cable I; and the backup The cable I and the corresponding storage data are encapsulated and stored.
6、 一种数据索引的故障恢复装置, 其特征在于, 包括:  6. A data indexing fault recovery device, comprising:
备份获取单元,用于扫描数据盘中的块空间的头部预留区域, 并获取存储 在所述头部预留区域中的备份索弓 I,所述备份索弓 I至少包括所述块空间中的存 储数据的数据标识、以及所述存储数据在所述块空间中的存储地址信息;其中, 所述存储数据存储在至少一个数据盘上,且每一所述数据盘包括至少一个用于 放置所述存储数据的块空间;  a backup acquisition unit, configured to scan a head reserved area of a block space in the data disk, and acquire a backup cable I stored in the reserved area of the header, where the backup cable includes at least the block space a data identifier of the stored data, and storage address information of the stored data in the block space; wherein the storage data is stored on at least one data disk, and each of the data disks includes at least one Placing the block space of the stored data;
索引恢复单元,用于生成标识存储数据在至少一个数据盘的存储位置信息 的数据索引, 所述数据索引至少包括所述备份索引。  And an index recovery unit, configured to generate a data index that identifies storage location information of the stored data on the at least one data disk, where the data index includes at least the backup index.
7、根据权利要求 6所述的数据索引的故障恢复装置,其特征在于,还包括: 索引生成单元,用于将存储数据放入所述数据盘的块空间中, 并生成所述 存储数据对应的所述数据索引;  The data recovery fault recovery apparatus according to claim 6, further comprising: an index generating unit, configured to put the stored data into a block space of the data disk, and generate the storage data corresponding to The data index;
索引备份单元,用于根据所述数据索引得到备份索引, 并将所述备份索引 存储在所述块空间的头部预留区域内。  And an index backup unit, configured to obtain a backup index according to the data index, and store the backup index in a header reserved area of the block space.
8、 根据权利要求 7所述的数据索引的故障恢复装置, 其特征在于, 所述索弓 I备份单元, 具体用于在每一个所述块空间中放置存储数据结束 时, 在所述块空间的头部预留区域内存储对应所述存储数据的备份索弓 I。  The data indexing fault recovery apparatus according to claim 7, wherein the cable arching unit is specifically configured to: when the storage data is placed in each of the block spaces, in the block space A backup cable corresponding to the stored data is stored in the reserved area of the head.
9、 根据权利要求 7所述的数据索引的故障恢复装置, 其特征在于, 所述索弓 I备份单元,具体用于从数据索弓 I中提取对应所述块空间中的存储 数据的数据标识、以及所述存储数据在所述块空间中的存储地址信息作为所述 备份索引;  The data indexing fault recovery apparatus according to claim 7, wherein the cable arching unit is specifically configured to extract, from the data cable, a data identifier corresponding to the stored data in the block space. And storage address information of the stored data in the block space as the backup index;
所述索引恢复单元, 具体用于通过扫描所述数据盘中的块空间, 获取所述 备份索引所在的块空间的块空间标识、 所述块空间所在的数据盘的数据盘标 识;并生成至少包括所述备份索引、所述块空间标识和数据盘标识的数据索引。  The index recovery unit is configured to: obtain, by scanning a block space in the data disk, a block space identifier of a block space where the backup index is located, and a data disk identifier of a data disk where the block space is located; and generate at least A data index including the backup index, the block space identifier, and the data disk identifier.
10、 根据权利要求 6-9任一所述的数据索引的故障恢复装置, 其特征在于, 所述索引备份单元, 还用于在根据所述数据索引得到所述备份索引之后, 将所述备份索弓 I存储在所述块空间的头部预留区域之外的数据存储空间,所述 数据存储空间中存储有所述备份索引对应的存储数据;并将所述备份索引与对 应的所述存储数据封装存储。 The data recovery fault recovery apparatus according to any one of claims 6-9, wherein the index backup unit is further configured to: after obtaining the backup index according to the data index, the backup The cable I stores a data storage space outside the reserved area of the head of the block space, The storage data corresponding to the backup index is stored in the data storage space; and the backup index and the corresponding storage data are encapsulated and stored.
PCT/CN2012/081176 2011-09-28 2012-09-10 Data index failure recovery method and device WO2013044726A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201110297834XA CN102314383B (en) 2011-09-28 2011-09-28 Failure recovery method and device for data index
CN201110297834.X 2011-09-28

Publications (1)

Publication Number Publication Date
WO2013044726A1 true WO2013044726A1 (en) 2013-04-04

Family

ID=45427570

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2012/081176 WO2013044726A1 (en) 2011-09-28 2012-09-10 Data index failure recovery method and device

Country Status (2)

Country Link
CN (1) CN102314383B (en)
WO (1) WO2013044726A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11475141B1 (en) * 2022-03-15 2022-10-18 My Job Matcher, Inc. Apparatus and methods for verifying lost user data

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102314383B (en) * 2011-09-28 2013-12-04 华为数字技术(成都)有限公司 Failure recovery method and device for data index
CN103312494A (en) * 2012-03-14 2013-09-18 中国人民银行印制科学技术研究所 Data scatter storage method, data recovery method and data card
CN102981779A (en) * 2012-11-22 2013-03-20 华为技术有限公司 Data processing method and device
CN103037203B (en) * 2012-12-11 2016-06-08 浙江宇视科技有限公司 A kind of index restoration methods based on block storage and device
CN103914481B (en) * 2013-01-07 2019-09-13 深圳市腾讯计算机系统有限公司 Date storage method and device
CN103559101A (en) * 2013-10-15 2014-02-05 华为技术有限公司 Data backup and recovery method of operating system, storage device and terminal
CN104318960B (en) * 2014-09-15 2018-06-05 华为数字技术(成都)有限公司 The restorative procedure and device of a kind of bad track of hard disk
CN104239564B (en) * 2014-09-28 2018-02-09 深圳市锐明技术股份有限公司 A kind of file index tissue and the method and device of reparation
CN104317833B (en) * 2014-10-10 2019-03-26 深圳市锐明技术股份有限公司 A kind of method and device of recovery system file index
CN104301682B (en) * 2014-10-22 2017-07-11 厦门市美亚柏科信息股份有限公司 The restoration methods and device of a kind of monitor video fragment
CN104516795A (en) * 2015-01-15 2015-04-15 浪潮(北京)电子信息产业有限公司 Data access method and system
CN105045684B (en) * 2015-07-16 2018-06-15 北京京东尚科信息技术有限公司 Index switching and the method and device of index control
CN105183396A (en) * 2015-09-22 2015-12-23 厦门雅迅网络股份有限公司 Storage method for enhancing vehicle-mounted DVR video data traceability
CN109213429B (en) * 2017-06-30 2021-07-23 伊姆西Ip控股有限责任公司 Storage management method and device
CN109101191B (en) * 2018-06-21 2021-07-16 东软集团股份有限公司 Data storage method, data storage device, storage medium and electronic equipment
CN111611258B (en) * 2020-05-27 2023-09-19 杭州海康威视系统技术有限公司 Stream data recovery method and storage device

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1492362A (en) * 2002-10-25 2004-04-28 联想(北京)有限公司 Data back up and recovering method of embedded data bank
US20060179083A1 (en) * 2005-02-08 2006-08-10 Raghavendra Kulkarni Systems and methods for storing, backing up and recovering computer data files
CN101477487A (en) * 2009-01-22 2009-07-08 上海交通大学 Multiple incremental files backup and recovery method
CN102314383A (en) * 2011-09-28 2012-01-11 成都市华为赛门铁克科技有限公司 Failure recovery method and device for data index

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101599080B (en) * 2009-07-22 2011-06-29 中国科学院计算技术研究所 Backup data organization system and method thereof
CN102147711B (en) * 2010-12-31 2014-04-02 华为数字技术(成都)有限公司 Storage method and device based on data content identification

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1492362A (en) * 2002-10-25 2004-04-28 联想(北京)有限公司 Data back up and recovering method of embedded data bank
US20060179083A1 (en) * 2005-02-08 2006-08-10 Raghavendra Kulkarni Systems and methods for storing, backing up and recovering computer data files
CN101477487A (en) * 2009-01-22 2009-07-08 上海交通大学 Multiple incremental files backup and recovery method
CN102314383A (en) * 2011-09-28 2012-01-11 成都市华为赛门铁克科技有限公司 Failure recovery method and device for data index

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11475141B1 (en) * 2022-03-15 2022-10-18 My Job Matcher, Inc. Apparatus and methods for verifying lost user data

Also Published As

Publication number Publication date
CN102314383A (en) 2012-01-11
CN102314383B (en) 2013-12-04

Similar Documents

Publication Publication Date Title
WO2013044726A1 (en) Data index failure recovery method and device
US10817386B2 (en) Virtual machine recovery method and virtual machine management device
CN104462563B (en) A kind of file memory method and system
JP5669867B2 (en) System and method for garbage collection in a deduplication data system
EP2863310B1 (en) Data processing method and apparatus, and shared storage device
US20170103001A1 (en) Data processing device and data processing method
WO2015101084A1 (en) Method, server and system for data recovery
CN105824846B (en) Data migration method and device
CN109635148B (en) Face picture storage method and device
WO2014067240A1 (en) Method and apparatus for recovering sqlite file deleted from mobile terminal
US10002137B2 (en) Efficient data storage utilization and recycling by gradual data degradation
CN104010151A (en) Method for compressing monitoring video file
CN114461456B (en) CDP backup method, system, storage medium and recovery method based on continuous writing
EP3839716A1 (en) Data storage method and apparatus and storage system
WO2019057081A1 (en) Data storage method, data query method, computer device, and storage medium
CN106713940B (en) Video stream storage method, video stream reading method and video stream reading device
CN101937378B (en) Method for carrying out back-up protection on data of storage equipment and computer system
CN104965835A (en) Method and apparatus for reading and writing files of a distributed file system
CN104915376B (en) A kind of archival compression method of file in cloud storage
CN104462148A (en) Data storage management method and device
CN116483284B (en) Method, device, medium and electronic equipment for reading and writing virtual hard disk
CN106998436B (en) Video backup method and device
CN112799872B (en) Erasure code encoding method and device based on key value pair storage system
WO2013091162A1 (en) Method, device, and system for recovering distributed storage data
US11303905B2 (en) Information processing system and information processing method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12836373

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12836373

Country of ref document: EP

Kind code of ref document: A1