WO2019205788A1 - 数据存储方法、存储服务器及云存储系统 - Google Patents

数据存储方法、存储服务器及云存储系统 Download PDF

Info

Publication number
WO2019205788A1
WO2019205788A1 PCT/CN2019/075698 CN2019075698W WO2019205788A1 WO 2019205788 A1 WO2019205788 A1 WO 2019205788A1 CN 2019075698 W CN2019075698 W CN 2019075698W WO 2019205788 A1 WO2019205788 A1 WO 2019205788A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
video data
key video
storage
storage server
Prior art date
Application number
PCT/CN2019/075698
Other languages
English (en)
French (fr)
Inventor
王伟
林起芊
汪渭春
Original Assignee
杭州海康威视系统技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 杭州海康威视系统技术有限公司 filed Critical 杭州海康威视系统技术有限公司
Publication of WO2019205788A1 publication Critical patent/WO2019205788A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/062—Securing storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Definitions

  • the present application relates to the field of information technology, and in particular, to a data storage method, a storage server, and a cloud storage system.
  • the original data In the traditional data security protection technology, the original data is backed up. If the original data is lost, the original data can be restored from the backed up data.
  • the data backup technology is directly used for security protection of all data, the storage space cost will be very high.
  • data backup is selectively performed according to the value of the data content. For example, for monitoring video, manually determine the content value of the video during the critical time period, and manually back up the key video data with high content value.
  • this method of selective backup reduces the storage space cost required for data backup to a certain extent, when the security performance of key video needs to be improved, if it is still stored in backup form, it is necessary to increase the backup of key video data. The quantity will undoubtedly increase the storage space cost, which causes the backup form of storage to fail to balance the storage space cost and the data security protection performance.
  • the present application provides a data storage method, a storage server, and a cloud storage system, so as to achieve both the storage space cost and the data security protection performance.
  • the specific technical solutions are as follows:
  • the embodiment of the present application provides a data storage method, which is applied to a storage server in a cloud storage system, and the method includes:
  • the key video data includes: video data whose heat value is greater than or equal to a preset heat threshold, and the key video data is obtained by using the first erasure code algorithm.
  • the slice data and the first preset number of first redundant data are stored in each storage server;
  • the embodiment of the present application provides a storage server, where the storage server includes:
  • a task execution module configured to acquire data information of key video data sent by the index server, where the key video data includes: video data whose heat value is greater than or equal to a preset heat threshold, and the key video data is based on the first correction
  • the plurality of slice data obtained by the ciphering algorithm and the first preset number of first redundant data are stored in each storage server; and based on the data information, each slice data that has been stored in the key video data is acquired; a second erasure code algorithm, performing redundancy calculation on each slice data that has been stored in the key video data, to obtain a second preset number of second redundancy of the key video data that is greater than the first preset number data;
  • a data storage module configured to store a second preset number of second redundant data of the key video data to each storage server.
  • the embodiment of the present application provides a cloud storage system, where the system includes: multiple storage servers and at least one index server;
  • a storage server configured to acquire data information of key video data sent by the index server, where the key video data includes: video data whose heat value is greater than or equal to a preset heat threshold, and the key video data is based on the first erasure
  • the plurality of slice data obtained by the code algorithm and the first preset number of first redundant data are stored in each storage server; and the slice data stored in the key video data is acquired based on the data information; a second erasure code algorithm, performing redundancy calculation on each slice data that has been stored in the key video data, to obtain a second preset number of second redundant data of the key video data that is greater than a first preset number;
  • the second preset number of second redundant data of the key video data is separately stored to each storage server;
  • An index server is configured to collect and analyze operation information of the operation of each video data stored in the storage server, to obtain a heat value of each video data, and determine, as the key, the video data whose heat value is greater than or equal to a preset threshold.
  • Video data acquiring data information of the key video data; and transmitting data information of the key video data to a storage server that has stored the key video data.
  • an embodiment of the present application provides a storage server, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through a bus; and the memory is configured to store the computer. a program for executing a program stored on the memory to implement the steps of the data storage method provided by the first aspect above.
  • an embodiment of the present application provides a computer readable storage medium, where the computer program stores a computer program, and when the computer program is executed by the processor, the steps of the data storage method provided by the first aspect are implemented.
  • FIG. 1 is a schematic flowchart of a data storage method according to an embodiment of the present application.
  • FIG. 2 is a schematic flow chart of a method for storing a number of shares according to another embodiment of the present application
  • FIG. 3 is a schematic structural diagram of a storage server according to an embodiment of the present application.
  • FIG. 4 is a schematic structural diagram of a storage server according to another embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of a cloud storage system according to an embodiment of the present application.
  • FIG. 6 is a schematic flowchart of a process of interaction between a storage server and an index server in a cloud storage system according to an embodiment of the present application
  • FIG. 7 is a schematic structural diagram of a storage server according to another embodiment of the present application.
  • the flow of the data storage method in an embodiment of the present application is applied to a storage server in a cloud storage system, where the cloud storage system includes multiple storage servers and at least one index server.
  • the storage server is a server for storing video data.
  • the index server is a server for managing storage servers, allocating storage resources, recording the physical location of video data, and responding to client requests.
  • the index server can be deployed in HA (High Availability) dual-system or cluster mode and provides services in a load-balanced mode.
  • the method can include:
  • S101 Obtain data information of key video data sent by the index server, where the key video data includes: video data whose heat value is greater than or equal to a preset heat threshold, and the key video data is obtained by using a plurality of slice data obtained by the first erasure code algorithm.
  • the first preset number of first redundant data forms are stored in each storage server.
  • the manner of obtaining the data information of the key video data may be actively sent by the index server according to a preset frequency and received by the storage server.
  • the storage server may store the request to the index server, and the index server determines whether the video data corresponding to the storage request is key video data, and if yes, sends the data information of the key video data. Any method for obtaining the data information of the key video data can be used in the present application, which is not limited in this embodiment.
  • the key video data if the key video data is stored by a storage server, when the security of the key video data needs to be improved, the key video can be obtained by the storage server storing the key video data as described above.
  • the data information of the data further enhances the security of the key video data through subsequent steps S102 to S104.
  • the key video data is stored in a plurality of storage servers, when the security of the key video data is improved, the data information may be acquired by a storage server storing the key video data to perform key video data. The security is improved.
  • the index server may select multiple storage servers that store the key video data, and obtain data information from each of the selected multiple storage servers to improve the security of the key video data.
  • the data information of the key video data is information used to characterize the uniqueness of the key video data.
  • the data information of the key video data may be identification information of the key video data itself, such as a unique identifier of each movie; or may be identification information of an encoder of the key video data, such as uniqueness of each camera in the monitoring system. Identify and monitor time ranges.
  • the heat value of the video data is a value reflecting the importance of the video data. The higher the heat value, the higher the importance of the video data. When the heat value of the video data is greater than or equal to the preset heat threshold, it generally represents that the video data has higher content. Value, therefore, the security protection performance of such data can be improved.
  • the key video data is already stored in the storage server before the security protection performance is improved.
  • the key video data may be stored in each storage server in the form of a plurality of slice data obtained based on the first erasure code algorithm and a first preset number of first redundant data.
  • the principle of the erasure code algorithm can be summarized as that the first number of slice data of the original data is subjected to a certain calculation to obtain a second quantity of redundant data, and the redundant data is the original data when the slice data is lost.
  • the reconstruction algorithm corresponding to the calculation of the redundant data may be used to restore the data with no loss or error. Lost or wrong data.
  • the key video data is stored in each storage server in the form of the foregoing plurality of slice data and the first preset number of first redundant data, so that when the security protection performance is improved, the slice operation is not required to be performed again.
  • the efficiency of data storage is stored.
  • the key video data to be stored in the form of security protection performance is stored in the storage server in the form of slice data and redundant data, and can also be used at a lower storage space cost before improving the security protection performance. A certain degree of data security protection, avoiding the risk of data loss before the security protection performance is improved.
  • at least one of the plurality of slice data and the first redundant data may be stored in a storage server different from the other data.
  • a certain key video data includes slice data 1, slice data 2, and redundant data 1 and redundant data 2
  • the slice data 1 may be stored in the storage server 1
  • the slice data 2 is stored in the storage server 2
  • redundant The remaining data 1 is stored in the storage server 3
  • the redundant data 2 is stored in the storage server 4.
  • the slice data 1 of the four data is stored in the storage server 1, and the remaining slice data 2, the redundant data 1 and the redundant data 2 are stored in a storage server different from the storage storage server 1.
  • the slice data of the key video data may specifically be that the video data is divided frame by frame according to the video frame of the key video data, and each frame is a slice data.
  • the slice data of the key video data may also be determined according to the quantitative relationship between the demand for the risk of data loss and the slice data, that is, the redundant operation relationship between the quantity requirement of the redundant data and the slice data.
  • the number of data is evenly divided into a corresponding number of slice data according to the number of slice data.
  • the redundant data is data that can provide original data restoration when the slice data of the original data is lost.
  • Each slice data that has been stored in the key video data may carry data information of the key video data, and thus, each slice data corresponding to the key video data may be acquired based on the data information.
  • S103 Perform redundancy calculation on each slice data that has been stored in the key video data by using the second erasure code algorithm to obtain a second preset number of second redundant data of the key video data.
  • the second preset number is greater than the first preset number.
  • the number of second redundant data may be greater than the number of the first redundant data to improve the key video data commitment data.
  • the ability to lose risk. Therefore, the slice data is redundantly calculated by using a second erasure code algorithm different from the first erasure code algorithm to obtain a second preset number of second redundant data that is greater than the first preset number.
  • the second erasure code algorithm may be an erasure code algorithm based on the second preset quantity and the slice data, and the redundant data can be used to restore the slice data.
  • the first erasure code algorithm may be designed to dynamically adjust the number of redundant data, so that the second preset number is input into the first erasure code algorithm to obtain a second erasure code algorithm.
  • a second predetermined number of second redundant data of the key video data may be separately stored to each storage server.
  • the second redundant data of the key video data can be stored in a plurality of ways. Exemplarily, a second predetermined number of second redundant data stores of key video data may be separately stored to the same storage server as the first redundant data and the slice data.
  • the slice data a 1 of the key video data a is stored in the storage server 1
  • the slice server a 2 is stored in the storage server 2
  • the first redundant data a 11 is stored in the storage server 3
  • the second redundant data can be stored in any of the storage server 1 to the storage server 3
  • the second redundant data Can be stored to the storage server 1 to the storage server 3 and the second redundant data
  • the storage server is in a different storage server.
  • a second predetermined number of second redundant data of key video data may be separately stored to a storage server different from the first redundant data and the slice data.
  • the slice data a 1 of the key video data a is stored in the storage server 1
  • the slice server a 2 is stored in the storage server 2
  • the first redundant data a 11 is stored in the storage server 3
  • the second redundant data Can be stored in the storage server 4
  • the second redundant data It can be stored in the storage server 5.
  • storage efficiency is high because storage server selection is not necessary.
  • a storage server is faulty or damaged, only one slice of data or redundant data is lost for the same key video data, and the risk of data loss is high. Therefore, for an application scenario that prioritizes backup efficiency, for example, a live broadcast system with strong demand for real-time performance, especially a live broadcast system with peak user traffic, a method of storing to any storage server may be adopted.
  • methods for storing to different storage servers may be employed.
  • Fig. 1 of the present application is described below with a complete example.
  • the slice data a 1 , a 2 , and 1 first redundant data a 11 are obtained .
  • the slice data a 1 is stored in the storage server 1;
  • the slice data a 2 is stored in the storage server 2;
  • the first redundant data a 11 is stored in the storage server 3.
  • the slice data a 1 , a 2 of the key video data a are acquired from the storage server based on the data information.
  • the slice data a 1 , a 2 of the key video data a are redundantly calculated, and two second redundant data of the key video data a are obtained.
  • Second redundant data of key video data a Second redundant data stored in the storage server 4 It is stored in the storage server 5.
  • the data storage method provided by the embodiment of the present application is applied to a cloud storage system including a plurality of storage servers and at least one index server, and data information of key video data sent by the storage server according to the obtained index server, wherein the key video is used.
  • the data is the video data stored in each storage server in the form of a plurality of slice data obtained based on the first erasure code algorithm and a first preset number of first redundant data, wherein the heat value is greater than or equal to the preset heat threshold. .
  • the data storage method of the present application may further include the following steps:
  • the security protection of the key video data can be implemented by the slice data and the second redundant data, so the first redundant data can be deleted to further save storage.
  • the flow of the data storage method in another embodiment of the present application may include:
  • S201 Obtain data information of key video data sent by the index server, where the key video data includes: video data whose heat value is greater than or equal to a preset heat threshold, and the key video data is multiple slice data obtained by using the first erasure code algorithm and The first preset number of first redundant data forms are stored in each storage server.
  • S203 Perform redundancy calculation on each slice data that has been stored in the key video data by using the second erasure code algorithm to obtain a second preset number of second redundant data of the key video data.
  • the second preset number is greater than the first preset number.
  • S201 to S204 are the same steps as S101 to S104 in the embodiment shown in FIG. 1 of the present application, and details are not described herein again. For details, refer to the description of the embodiment shown in FIG. 1 of the present application.
  • the first sub-data is restored by using the second erasure code algorithm.
  • the first sub-data is slice data and/or redundant data of the key video data
  • the second sub-data is all the slice data of the key video data and all the redundant data except the first sub-data.
  • the method for detecting the loss of the first sub-data of the key video data may be an operation error report sent by the storage server that receives the first sub-data, and the operation error report corresponds to the lost data, which is applicable to the operation error.
  • the resulting data loss situation It is also possible to receive a device failure report sent by the index server, which corresponds to the lost data.
  • This method is suitable for data loss caused by damage or failure of the storage server where the first sub data is located. It can also be a data loss instruction sent by the receiving user. Any manner of detecting the loss of the first sub-data of the key video data can be used in the present application, which is not limited in this embodiment.
  • the total amount of data lost by the first sub-data is the number of sliced data and/or redundant data of the lost key video data, that is, when only the sliced data is lost, the total amount of data lost by the first sub-data is lost.
  • the number of sliced data; or, when only redundant data is lost, the total amount of data lost by the first sub-data is the number of lost redundant data; or, when both the sliced data and the redundant data are lost,
  • the total amount of data lost in the first sub-data is the total number of lost slice data and redundant data.
  • the number of lost slice data can be restored as long as the number of the lost slice data does not exceed the number of redundant data obtained by the second erasure code algorithm.
  • the lost slice data so that the key video data is obtained from the slice data, the effect of data security protection is realized.
  • all slice data of key video data a in each storage server is a 1 , a 2
  • all second redundant data is When a 1 , a 2 , as well as Any data loss, such as the loss of a 1 detected, or a 2 Loss of arbitrary data is lost, and the total amount of lost data is not more than two, that is, the number of second redundant data less than or equal to the key video data a is two.
  • a 1 loss is restored.
  • the first erasure code algorithm and the first redundant data may also be used for restoration. For example, a 1 is detected missing, according to the respective storage servers and a 2 a 11, using the first erasure code algorithm, restore a 1.
  • the lost slice data can be restored because the number of lost slice data is not greater than the number of redundant data. Therefore, the second preset quantity of the second redundant data obtained by the second erasure code algorithm needs to be greater than the first preset quantity of the first redundant data, so as to increase the amount of data that can be lost by the key video data. Thereby improving the ability to bear the risk of data loss and improve the security of data protection.
  • the index server manages the storage server, allocates the storage resource, records the physical location of the video data, responds to the client request, and the user has The operational information of the stored raw video data for operation is also obtained by the index server.
  • the heat value of the above-mentioned key video may be obtained by the index server, by counting and analyzing the operation information of the user operating the video data stored in the storage server.
  • the operation information may specifically be the number of times of playing, the number of times of downloading, the number of times of sharing, the number of times of sharing, the locking of preventing the coverage of the video data, the determining of the video data as an important collection, the mark, and the like, which can reflect the value of the content of the original video data.
  • the heat value may be based on the preset score corresponding to each operation information, and the operation information in the original video data statistical information is scored, and the total score result is used as the heat value of the original video data.
  • the default score of the search information is 1 point
  • the preset score of the lock information is 2 points
  • the preset score of the mark information is 1.5 points.
  • the statistical information of the original video data a is 5 times of retrieval, locked, and uncollected, then the score of the original video data a is 5 points, 2 points, 0 points, and the total score result is 7 points, that is, the original video data.
  • the heat value of a is 7.
  • the preset heat threshold is 5
  • the heat value of the video data a is 7>5
  • the video data a is the key video data.
  • the heat value "7" of the key video data a is recorded, and the data information "a" of the key video data is acquired.
  • the preset heat threshold may be determined based on historical experience.
  • the step of the index server acquiring the data information of the key video data may specifically include:
  • the video data corresponding to the heat value is determined as key video data, and the data information of the video data is obtained.
  • S103 of the embodiment shown in FIG. 1 of the present application may include:
  • the chipping data corresponding to the heat value K of the key video data is used to perform redundancy calculation on each slice data of the key video data, and the second preset number of second key redundancy of the key video data is greater than the first preset number.
  • the key video data with different content values may be further classified by the heat value of the quantified heat value. Therefore, the corresponding redundant data quantity and the erasure code algorithm are determined according to different heat levels to perform differential storage corresponding to the key video data heat level. That is, the amount of redundant data of key video data with higher content value and the amount of redundant data of key video data with lower content value.
  • the data loss risk tolerance of key video data can be further improved with a more reasonable storage space allocation, thereby better adapting to the massive video data. A large number of critical video data security enhancements.
  • the heat value of the key video data a is 3, and is stored in each storage server in the form of 2 slice data and 1 redundant data obtained by the first erasure code algorithm, and the heat value of the preset number 3 is preset.
  • the corresponding second preset number is 2.
  • the heat value of the key video data b is 4, and is stored in each storage server in the form of 2 slice data and 1 redundant data obtained by the first erasure code algorithm, and the heat value corresponding to the preset number 4 is corresponding.
  • the second preset number is 5.
  • the slice data of the key video data a is redundantly calculated by using the erasure code algorithm corresponding to the heat value of the key video data a, and the two second redundant data of the key video data a are obtained.
  • the two second redundant data of the key video data a are separately stored to the respective storage servers.
  • the key data of the key video data b is redundantly calculated, and five second redundant data of the key video data b are obtained, which will be key.
  • the five second redundant data of the video data b are respectively stored to the respective storage servers.
  • the second preset quantity of the second redundant data corresponding to the heat value of the key video data may be determined for each key video data according to the respective heat value of each key video data, thereby being differentiated
  • the amount of redundant data is stored so that when there is a large amount of key video data, a security performance improvement corresponding to the heat value of the key video data is achieved.
  • the higher the critical value of the video data the higher the importance of the enhanced video security, and the higher the second preset amount of the second redundant data. Compared to the same security performance improvement for all key video data, the security and storage costs are relatively reasonable.
  • the data information may also be synchronized to each index server in the cloud storage system to ensure that the available data information still exists when the data information in the current index server is abnormal.
  • the management pressure of the storage server on the same index server can be dispersed.
  • an embodiment of the present application further provides a storage server.
  • the structure of the storage server in an embodiment of the present application may include:
  • the task execution module 301 is configured to obtain data information of key video data sent by the index server, where the key video data includes: video data whose heat value is greater than or equal to a preset heat threshold, and the key video data is obtained by using the first erasure code algorithm.
  • the plurality of slice data and the first preset number of first redundant data are stored in each storage server; and each slice data in which the key video data has been stored is acquired based on the data information. Performing redundancy calculation on each slice data of the key video data by using a second erasure code algorithm to obtain a second preset number of second redundant data of the key video data; the second preset number is greater than the First preset quantity;
  • the data storage module 302 is configured to store a second preset number of second redundant data of the key video data to each storage server.
  • the storage server provided by the embodiment of the present application is applied to a cloud storage system including multiple storage servers and at least one index server, and data information of key video data sent by the storage server according to the obtained index server, where key video data is included.
  • the video data stored in each storage server is stored in the form of a plurality of slice data obtained based on the first erasure code algorithm and a first preset number of first redundant data, wherein the heat value is greater than or equal to the preset heat threshold.
  • the data storage module 302 is further configured to:
  • the storage server may include:
  • the task execution module 401 is configured to obtain data information of key video data sent by the index server, where the key video data includes: video data whose heat value is greater than or equal to a preset heat threshold, and the key video data is obtained by using the first erasure code algorithm.
  • the plurality of slice data and the first preset number of first redundant data are stored in each storage server; and each slice data in which the key video data has been stored is acquired based on the data information. Performing redundancy calculation on each slice data of the key video data by using the second erasure code algorithm to obtain a second preset number of second redundant data of the key video data, where the second preset quantity is greater than the First preset quantity;
  • the data storage module 402 is configured to store a second preset number of second redundant data of the key video data to each storage server.
  • 401 and 402 are the same modules as 301, 302 in the embodiment shown in FIG.
  • the data reconstruction module 403 is configured to: when the first sub data of the key video data is detected to be lost, if the total data quantity of the first sub data is less than or equal to the second preset quantity, according to the key data in each storage server
  • the second sub-data is used to restore the first sub-data by using the second erasure code algorithm, wherein the first sub-data is slice data and/or redundant data of the key video data, and the second sub-data is all slice data of the key video data. And all redundant data, except for the first sub data.
  • the foregoing heat value is obtained by the index server, by counting and analyzing the operation information of the user operating the video data stored in the storage server.
  • the index server may include:
  • the statistics module is configured to collect and analyze the operation information of the operation of each video data stored in the storage server, and obtain the heat value of each video data; and determine the video data whose heat value is greater than or equal to the preset threshold as the key video data. , obtaining data information of key video data;
  • a task sending module configured to send data information of the key video data to a storage server that has stored the key video data.
  • the index server may further include: a memory synchronization module, configured to synchronize data information of the key video data to each index server.
  • the task execution module 301 may be specifically configured to:
  • the structure of a cloud storage system may include: multiple storage servers and at least one index server.
  • the storage server 501 is configured to obtain data information of key video data sent by the index server, where the key video data includes: video data whose heat value is greater than or equal to a preset heat threshold, and the key video data is obtained by using the first erasure code algorithm.
  • the slice data and the first preset number of first redundant data are stored in each storage server; based on the data information, each slice data that has been stored in the key video data is acquired; and the key video is used by using the second erasure code algorithm Performing redundancy calculation on each slice data of the data to obtain a second preset number of second redundant data of the key video data that is greater than a first preset number; and second predetermined number of second key data of the key video data The remaining data is stored separately to each storage server;
  • the index server 502 is configured to collect and analyze operation information of the operation of each video data stored in the storage server, to obtain a heat value of each video data, and determine video data whose heat value is greater than or equal to a preset threshold as a key video. Data, obtaining data information of key video data; transmitting data information of key video data to a storage server that has stored the key video data.
  • the cloud storage system includes a plurality of storage servers and at least one index server, and the data information of the key video data sent by the storage server according to the obtained index server, wherein the key video data is greater than or equal to the heat value.
  • Presetting the heat threshold the video data stored in each storage server in the form of a plurality of slice data obtained based on the first erasure code algorithm and a first preset number of first redundant data.
  • the data storage method provided by the embodiment of the present application is introduced from the interaction process between the storage server and the index server in the cloud storage system shown in the embodiment of FIG.
  • the index server collects and analyzes operation information of the user performing operations on each video data stored in the storage server, and obtains a heat value of each video data.
  • the index server determines video data whose heat value is greater than or equal to a preset threshold as key video data, and acquires data information of the key video data.
  • the index server sends the data information of the key video data to the storage server that has stored the key video data.
  • the storage server acquires data information of key video data sent by the index server. Based on the data information, each slice data in which key video data has been stored is obtained.
  • the second erasure code algorithm is used to perform redundancy calculation on each slice data of the key video data to obtain a second preset number of second redundant data of the key video data, where the second preset quantity is greater than the first preset Quantity.
  • the storage server stores a second preset number of second redundant data of the key video data to each storage server.
  • step S604 of the foregoing embodiment shown in FIG. 6 may specifically include:
  • the method may further include:
  • the second sub-data is used to restore the first sub-data by using the second erasure code algorithm, wherein the first sub-data is slice data and/or redundant data of the key video data, and the second sub-data is all slice data of the key video data. And all redundant data, except for the first sub data.
  • the method may further include:
  • the storage server 700 may include:
  • the memory 703 is configured to store a computer program.
  • the processor 701 is configured to implement the step of applying the data storage method applied to the storage server in any of the foregoing embodiments when executing the computer program stored on the memory 703.
  • the storage server provided by the embodiment of the present application, the data information of the key video data sent by the storage server according to the obtained index server, wherein the key video data is a heat value greater than or equal to a preset heat threshold, based on the first erasure
  • the plurality of slice data obtained by the code algorithm and the first preset number of first redundant data are stored in the video data in each storage server. Obtaining, according to the data information of the key video data, each slice data that has been stored in the key video data; using the second erasure code algorithm to perform redundancy calculation on each slice data of the key video data, to obtain that the key video data is greater than the first preset The second predetermined number of second redundant data of the quantity.
  • the embodiment of the present application further provides an index server, where the index server may include:
  • a processor a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus.
  • a memory for storing computer programs.
  • the processor when used to execute the computer program stored on the memory, implements the steps of the data storage method applied to the index server in any of the above embodiments.
  • the index server provided by the embodiment of the present application obtains the operation statistics of the operation of each original video data stored in the storage server by the index server, and obtains the heat value of the key video data, thereby determining the key video according to the heat value.
  • Data and obtaining data information of key video data, and transmitting data information to a storage server that has stored the key video data.
  • the key video data is determined according to the operation information of the video data by the user, which improves the accuracy and convenience of selecting key video data.
  • the data information of the key video data sent by the index server enables the storage server to acquire each slice data stored in the key video data according to the data information; and use the second erasure code algorithm to redundantize each slice data of the key video data. And calculating, obtaining a second preset number of second redundant data of the key video data that is greater than the first preset number. And storing a second preset number of second redundant data of the key video data to each storage server.
  • redundant data takes up less storage space than backup while improving the security performance of key video data. Therefore, the problem of the storage space cost and the data security protection performance cannot be achieved when the data security protection performance is improved in the form of backup.
  • the above memory may include a RAM (Random Access Memory), and may also include NVM (Non-Volatile Memory), such as at least one disk storage.
  • the memory may also be at least one storage device located away from the processor.
  • the processor may be a general-purpose processor, including a CPU (Central Processing Unit), an NP (Network Processor), or the like; or a DSP (Digital Signal Processor) or an ASIC (Application) Specific Integrated Circuit, FPGA (Field-Programmable Gate Array) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components.
  • CPU Central Processing Unit
  • NP Network Processor
  • DSP Digital Signal Processor
  • ASIC Application) Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • other programmable logic device discrete gate or transistor logic device, discrete hardware components.
  • a computer readable storage medium provided by an embodiment of the present application is included in a storage server, where the computer readable storage medium stores a computer program, and when the computer program is executed by the processor, the storage server is implemented in any one of the foregoing embodiments. The steps of the data storage method.
  • the computer readable storage medium provided by the embodiment of the present application is included in a storage server.
  • the data information of the key video data sent by the storage server according to the obtained index server may be implemented.
  • the video data is a video stored in each storage server in the form of a plurality of slice data obtained based on the first erasure code algorithm and a first preset number of first redundant data, wherein the heat value is greater than or equal to the preset heat threshold. data.
  • a computer readable storage medium provided by another embodiment of the present application is included in an index server, where the computer readable storage medium stores a computer program, and when the computer program is executed by the processor, the index is applied to any of the above embodiments.
  • the steps of the server's data storage method are included in an index server, where the computer readable storage medium stores a computer program, and when the computer program is executed by the processor, the index is applied to any of the above embodiments.
  • the computer readable storage medium provided by the embodiment of the present application is included in an index server.
  • the user can perform operations on the original video data stored in the storage server by counting and analyzing the user by the index server.
  • the operation information obtains the heat value of the key video data, thereby determining the key video data according to the heat value, and acquiring the data information of the key video data, and transmitting the data information to the storage server that has stored the key video data.
  • the key video data is determined according to the operation information of the video data by the user, which improves the accuracy and convenience of selecting key video data.
  • the data information of the key video data sent by the index server enables the storage server to acquire each slice data stored in the key video data according to the data information; and use the second erasure code algorithm to redundantize each slice data of the key video data. And calculating, obtaining a second preset number of second redundant data of the key video data that is greater than the first preset number. And storing a second preset number of second redundant data of the key video data to each storage server.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Quality & Reliability (AREA)
  • Databases & Information Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Bioethics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Hardware Design (AREA)
  • Computer Security & Cryptography (AREA)
  • Software Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本申请实施例提供的数据存储方法、存储服务器以及云存储系统中，一种数据存储方法，应用于包括多个存储服务器及至少一个索引服务器的云存储系统中，通过由存储服务器根据获取的索引服务器发送的关键视频数据的数据信息，获取关键视频数据已存储的各切片数据；利用第二纠删码算法和关键视频数据的各切片数据得到关键视频数据的大于第一预设数量的第二预设数量个第二冗余数据。将第二冗余数据，分别存储至各存储服务器。与传统的备份形式的存储方式相比，通过提升关键视频的冗余数据数量，解决了以备份形式提升数据安全防护性能时，无法兼顾存储空间成本与数据安全防护性能的问题。

Description

数据存储方法、存储服务器及云存储系统
本申请要求于2018年04月23日提交中国专利局、申请号为201810366428.6发明名称为“数据存储方法、存储服务器及云存储系统”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及信息技术领域，特别是涉及一种数据存储方法、存储服务器及云存储系统。
背景技术
系统故障或设备损坏等突发问题在信息技术中不可避免，而这些不可避免的问题造成的数据丢失，将影响系统的正常运行。特别是对于云存储系统，一旦发生这类问题，将导致海量数据的丢失，造成大量损失。因此，在云存储系统中，能够防止数据意外丢失、从而保证数据可用性和完整性的数据安全防护技术是必不可少的。
传统的数据安全防护技术中，对原始数据进行备份。如果原始数据丢失，可以通过备份的数据还原该原始数据。但是，对于具有庞大数据量的云存储系统，如果直接对所有数据采用数据备份技术进行安全防护，存储空间成本将非常高。
为了解决上述问题，相关技术所提供的数据存储方法中，根据数据内容价值，有选择性地进行数据备份。例如：对于监控视频，人工判断关键时间段内视频的内容价值，对内容价值高的关键视频数据手动备份。虽然这种选择性备份的方法在一定程度上降低了数据备份所需的存储空间成本，但在需要对关键视频进行安全性能提升时，如果仍然以备份形式存储，则需要增加关键视频数据的备份数量，无疑将增加存储空间成本，这就造成了备份形式的存储无法兼顾存储空间成本与数据安全防护性能提升的问题。
发明内容
本申请提供了一种数据存储方法、存储服务器及云存储系统，以实现兼顾存储空间成本与数据安全防护性能的目的。具体技术方案如下：
第一方面，本申请实施例提供了一种数据存储方法，应用于云存储系统 中的存储服务器，该方法包括：
获取所述索引服务器发送的关键视频数据的数据信息，所述关键视频数据包括：热度值大于或者等于预设热度阈值的视频数据，所述关键视频数据以基于第一纠删码算法得到的多个切片数据和第一预设数量个第一冗余数据的形式存储在各存储服务器中；
基于所述数据信息，获取所述关键视频数据已存储的各切片数据；
利用第二纠删码算法，对所述关键视频数据已存储的各切片数据进行冗余计算，得到所述关键视频数据的大于第一预设数量的第二预设数量个第二冗余数据；
将所述关键视频数据的第二预设数量个第二冗余数据，分别存储至各存储服务器。
第二方面，本申请实施例提供了一种存储服务器，该存储服务器包括：
任务执行模块，用于获取所述索引服务器发送的关键视频数据的数据信息，所述关键视频数据包括：热度值大于或者等于预设热度阈值的视频数据，所述关键视频数据以基于第一纠删码算法得到的多个切片数据和第一预设数量个第一冗余数据的形式存储在各存储服务器中；基于所述数据信息，获取所述关键视频数据已存储的各切片数据；利用第二纠删码算法，对所述关键视频数据已存储的各切片数据进行冗余计算，得到所述关键视频数据的大于所述第一预设数量的第二预设数量个第二冗余数据；
数据存储模块，用于将所述关键视频数据的第二预设数量个第二冗余数据，分别存储至各存储服务器。
第三方面，本申请实施例提供了一种云存储系统，该系统包括：多个存储服务器及至少一个索引服务器；
存储服务器，用于获取所述索引服务器发送的关键视频数据的数据信息，所述关键视频数据包括：热度值大于或者等于预设热度阈值的视频数据，所述关键视频数据以基于第一纠删码算法得到的多个切片数据和第一预设数量个第一冗余数据的形式存储在各存储服务器中；基于所述数据信息，获取所述关键视频数据已存储的各切片数据；利用第二纠删码算法，对所述关键视频数据已存储的各切片数据进行冗余计算，得到所述关键视频数据的大于第 一预设数量的第二预设数量个第二冗余数据；将所述关键视频数据的第二预设数量个第二冗余数据，分别存储至各存储服务器；
索引服务器，用于统计并分析用户对存储服务器中已存储的各视频数据进行操作的操作信息，得到各视频数据的热度值；将所述热度值大于或等于预设阈值的视频数据确定为关键视频数据，获取所述关键视频数据的数据信息；向已存储所述关键视频数据的存储服务器发送关键视频数据的数据信息。
第四方面，本申请实施例提供了一种存储服务器，包括处理器、通信接口、存储器和通信总线，其中，处理器，通信接口，存储器通过总线完成相互间的通信；存储器，用于存放计算机程序；处理器，用于执行存储器上所存放的程序，实现上述第一方面提供的数据存储方法的步骤。
第五方面，本申请实施例提供了一种计算机可读存储介质，该存储介质内存储有计算机程序，该计算机程序被处理器执行时实现上述第一方面提供的数据存储方法的步骤。
由上述的技术方案可见，与传统的备份形式的存储方式相比，通过提升关键视频数据的冗余数据数量，在提升了关键视频数据安全防护性能的同时，冗余数据比备份占用更少的存储空间。从而解决了以备份形式提升数据安全防护性能时，无法兼顾存储空间成本与数据安全防护性能的问题。
附图说明
为了更清楚地说明本申请实施例和现有技术的技术方案，下面对实施例和现有技术中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为为本申请一实施例的数据存储方法的流程示意图；
图2为本申请另一实施例的数存储份方法的流程示意图；
图3为本申请一实施例的存储服务器的结构示意图；
图4为本申请另一实施例的存储服务器的结构示意图；
图5为本申请一实施例的云存储系统的结构示意图；
图6为本申请一实施例的云存储系统中存储服务器与索引服务器交互过程的流程示意图；
图7为本申请另一实施例的存储服务器的结构示意图。
具体实施方式
为使本申请的目的、技术方案、及优点更加清楚明白，以下参照附图并举实施例，对本申请进一步详细说明。显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
下面首先对本申请一实施例的数据存储方法进行介绍。
如图1所示，本申请一实施例的数据存储方法的流程，应用于云存储系统中的存储服务器，该云存储系统中包括多个存储服务器及至少一个索引服务器。其中，存储服务器为用于存储视频数据的服务器。索引服务器为用于管理存储服务器、分配存储资源、记录视频数据物理位置以及响应客户端请求的服务器。索引服务器的部署方式可以是HA(High Available，高可用性)双机或者是集群模式的，并以负载均衡的模式对外提供服务。该方法可以包括：
S101，获取索引服务器发送的关键视频数据的数据信息，关键视频数据包括：热度值大于或者等于预设热度阈值的视频数据，关键视频数据以基于第一纠删码算法得到的多个切片数据和第一预设数量个第一冗余数据的形式存储在各存储服务器中。
关键视频数据的数据信息的获取方式，可以是由索引服务器按预设频率主动发送，存储服务器接收的。也可以是存储服务器向索引服务器存储请求，索引服务器响应该存储请求，判断该存储请求对应的视频数据是否为关键视频数据，若是，则发送关键视频数据的数据信息。任何获取关键视频数据的数据信息的方法，均可用于本申请，本实施例对此不作限制。
另外，针对一份关键视频数据，如果由一个存储服务器存储该关键视频数据，在需要提升该关键视频数据的安全性时，可以由该存储有关键视频数据的存储服务器按照上述方式获取该关键视频数据的数据信息，进而通过后续步骤S102至S104实现该关键视频数据的安全性提升。相应的，如果该关键视频数据被存储在多个存储服务器中，则在提升该关键视频数据的安全性时， 可以由存储有该关键视频数据的一个存储服务器获取数据信息，进行关键视频数据的安全性提升，也可以是索引服务器选择存储有该关键视频数据的多个存储服务器，由所选择的多个存储服务器各自获取数据信息，进行关键视频数据的安全性提升。关键视频数据的数据信息为用于表征关键视频数据唯一性的信息。示例性的，关键视频数据的数据信息可以是关键视频数据自身的标识信息，例如每个电影的唯一标识；也可以是关键视频数据的编码器的标识信息，例如监控系统中每个摄像机的唯一标识和监控时间范围。
视频数据的热度值为反映视频数据重要性的数值，热度值越高，视频数据的重要性越高，视频数据的热度值大于或者等于预设热度阈值时，通常代表该视频数据具有较高内容价值，因此，可以对这类数据进行安全防护性能的提升。需要说明的是，在进行安全防护性能提升之前，关键视频数据是已经存储在存储服务器中的。具体的，关键视频数据可以是以基于第一纠删码算法得到的多个切片数据和第一预设数量个第一冗余数据的形式存储在各存储服务器中。其中，纠删码算法的原理可以概括为将原始数据的第一数量的切片数据通过一定的计算，得到第二数量个冗余数据，冗余数据为原始数据的切片数据丢失时能够提供原始数据还原依据的数据。当其中任意数据丢失或错误、且丢失或错误的数据个数小于或等于第二数量时，均可以通过与得到冗余数据的计算对应的重构算法，利用未丢失或者未错误的数据还原出丢失或者错误的数据。
将关键视频数据以上述多个切片数据和第一预设数量个第一冗余数据的形式存储在各存储服务器中，可以在进行安全防护性能提升的存储时，不必再次进行切片的操作，提升了数据存储的效率。并且，对要以安全防护性能提升形式存储的关键视频数据，预先以切片数据和冗余数据的形式存储在存储服务器中，也可以在提升安全防护性能之前，以较低的存储空间成本起到一定程度的数据安全防护作用，避免了安全防护性能提升之前的数据丢失风险。此外，可以将上述多个切片数据和第一冗余数据中的至少一个数据存储在与其他数据不同的存储服务器中。例如，某一关键视频数据包括切片数据1、切片数据2以及冗余数据1和冗余数据2，则可以将切片数据1存储在存储服务器1中，切片数据2存储在存储服务器2中，冗余数据1存储在存储服务器3中，冗余数据2存储在存储服务器4中。或者，将4个数据中的切片数据1存储在存 储服务器1中，余下的切片数据2、冗余数据1和冗余数据2存储在与存储存储服务器1不同的存储服务器中。
其中，关键视频数据的切片数据具体可以为，按关键视频数据的视频帧逐帧分割该视频数据，每一帧即为一个切片数据。关键视频数据的切片数据还可以是，根据对承担数据丢失风险能力的需求与切片数据之间的数量关系，也就是冗余数据的数量需求与切片数据之间的冗余运算关系，确定出切片数据的数量，根据切片数据的数量均匀分割出对应数量个切片数据。其中，冗余数据为原始数据的切片数据丢失时能够提供原始数据还原依据的数据。
S102，基于数据信息，获取关键视频数据已存储的各切片数据。
关键视频数据已存储的各切片数据可以携带关键视频数据的数据信息，由此，可以基于数据信息，获取与关键视频数据对应的各切片数据。
S103，利用第二纠删码算法，对关键视频数据已存储的各切片数据进行冗余计算，得到关键视频数据的第二预设数量个第二冗余数据。其中，第二预设数量大于第一预设数量。
由于冗余数据的数量与数据承担数据丢失风险的能力成正相关，为了实现数据安全防护性能的提升，第二冗余数据的数量可以大于第一冗余数据的数量，以提升关键视频数据承担数据丢失风险的能力。因此，采用与第一纠删码算法不同的第二纠删码算法对切片数据进行冗余计算，以得到大于第一预设数量的第二预设数量个第二冗余数据。
其中，第二纠删码算法具体可以是基于第二预设数量和切片数据，以冗余数据能够还原切片数据为原则，得到的纠删码算法。也可以是将第一纠删码算法设计为冗余数据数量可动态调整的算法，从而将第二预设数量输入第一纠删码算法，得到第二纠删码算法。
S104，将关键视频数据的第二预设数量个第二冗余数据，分别存储至各存储服务器。
由于关键视频数据的切片数据已存储，因此，可以将关键视频数据的第二预设数量个第二冗余数据，分别存储至各存储服务器。关键视频数据的第二冗余数据的存储方式可以是多种的。示例性的，可以将关键视频数据的第 二预设数量个第二冗余数据存储分别存储至与第一冗余数据和切片数据相同的存储服务器。例如，关键视频数据a的切片数据a 1存储在存储服务器1中，切片服务器a 2存储在存储服务器2中，第一冗余数据a 11存储在存储服务器3中，则第二冗余数据
Figure PCTCN2019075698-appb-000001
可以存储至存储服务器1至存储服务器3中的任一存储服务器中，第二冗余数据
Figure PCTCN2019075698-appb-000002
可以存储至存储服务器1至存储服务器3中与第二冗余数据
Figure PCTCN2019075698-appb-000003
的存储服务器不同的存储服务器中。或者，示例性的，可以将关键视频数据的第二预设数量个第二冗余数据，分别存储至与第一冗余数据和切片数据不同的存储服务器。例如，关键视频数据a的切片数据a 1存储在存储服务器1中，切片服务器a 2存储在存储服务器2中，第一冗余数据a 11存储在存储服务器3中，则第二冗余数据
Figure PCTCN2019075698-appb-000004
可以存储至存储服务器4中，第二冗余数据
Figure PCTCN2019075698-appb-000005
可以存储至存储服务器5中。
对于存储至任意存储服务器的情况，由于不必进行存储服务器选取，因此存储效率较高。而对于存储至不同存储服务器的情况，如果某一台存储服务器故障或损坏，同一关键视频数据仅有一个切片数据或冗余数据丢失，抗数据丢失风险能力较高。因此，对于优先考虑备份效率的应用场景，例如对实时性有很强需求的直播系统，特别是用户流量高峰期的直播系统，可以采用存储至任意存储服务器的方法。对于优先考虑抗数据丢失风险能力的应用场景，例如视频监控系统，可以采用存储至不同存储服务器的方法。
下面以一个完整的实例说明本申请图1实施例。示例性的：针对关键视频数据a，基于第一纠删码算法，得到切片数据a 1、a 2、以及1个第一冗余数据a 11。将切片数据a 1存储在存储服务器1中；将切片数据a 2存储至存储服务器2中；将第一冗余数据a 11存储至存储服务器3中。获取索引服务器发送的关键视频数据的数据信息。根据数据信息，从存储服务器中获取关键视频数据a的切片数据a 1、a 2。利用第二纠删码算法，对关键视频数据a的切片数据a 1、a 2进行冗余计算，得到关键视频数据a的2个第二冗余数据
Figure PCTCN2019075698-appb-000006
将关键视频数据a的第二冗余数据
Figure PCTCN2019075698-appb-000007
存储至存储服务器4中、的第二冗余数据
Figure PCTCN2019075698-appb-000008
存储至存储服务器5中。
本申请实施例提供的数据存储方法，应用于包括多个存储服务器及至少一个索引服务器的云存储系统中，通过由存储服务器根据获取的索引服务器发送的关键视频数据的数据信息，其中，关键视频数据为热度值大于或者等 于预设热度阈值的、以基于第一纠删码算法得到的多个切片数据和第一预设数量个第一冗余数据的形式存储在各存储服务器中的视频数据。基于关键视频数据的数据信息，获取关键视频数据已存储的各切片数据；利用第二纠删码算法，对关键视频数据的各切片数据进行冗余计算，得到关键视频数据的大于第一预设数量的第二预设数量个第二冗余数据。将关键视频数据的第二预设数量个第二冗余数据，分别存储至各存储服务器。与传统的备份形式的存储方式相比，通过提升关键视频的冗余数据数量，在提升了关键视频数据安全防护性能的同时，冗余数据比备份占用更少的存储空间。从而解决了以备份形式提升数据安全防护性能时，无法兼顾存储空间成本与数据安全防护性能的问题。
可选的，在图1所示实施例的步骤S104之后，本申请的数据存储方法还可以包括如下步骤：
删除各存储服务器中关键视频数据的第一预设数量个第一冗余数据。
由于在将关键视频数据的第二冗余数据分别存储至各存储服务器后，关键视频数据的安全防护可以由切片数据与第二冗余数据实现，因此可以删除第一冗余数据，以进一步节省存储空间。
如图2所示，本申请另一实施例的数据存储方法的流程，可以包括：
S201，获取索引服务器发送的关键视频数据的数据信息，关键视频数据包括：热度值大于或者等于预设热度阈值的视频数据，关键视频数据以基于第一纠删码算法得到的多个切片数据和第一预设数量个第一冗余数据的形式存储在各存储服务器中。
S202，基于数据信息，获取关键视频数据已存储的各切片数据。
S203，利用第二纠删码算法，对关键视频数据已存储的各切片数据进行冗余计算，得到关键视频数据的第二预设数量个第二冗余数据。其中，第二预设数量大于第一预设数量。
S204，将关键视频数据的第二预设数量个第二冗余数据，分别存储至各存储服务器。
S201至S204为与本申请图1所示实施例中S101至S104相同的步骤，在此不再赘述，详见图1所示实施例的描述。
S205，在检测到关键视频数据的第一子数据丢失时，如果第一子数据丢失的总数据数量小于或等于第二预设数量，则根据各存储服务器中关键视频数据的第二子数据，利用第二纠删码算法，还原第一子数据。其中，第一子数据为关键视频数据的切片数据和/或冗余数据，第二子数据为关键视频数据的所有切片数据和所有冗余数据中、除第一子数据以外的数据。
检测到关键视频数据的第一子数据丢失的方式，具体可以为接收第一子数据所在存储服务器发送的操作错误报告，该操作错误报告与丢失的数据相对应，这种方式适用于因操作失误引起的数据丢失情况。也可以为接收索引服务器发送的设备故障报告，该故障报告与丢失的数据相对应，这种方式适用于因第一子数据所在存储服务器损坏或故障引起的数据丢失情况。还可以是接收用户发送的数据丢失指令。任何能够检测到关键视频数据的第一子数据丢失的方式均可用于本申请，本实施例对此不作限制。其中，第一子数据丢失的总数据数量是丢失的关键视频数据的切片数据和/或冗余数据的个数，也就是当仅丢失切片数据时，第一子数据丢失的总数据数量为丢失的切片数据的个数；或者，当仅丢失冗余数据时，第一子数据丢失的总数据数量为丢失的冗余数据的个数；或者，当切片数据和冗余数据均存在丢失时，第一子数据丢失的总数据数量为丢失的切片数据和冗余数据的总个数。
由于基于冗余数据，可以对丢失了部分数据的关键视频数据进行还原，因此，只要保证丢失的切片数据的数量不超过利用第二纠删码算法得到的冗余数据的数量，即可还原出丢失的切片数据，从而由切片数据得到关键视频数据，实现了数据安全防护的效果。
例如，各存储服务器中关键视频数据a的所有切片数据为a 1、a 2，所有第二冗余数据为
Figure PCTCN2019075698-appb-000009
当检测到a 1、a 2、
Figure PCTCN2019075698-appb-000010
以及
Figure PCTCN2019075698-appb-000011
中任意数据丢失，如检测到a 1丢失，或者a 2、
Figure PCTCN2019075698-appb-000012
丢失等任意数据丢失的情况，且丢失的总数据数量不大于2个，也就是小于或者等于关键视频数据a的第二冗余数据的数 量2个。以检测到a 1丢失为例，则根据各存储服务器中的a 2、
Figure PCTCN2019075698-appb-000013
以及
Figure PCTCN2019075698-appb-000014
利用第二纠删码算法，还原出a 1。
当然，如果第一子数据丢失的数量小于或者等于第一预设数量时，也可以利用第一纠删码算法和第一冗余数据进行还原。例如，检测到a 1丢失，则根据各存储服务器中的a 2以及a 11，利用第一纠删码算法，还原出a 1。
同时，在上述实施例的基础上进一步可以理解，由于丢失的切片数据数量不大于冗余数据的数量时，才可以还原出丢失的切片数据。因此，通过第二纠删码算法得到的第二冗余数据的数量第二预设数量，需要大于第一冗余数据的数量第一预设数量，才能提升关键视频数据可丢失的数据数量，从而提高承担数据丢失风险的能力，提升数据防护安全性能。
由于关键视频数据的热度值，是由用户对该视频数据的操作决定的，而索引服务器管理存储服务器、分配存储资源、记录视频数据物理位置、响应客户端请求的服务器，用户对存储服务器中已存储的各原始视频数据进行操作的操作信息也是索引服务器才能获取的。
因此，可选的，上述关键视频的热度值可以由索引服务器、通过统计并分析用户对存储服务器中已存储的各视频数据进行操作的操作信息得到。
其中，操作信息具体可以为播放次数、下载次数、收藏次数、分享次数以及防止视频数据被覆盖的锁定、确定视频数据为重要的收藏、标记等能够反映原始视频数据内容价值的操作信息。
热度值可以是根据与上述各操作信息对应的预设分值，对原始视频数据统计信息中的各操作信息进行评分，将总评分结果作为该原始视频数据的热度值。例如，检索信息的预设分值为1分、锁定信息的预设分值为2分，以及标记信息的预设分值为1.5分。原始视频数据a的统计信息是检索次数5次、被锁定、以及未收藏，则原始视频数据a的评分结果为5分，2分，0分，总评分结果为7分，也就是原始视频数据a的热度值为7。当预设热度阈值为5时，视频数据a的热度值7＞5，则视频数据a为关键视频数据。记录该关键视频数据a的热度值“7”，并获取该关键视频数据的数据信息“a”。其中，预设热度阈值可以是根据历史经验确定的。
相应的，索引服务器获取关键视频数据的数据信息的步骤，具体可以包括：
统计并分析用户对存储服务器中已存储的各视频数据进行操作的操作信息，得到热度值；
判断热度值是否大于或者等于预设热度阈值；
若是，则将该热度值对应的视频数据确定为关键视频数据，并获取该视频数据的数据信息。
可选的，本申请图1所示实施例的S103，可以包括：
利用与关键视频数据的热度值K对应的纠删码算法，对关键视频数据的各切片数据进行冗余计算，得到关键视频数据的大于第一预设数量的第二预设数量个第二冗余数据，其中，K为热度值编号，第二预设数量与热度值K对应。
由于不同视频数据的内容价值可能不同，因此，可以在确定出关键视频数据的基础上，对于内容价值不同的关键视频数据，进一步以量化的热度值进行热度级别划分。从而根据不同热度级别确定相应的冗余数据数量和纠删码算法，以进行对应于关键视频数据热度级别的差异化存储。也就是内容价值更高的关键视频数据的冗余数据数量、较内容价值较低的关键视频数据的冗余数据数量更多。通过对应于关键视频数据冗余级别的差异化冗余数据数量的确定，以更加合理的存储空间分配，进一步提升关键视频数据的数据丢失风险承担能力，从而更好地适应海量视频数据带来的大量关键视频数据的安全防护性能提升的情况。
例如：关键视频数据a的热度值编号为3，以基于第一纠删码算法得到的2个切片数据和1个冗余数据的形式，存储在各存储服务器中，预设编号3的热度值对应的第二预设数量为2。关键视频数据b的热度值编号为4，以基于第一纠删码算法得到的2个切片数据和1个冗余数据的形式，存储在各存储服务器中，预设编号4的热度值对应的第二预设数量为5。对此，利用与关键视频数据a的编号为3的热度值对应的纠删码算法，对关键视频数据a的各切片数据进行冗余计算，得到关键视频数据a的2个第二冗余数据，将关键视频数据a的2个第二冗余数据分别存储至各存储服务器。利用与关键视频数据b的编 号为4的热度值对应的纠删码算法，对关键视频数据b的各切片数据进行冗余计算，得到关键视频数据b的5个第二冗余数据，将关键视频数据b的5个第二冗余数据分别存储至各存储服务器。由此，可以分别根据各个关键视频数据各自的热度值，针对每个关键视频数据，确定与该关键视频数据的热度值相对应的第二冗余数据的第二预设数量，从而以差异化的冗余数据数量进行存储，以便在存在大量关键视频数据时，实现与关键视频数据的热度值相对应的安全性能提升。其中，热度值越高的关键视频数据重要性越高，提升的安全性能越高，第二冗余数据的第二预设数量越大。与对所有关键视频数据进行相同的安全性能提升相比，相对而言对安全性能和存储成本的兼顾更加合理。
当然，在确定出关键视频数据的数据信息后，还可以将该数据信息同步至云存储系统中的各索引服务器，以保证在当前索引服务器中的数据信息异常时，依然存在可用的数据信息。同时，可以分散同一台索引服务器对存储服务器的管理压力。
相应于上述方法实施例，本申请一实施例还提供了存储服务器。
如图3所示，本申请一实施例的存储服务器的结构，该存储服务器可以包括：
任务执行模块301，用于获取索引服务器发送的关键视频数据的数据信息，关键视频数据包括：热度值大于或者等于预设热度阈值的视频数据，关键视频数据以基于第一纠删码算法得到的多个切片数据和第一预设数量个第一冗余数据的形式存储在各存储服务器中；基于数据信息，获取关键视频数据已存储的各切片数据。利用第二纠删码算法，对关键视频数据的各切片数据进行冗余计算，得到所述关键视频数据的第二预设数量个第二冗余数据；所述第二预设数量大于所述第一预设数量；
数据存储模块302，用于将所述关键视频数据的第二预设数量个第二冗余数据，分别存储至各存储服务器。
本申请实施例提供的存储服务器，应用于包括多个存储服务器及至少一 个索引服务器的云存储系统中，通过由存储服务器根据获取的索引服务器发送的关键视频数据的数据信息，其中，关键视频数据为热度值大于或者等于预设热度阈值的、以基于第一纠删码算法得到的多个切片数据和第一预设数量个第一冗余数据的形式存储在各存储服务器中的视频数据。基于关键视频数据的数据信息，获取关键视频数据已存储的各切片数据；利用第二纠删码算法，对关键视频数据的各切片数据进行冗余计算，得到关键视频数据的大于第一预设数量的第二预设数量个第二冗余数据。将关键视频数据的第二预设数量个第二冗余数据，分别存储至各存储服务器。与传统的备份形式的存储方式相比，通过提升关键视频的冗余数据数量，在提升了关键视频数据安全防护性能的同时，冗余数据比备份占用更少的存储空间。从而解决了以备份形式提升数据安全防护性能时，无法兼顾存储空间成本与数据安全防护性能的问题。
可选的，如图3实施例所示的存储服务器中，数据存储模块302，还可以用于：
删除各存储服务器中关键视频数据的第一预设数量个第一冗余数据。
如图4所示，本申请另一实施例的存储服务器的结构，该存储服务器可以包括：
任务执行模块401，用于获取索引服务器发送的关键视频数据的数据信息，关键视频数据包括：热度值大于或者等于预设热度阈值的视频数据，关键视频数据以基于第一纠删码算法得到的多个切片数据和第一预设数量个第一冗余数据的形式存储在各存储服务器中；基于数据信息，获取关键视频数据已存储的各切片数据。利用第二纠删码算法，对关键视频数据的各切片数据进行冗余计算，得到所述关键视频数据的第二预设数量个第二冗余数据，所述第二预设数量大于所述第一预设数量；
数据存储模块402，用于将关键视频数据的第二预设数量个第二冗余数据，分别存储至各存储服务器。
401和402为与图3所示实施例中301、302相同的模块。
数据重构模块403，用于在检测到关键视频数据的第一子数据丢失时，如 果第一子数据的总数据数量小于或等于第二预设数量，则根据各存储服务器中关键数据的第二子数据，利用第二纠删码算法，还原第一子数据，其中，第一子数据为关键视频数据的切片数据和/或冗余数据，第二子数据为关键视频数据的所有切片数据和所有冗余数据中、除第一子数据以外的数据。
可选的，上述热度值由索引服务器、通过统计并分析用户对存储服务器中已存储的各视频数据进行操作的操作信息得到。
其中，索引服务器可以包括：
统计模块，用于统计并分析用户对存储服务器中已存储的各视频数据进行操作的操作信息，得到各视频数据的热度值；将热度值大于或等于预设阈值的视频数据确定为关键视频数据，获取关键视频数据的数据信息；
任务下发模块，用于向已存储所述关键视频数据的存储服务器发送关键视频数据的数据信息。
当然，索引服务器还可以包括：内存同步模块，用于将所述关键视频数据的数据信息同步至各索引服务器。
可选的，如图3所示实施例的存储服务器中，任务执行模块301具体可以用于：
利用与关键视频数据的热度值K对应的纠删码算法，对关键视频数据的各切片数据进行冗余计算，得到关键视频数据的第二预设数量个第二冗余数据，其中，K为所述热度值编号，第二预设数量与热度值K对应。
如图5所示，本申请一实施例的云存储系统的结构，可以包括：多个存储服务器及至少一个索引服务器。
存储服务器501，用于获取索引服务器发送的关键视频数据的数据信息，关键视频数据包括：热度值大于或者等于预设热度阈值的视频数据，关键视频数据以基于第一纠删码算法得到的多个切片数据和第一预设数量个第一冗余数据的形式存储在各存储服务器中；基于数据信息，获取关键视频数据已存储的各切片数据；利用第二纠删码算法，对关键视频数据的各切片数据进 行冗余计算，得到所述关键视频数据的大于第一预设数量的第二预设数量个第二冗余数据；将关键视频数据的第二预设数量个第二冗余数据，分别存储至各存储服务器；
索引服务器502，用于统计并分析用户对存储服务器中已存储的各视频数据进行操作的操作信息，得到各视频数据的热度值；将热度值大于或等于预设阈值的视频数据确定为关键视频数据，获取关键视频数据的数据信息；向已存储所述关键视频数据的存储服务器发送关键视频数据的数据信息。
本申请实施例提供的云存储系统，包括多个存储服务器及至少一个索引服务器，通过由存储服务器根据获取的索引服务器发送的关键视频数据的数据信息，其中，关键视频数据为热度值大于或者等于预设热度阈值的、以基于第一纠删码算法得到的多个切片数据和第一预设数量个第一冗余数据的形式存储在各存储服务器中的视频数据。基于关键视频数据的数据信息，获取关键视频数据已存储的各切片数据；利用第二纠删码算法，对关键视频数据的各切片数据进行冗余计算，得到关键视频数据的大于第一预设数量的第二预设数量个第二冗余数据。将关键视频数据的第二预设数量个第二冗余数据，分别存储至各存储服务器。与传统的备份形式的存储方式相比，通过提升关键视频的冗余数据数量，在提升了关键视频数据安全防护性能的同时，冗余数据比备份占用更少的存储空间。从而解决了以备份形式提升数据安全防护性能时，无法兼顾存储空间成本与数据安全防护性能的问题。
为了便于理解，如图6所示，从图5实施例所示的云存储系统中存储服务器与索引服务器的交互过程，介绍本申请实施例所提供的数据存储方法。
S601，索引服务器统计并分析用户对存储服务器中已存储的各视频数据进行操作的操作信息，得到各视频数据的热度值。
S602，索引服务器将热度值大于或等于预设阈值的视频数据确定为关键视频数据，获取关键视频数据的数据信息。
S603，索引服务器向已存储所述关键视频数据的存储服务器发送关键视频数据的数据信息。
S604，存储服务器获取索引服务器发送的关键视频数据的数据信息。基 于数据信息，获取关键视频数据已存储的各切片数据。利用第二纠删码算法，对关键视频数据的各切片数据进行冗余计算，得到所述关键视频数据的第二预设数量个第二冗余数据，第二预设数量大于第一预设数量。
S605，存储服务器将关键视频数据的第二预设数量个第二冗余数据，分别存储至各存储服务器。
可选的，上述图6所示实施例的步骤S604，具体可以包括：
利用与关键视频数据的热度值K对应的纠删码算法，对关键视频数据的各切片数据进行冗余计算，得到关键视频数据的第二预设数量个第二冗余数据，其中，K为所述热度值编号，第二预设数量与热度值K对应。
可选的，在上述图6所示实施例的步骤S605之后，还可以包括：
在检测到所述关键视频数据的第一子数据丢失时，如果第一子数据丢失的总数据数量小于或等于所述第二预设数量，则根据各存储服务器中所述关键视频数据的第二子数据，利用第二纠删码算法，还原第一子数据，其中，第一子数据为关键视频数据的切片数据和/或冗余数据，第二子数据为关键视频数据的所有切片数据和所有冗余数据中、除第一子数据以外的数据。
可选的，在上述图6所示实施例的步骤S605之后，还可以包括：
删除各存储服务器中关键视频数据的第一预设数量个第一冗余数据。
相应于上述实施例，本申请另一实施例还提供了一种存储服务器，如图7所示，存储服务器700，可以包括：
处理器701、通信接口702、存储器703和通信总线704，其中，处理器701，通信接口702，存储器通703过通信总线704完成相互间的通信。
存储器703，用于存放计算机程序。
处理器701，用于执行上述存储器703上所存放的计算机程序时，实现上述任一实施例中应用于存储服务器的数据存储方法的步骤。
本申请实施例提供的存储服务器，通过由存储服务器根据获取的索引服 务器发送的关键视频数据的数据信息，其中，关键视频数据为热度值大于或者等于预设热度阈值的、以基于第一纠删码算法得到的多个切片数据和第一预设数量个第一冗余数据的形式存储在各存储服务器中的视频数据。基于关键视频数据的数据信息，获取关键视频数据已存储的各切片数据；利用第二纠删码算法，对关键视频数据的各切片数据进行冗余计算，得到关键视频数据的大于第一预设数量的第二预设数量个第二冗余数据。将关键视频数据的第二预设数量个第二冗余数据，分别存储至各存储服务器。与传统的备份形式的存储方式相比，通过提升关键视频的冗余数据数量，在提升了关键视频数据安全防护性能的同时，冗余数据比备份占用更少的存储空间。从而解决了以备份形式提升数据安全防护性能时，无法兼顾存储空间成本与数据安全防护性能的问题。
相应于上述实施例，本申请实施例还提供了一种索引服务器，该索引服务器可以包括：
处理器、通信接口、存储器和通信总线，其中，处理器，通信接口，存储器通过通信总线完成相互间的通信。
存储器，用于存放计算机程序。
处理器，用于执行上述存储器上所存放的计算机程序时，实现上述任一实施例中应用于索引服务器的数据存储方法的步骤。
本申请实施例提供的索引服务器，通过由索引服务器统计并分析用户对存储服务器中已存储的各原始视频数据进行操作的操作统计信息，得到关键视频数据的热度值，从而根据热度值确定关键视频数据、并获取关键视频数据的数据信息，向已存储所述关键视频数据的存储服务器发送数据信息。与人工选取关键视频并对关键视频手动备份的方式相比，根据用户对视频数据的操作信息确定关键视频数据，提升了关键视频数据选取的精准度和便捷性。同时，索引服务器发送的关键视频数据的数据信息使存储服务器能够根据该数据信息，获取关键视频数据已存储的各切片数据；利用第二纠删码算法，对关键视频数据的各切片数据进行冗余计算，得到关键视频数据的大于第一预设数量的第二预设数量个第二冗余数据。将关键视频数据的第二预设数量个第二冗余数据，分别存储至各存储服务器。与传统的备份形式的存储方式 相比，通过提升关键视频的冗余数据数量，在提升了关键视频数据安全防护性能的同时，冗余数据比备份占用更少的存储空间。从而解决了以备份形式提升数据安全防护性能时，无法兼顾存储空间成本与数据安全防护性能的问题。
上述存储器可以包括RAM(Random Access Memory，随机存取存储器)，也可以包括NVM(Non-Volatile Memory，非易失性存储器)，例如至少一个磁盘存储器。可选的，存储器还可以是至少一个位于远离于上述处理器的存储装置。
上述处理器可以是通用处理器，包括CPU(Central Processing Unit，中央处理器)、NP(Network Processor，网络处理器)等；还可以是DSP(Digital Signal Processor，数字信号处理器)、ASIC(Application Specific Integrated Circuit，专用集成电路)、FPGA(Field-Programmable Gate Array，现场可编程门阵列)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。
本申请一实施例提供的计算机可读存储介质，包含于存储服务器，该计算机可读存储介质内存储有计算机程序，该计算机程序被处理器执行时，实现上述任一实施例中应用于存储服务器的数据存储方法的步骤。
本申请实施例提供的计算机可读存储介质，包含于存储服务器，存储的计算机程序被处理器执行时，可以实现通过由存储服务器根据获取的索引服务器发送的关键视频数据的数据信息，其中，关键视频数据为热度值大于或者等于预设热度阈值的、以基于第一纠删码算法得到的多个切片数据和第一预设数量个第一冗余数据的形式存储在各存储服务器中的视频数据。基于关键视频数据的数据信息，获取关键视频数据已存储的各切片数据；利用第二纠删码算法，对关键视频数据的各切片数据进行冗余计算，得到关键视频数据的大于第一预设数量的第二预设数量个第二冗余数据。将关键视频数据的第二预设数量个第二冗余数据，分别存储至各存储服务器。与传统的备份形式的存储方式相比，通过提升关键视频的冗余数据数量，在提升了关键视频数据安全防护性能的同时，冗余数据比备份占用更少的存储空间。从而解决了以备份形式提升数据安全防护性能时，无法兼顾存储空间成本与数据安全防护性能的问题。
本申请另一实施例提供的计算机可读存储介质，包含于索引服务器，该计算机可读存储介质内存储有计算机程序，该计算机程序被处理器执行时，实现上述任一实施例中应用于索引服务器的数据存储方法的步骤。
本申请实施例提供的计算机可读存储介质，包含于索引服务器，存储的计算机程序被处理器执行时，可以实现通过由索引服务器统计并分析用户对存储服务器中已存储的各原始视频数据进行操作的操作信息，得到关键视频数据的热度值，从而根据热度值确定关键视频数据、并获取关键视频数据的数据信息，向已存储所述关键视频数据的存储服务器发送数据信息。与人工选取关键视频并对关键视频手动备份的方式相比，根据用户对视频数据的操作信息确定关键视频数据，提升了关键视频数据选取的精准度和便捷性。同时，索引服务器发送的关键视频数据的数据信息使存储服务器能够根据该数据信息，获取关键视频数据已存储的各切片数据；利用第二纠删码算法，对关键视频数据的各切片数据进行冗余计算，得到关键视频数据的大于第一预设数量的第二预设数量个第二冗余数据。将关键视频数据的第二预设数量个第二冗余数据，分别存储至各存储服务器。与传统的备份形式的存储方式相比，通过提升关键视频的冗余数据数量，在提升了关键视频数据安全防护性能的同时，冗余数据比备份占用更少的存储空间。从而解决了以备份形式提升数据安全防护性能时，无法兼顾存储空间成本与数据安全防护性能的问题。
本说明书中的各个实施例均采用相关的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于存储服务器、索引服务器和云存储系统实施例而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素 的过程、方法、物品或者设备中还存在另外的相同要素。
以上所述仅为本申请的较佳实施例而已，并不用以限制本申请，凡在本申请的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本申请保护的范围之内。

Claims (13)

  1. 一种数据存储方法，其特征在于，应用于云存储系统中的存储服务器，所述云存储系统中包括多个存储服务器及至少一个索引服务器，所述方法包括：
    获取所述索引服务器发送的关键视频数据的数据信息，所述关键视频数据包括：热度值大于或者等于预设热度阈值的视频数据，所述关键视频数据以基于第一纠删码算法得到的多个切片数据和第一预设数量个第一冗余数据的形式存储在各存储服务器中；
    基于所述数据信息，获取所述关键视频数据已存储的各切片数据；
    利用第二纠删码算法，对所述关键视频数据已存储的各切片数据进行冗余计算，得到所述关键视频数据的第二预设数量个第二冗余数据；其中，所述第二预设数量大于所述第一预设数量；
    将所述关键视频数据的第二预设数量个第二冗余数据，分别存储至各存储服务器。
  2. 根据权利要求1所述的方法，其特征在于，在所述将所述关键视频数据的第二预设数量个第二冗余数据，分别存储至各存储服务器之后，所述方法还包括：
    删除各存储服务器中所述关键视频数据的第一预设数量个第一冗余数据。
  3. 根据权利要求1所述的方法，其特征在于，在所述将所述关键视频数据的第二预设数量个第二冗余数据，分别存储至各存储服务器之后，所述方法还包括：
    在检测到所述关键视频数据的第一子数据丢失时，如果所述第一子数据丢失的总数据数量小于或等于所述第二预设数量，则根据各存储服务器中所述关键视频数据的第二子数据，利用所述第二纠删码算法，还原所述第一子数据，其中，所述第一子数据为所述关键视频数据的切片数据和/或冗余数据，所述第二子数据为所述关键视频数据的所有切片数据和所有冗余数据中、除所述第一子数据以外的数据。
  4. 根据权利要求1所述的方法，其特征在于，所述热度值由所述索引服务器、通过统计并分析用户对所述存储服务器中已存储的各视频数据进行操作的操作信息得到。
  5. 根据权利要求1-4任一项所述的方法，其特征在于，所述利用第二纠删码算法，对所述关键视频数据已存储的各切片数据进行冗余计算，得到所述关键视频数据的第二预设数量个第二冗余数据，包括：
    利用与所述关键视频数据的热度值K对应的纠删码算法，对所述关键视频数据的各切片数据进行冗余计算，得到所述关键视频数据的第二预设数量个第二冗余数据，其中，所述K为所述热度值编号，所述第二预设数量与所述热度值K对应。
  6. 一种存储服务器，其特征在于，所述存储服务器包括：
    任务执行模块，用于获取所述索引服务器发送的关键视频数据的数据信息，所述关键视频数据包括：热度值大于或者等于预设热度阈值的视频数据，所述关键视频数据以基于第一纠删码算法得到的多个切片数据和第一预设数量个第一冗余数据的形式存储在各存储服务器中；基于所述数据信息，获取所述关键视频数据已存储的各切片数据；利用第二纠删码算法，对所述关键视频数据已存储的各切片数据进行冗余计算，得到所述关键视频数据的第二预设数量个第二冗余数据；其中，所述第二预设数量大于所述第一预设数量；
    数据存储模块，用于将所述关键视频数据的第二预设数量个第二冗余数据，分别存储至各存储服务器。
  7. 根据权利要求6所述的存储服务器，其特征在于，所述数据存储模块，还用于：
    删除各存储服务器中所述关键视频数据的第一预设数量个第一冗余数据。
  8. 根据权利要求6所述的存储服务器，其特征在于，所述存储服务器还包括：
    数据重构模块，用于在检测到所述关键视频数据的第一子数据丢失时， 如果所述第一子数据丢失的总数据数量小于或等于所述第二预设数量，则根据各存储服务器中所述关键视频数据的第二子数据，利用所述第二纠删码算法，还原所述第一子数据，其中，所述第一子数据为所述关键视频数据的切片数据和/或冗余数据，所述第二子数据为所述关键视频数据的所有切片数据和所有冗余数据中、除所述第一子数据以外的数据。
  9. 根据权利要求6所述的存储服务器，其特征在于，所述热度值由所述索引服务器、通过统计并分析用户对所述存储服务器中已存储的各视频数据进行操作的操作信息得到。
  10. 根据权利要求6-9任一项所述的存储服务器，其特征在于，所述任务执行模块，具体用于：
    利用与所述关键视频数据的热度值K对应的纠删码算法，对所述关键视频数据的各切片数据进行冗余计算，得到所述关键视频数据的第二预设数量个第二冗余数据，其中，所述K为所述热度值编号，所述第二预设数量与所述热度值K对应。
  11. 一种云存储系统，其特征在于，所述系统包括：多个存储服务器及至少一个索引服务器；
    所述存储服务器，用于获取所述索引服务器发送的关键视频数据的数据信息，所述关键视频数据包括：热度值大于或者等于预设热度阈值的视频数据，所述关键视频数据以基于第一纠删码算法得到的多个切片数据和第一预设数量个第一冗余数据的形式存储在各存储服务器中；基于所述数据信息，获取所述关键视频数据已存储的各切片数据；利用第二纠删码算法，对所述关键视频数据的各切片数据进行冗余计算，得到第二预设数量个第二冗余数据；其中，所述第二预设数量大于所述第一预设数量；将所述关键视频数据的第二预设数量个第二冗余数据，分别存储至各存储服务器；
    所述索引服务器，用于统计并分析用户对存储服务器中已存储的各视频数据进行操作的操作信息，得到各视频数据的热度值；将所述热度值大于或等于预设阈值的视频数据确定为关键视频数据，获取所述关键视频数据的数据信息；向已存储所述关键视频数据的存储服务器发送关键视频数据的数据 信息。
  12. 一种存储服务器，其特征在于，包括处理器、通信接口、存储器和通信总线，其中，处理器，通信接口，存储器通过总线完成相互间的通信；存储器，用于存放计算机程序；处理器，用于执行存储器上所存放的程序，实现如权利要求1-5任一所述的方法步骤。
  13. 一种计算机可读存储介质，其特征在于，所述存储介质内存储有计算机程序，所述计算机程序被处理器执行时实现如权利要求1-5任一所述的方法步骤。
PCT/CN2019/075698 2018-04-23 2019-02-21 数据存储方法、存储服务器及云存储系统 WO2019205788A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810366428.6 2018-04-23
CN201810366428.6A CN110389715B (zh) 2018-04-23 2018-04-23 数据存储方法、存储服务器及云存储系统

Publications (1)

Publication Number Publication Date
WO2019205788A1 true WO2019205788A1 (zh) 2019-10-31

Family

ID=68284468

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/075698 WO2019205788A1 (zh) 2018-04-23 2019-02-21 数据存储方法、存储服务器及云存储系统

Country Status (2)

Country Link
CN (1) CN110389715B (zh)
WO (1) WO2019205788A1 (zh)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113127258A (zh) * 2019-12-30 2021-07-16 华为技术有限公司 一种数据备份方法、装置、设备及介质
CN114416437B (zh) * 2022-01-17 2022-11-25 杭州字节方舟科技有限公司 一种自适应冗余级别的数据存储方法
CN114116321A (zh) * 2022-01-25 2022-03-01 苏州浪潮智能科技有限公司 一种冗余数据管理方法、装置、计算机设备和存储介质
CN115113819A (zh) * 2022-06-29 2022-09-27 京东方科技集团股份有限公司 一种数据存储的方法、单节点服务器及设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110029840A1 (en) * 2009-07-31 2011-02-03 Microsoft Corporation Erasure Coded Storage Aggregation in Data Centers
CN103118133A (zh) * 2013-02-28 2013-05-22 浙江大学 基于文件访问频次的混合云存储方法
CN103631666A (zh) * 2012-08-24 2014-03-12 中兴通讯股份有限公司 数据冗余容错的适配管理设备、服务设备、系统及方法
CN103838860A (zh) * 2014-03-19 2014-06-04 华存数据信息技术有限公司 一种基于动态副本策略的文件存储系统及其存储方法
CN106649406A (zh) * 2015-11-04 2017-05-10 华为技术有限公司 一种自适应存储文件的方法和装置

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101840377A (zh) * 2010-05-13 2010-09-22 上海交通大学 基于rs纠删码的数据存储方法
CN105320899B (zh) * 2014-07-22 2018-01-09 北京大学 一种面向用户的云存储数据完整性保护方法
CN105335150B (zh) * 2014-08-13 2019-03-19 苏宁易购集团股份有限公司 纠删码数据的快速编解码方法和系统
CN106331001B (zh) * 2015-06-23 2019-06-18 华中科技大学 一种适合移动设备访问的云存储方法和系统
US10318389B2 (en) * 2016-07-15 2019-06-11 Quantum Corporation Joint de-duplication-erasure coded distributed storage
CN106603673A (zh) * 2016-12-19 2017-04-26 上海交通大学 基于纠删码的细粒度云存储调度方法
CN106776111A (zh) * 2017-01-06 2017-05-31 东北大学 一种基于lrc纠删码的可恢复云存储系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110029840A1 (en) * 2009-07-31 2011-02-03 Microsoft Corporation Erasure Coded Storage Aggregation in Data Centers
CN103631666A (zh) * 2012-08-24 2014-03-12 中兴通讯股份有限公司 数据冗余容错的适配管理设备、服务设备、系统及方法
CN103118133A (zh) * 2013-02-28 2013-05-22 浙江大学 基于文件访问频次的混合云存储方法
CN103838860A (zh) * 2014-03-19 2014-06-04 华存数据信息技术有限公司 一种基于动态副本策略的文件存储系统及其存储方法
CN106649406A (zh) * 2015-11-04 2017-05-10 华为技术有限公司 一种自适应存储文件的方法和装置

Also Published As

Publication number Publication date
CN110389715B (zh) 2021-07-09
CN110389715A (zh) 2019-10-29

Similar Documents

Publication Publication Date Title
WO2019205788A1 (zh) 数据存储方法、存储服务器及云存储系统
US9858013B1 (en) Scalable distributed storage system infrastructure
CN108418711B (zh) 故障管理的存储介质
US10108630B2 (en) Cluster unique identifier
WO2018001110A1 (zh) 一种基于纠删码的存储数据重构方法和装置、存储节点
US9450700B1 (en) Efficient network fleet monitoring
CN107544832B (zh) 一种虚拟机进程的监控方法、装置和系统
AU2018202230A1 (en) Client-configurable security options for data streams
US11593029B1 (en) Identifying a parent event associated with child error states
WO2018233630A1 (zh) 故障发现
US20160088103A1 (en) System and method for optimizing visual session recording for user account management in a computing environment
US8589441B1 (en) Information processing system and method for controlling the same
US10303678B2 (en) Application resiliency management using a database driver
US9652307B1 (en) Event system for a distributed fabric
WO2019237774A1 (zh) 一种云存储系统中的视频数据存储方法及装置
WO2020048488A1 (zh) 一种数据存储方法及存储装置
CN111176888A (zh) 云存储的容灾方法、装置及系统
US20220237326A1 (en) System and method for certifying integrity of data assets
CN113312656B (zh) 数据轮转方法、装置、设备及系统
US20120158673A1 (en) Storing and publishing contents of a content store
EP3349416B1 (en) Relationship chain processing method and system, and storage medium
CN109510730A (zh) 分布式系统及其监控方法、装置、电子设备及存储介质
US10248508B1 (en) Distributed data validation service
CN106998436B (zh) 录像备份方法及装置
CN115756955A (zh) 一种数据备份、数据恢复的方法、装置及计算机设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19792467

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19792467

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 19792467

Country of ref document: EP

Kind code of ref document: A1