US7899988B2 - Video media data storage system and related methods - Google Patents

Video media data storage system and related methods Download PDF

Info

Publication number
US7899988B2
US7899988B2 US12/039,129 US3912908A US7899988B2 US 7899988 B2 US7899988 B2 US 7899988B2 US 3912908 A US3912908 A US 3912908A US 7899988 B2 US7899988 B2 US 7899988B2
Authority
US
United States
Prior art keywords
data storage
video media
data
storage devices
pluralities
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related, expires
Application number
US12/039,129
Other versions
US20090222622A1 (en
Inventor
Mihai G. Petrescu
Hilton S. Creve
Tung M. Tran
Todd S. Roth
Theodore H. Korte
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
HBC Solutions Inc
Original Assignee
Harris Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harris Corp filed Critical Harris Corp
Priority to US12/039,129 priority Critical patent/US7899988B2/en
Assigned to HARRIS CORPORATION reassignment HARRIS CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: PETRESCU, MIHAI G., CREVE, HILTON S., KORTE, THEODORE H., ROTH, TODD S., TRAN, TUNG M.
Priority to EP09715995.8A priority patent/EP2271986B1/en
Priority to PCT/US2009/034942 priority patent/WO2009108609A1/en
Priority to CN2009801067241A priority patent/CN101960429B/en
Priority to CA2715967A priority patent/CA2715967C/en
Publication of US20090222622A1 publication Critical patent/US20090222622A1/en
Publication of US7899988B2 publication Critical patent/US7899988B2/en
Application granted granted Critical
Assigned to HBC SOLUTIONS, INC. reassignment HBC SOLUTIONS, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: EAGLE TECHNOLOGY INC., HARRIS CORPORATION
Assigned to WILMINGTON TRUST, NATIONAL ASSOCIATION reassignment WILMINGTON TRUST, NATIONAL ASSOCIATION SECURITY AGREEMENT Assignors: HBC SOLUTIONS, INC.
Assigned to WILMINGTON TRUST, NATIONAL ASSOCIATION reassignment WILMINGTON TRUST, NATIONAL ASSOCIATION SECURITY AGREEMENT Assignors: HB CANADA COMMUNICATIONS LTD
Assigned to PNC BANK, NATIONAL ASSOCIATION, AS AGENT reassignment PNC BANK, NATIONAL ASSOCIATION, AS AGENT SECURITY AGREEMENT Assignors: HBC SOLUTIONS, INC.
Assigned to HBC SOLUTIONS, INC. reassignment HBC SOLUTIONS, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: EAGLE TECHNOLOGY, LLC, HARRIS CORPORATION
Expired - Fee Related legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14Error detection or correction of the data by redundancy in operation
    • G06F11/1402Saving, restoring, recovering or retrying
    • G06F11/1415Saving, restoring, recovering or retrying at system level
    • G06F11/1435Saving, restoring, recovering or retrying at system level using file system or storage system metadata
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16Error detection or correction of the data by redundancy in hardware
    • G06F11/20Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
    • G06F11/2053Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant
    • G06F11/2056Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant by mirroring
    • G06F11/2084Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant by mirroring on the same storage unit
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16Error detection or correction of the data by redundancy in hardware
    • G06F11/20Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
    • G06F11/2053Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant
    • G06F11/2056Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant by mirroring
    • G06F11/2087Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant by mirroring with a common controller
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21Server components or server architectures
    • H04N21/218Source of audio or video content, e.g. local disk arrays
    • H04N21/21815Source of audio or video content, e.g. local disk arrays comprising local storage units
    • H04N21/2182Source of audio or video content, e.g. local disk arrays comprising local storage units involving memory arrays, e.g. RAID disk arrays
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/231Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
    • H04N21/23116Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion involving data replication, e.g. over plural servers
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/232Content retrieval operation locally within server, e.g. reading video streams from disk arrays
    • H04N21/2323Content retrieval operation locally within server, e.g. reading video streams from disk arrays using file mapping
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2211/00Indexing scheme relating to details of data-processing equipment not covered by groups G06F3/00 - G06F13/00
    • G06F2211/10Indexing scheme relating to G06F11/10
    • G06F2211/1002Indexing scheme relating to G06F11/1076
    • G06F2211/1016Continuous RAID, i.e. RAID system that allows streaming or continuous media, e.g. VOD
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2211/00Indexing scheme relating to details of data-processing equipment not covered by groups G06F3/00 - G06F13/00
    • G06F2211/10Indexing scheme relating to G06F11/10
    • G06F2211/1002Indexing scheme relating to G06F11/1076
    • G06F2211/1045Nested RAID, i.e. implementing a RAID scheme in another RAID scheme

Definitions

  • the present invention relates to the field of data storage, and, more particularly, to video data storage networks and related methods.
  • crash is defined as the sudden failure of a software application or operating system, or of a hardware device such as a hard disk. While a software crash usually creates an interruption in service, the dreaded hard disk crash also tends to cause the loss of data. Due to the mechanical nature of a fast-spinning disk drive (e.g., 10,000 RPM), crashes are usually permanent and the need for data protection becomes critical.
  • a fast-spinning disk drive e.g. 10,000 RPM
  • MTBF mean time between failure
  • BER bit error rate
  • Other errors that can result in seek, read and write failures are usually masked by successful retries. Since many drives now come with MTBF's upward of one million hours, BER's on the order of 1 in 1 ⁇ 10 15 and warranties up to 5 years, it makes economic sense for the vendors to be able to distinguish between a drive that has failed from one that is merely “having occasional trouble.” To reduce warranty replacement costs, drives hedge against excessive “failures” by employing many internal data recovery mechanisms, including error correction, retries, and sector remapping. Only when a drive exceeds its retry count and runs out of sectors for remapping is it considered “failed.”
  • AFR annualized failure rate
  • RAID redundant array of independent drives
  • RAIDs employ “striping,” meaning that the data stream is divided into segments or blocks that are stored on separate drives in the RAID. Parity data may be used with data striping, which allows data from a faulty drive or sector(s) to be reconstructed from data on the other data drives in the RAID and the parity data.
  • Another RAID technique is mirroring data on multiple drives within a same RAID set. That is, a copy of a data set is stored/maintained on a separate drive so that if one of the drives goes down, the duplicate data set is immediately available from the mirror drive.
  • U.S. Pat. No. 7,225,315 is directed to a file system including a storage system having a plurality of volumes, a volume allocation table adapted to set the plurality of volumes for each directory, a file allocation table that stores attributes and divided block information of the file, a block reading table in which numbers of blocks read out in one reading operation for each volume are respectively set, and a read control module that controls data read from the volume.
  • a read control module when a read command is received, determines a volume to be read from the volume allocation table.
  • the read control module further determines the number of blocks read for each volume by referring to the block reading table, determines the blocks read for each volume based on the volume, the number of blocks, and the block information, and reads from each volume in parallel.
  • a video media data storage system which may include first and second pluralities of data storage devices each arranged in a redundant array of independent drives (RAID) configuration for permitting writing and reading of video media data.
  • the system may further include at least one memory controller coupled to the first and second pluralities of data storage devices for performing mirrored writing of video media data to both the first and second pluralities of data storage devices.
  • the at least one memory controller may also be for generating respective first and second file allocation tables (FATs) including video media data time stamps and validity information for both of the first and second pluralities of data storage devices, and selectively reading video media data from the first and second pluralities of data storage devices based upon the first and second FATs.
  • FATs file allocation tables
  • the at least one memory controller may also be for performing data recovery between the first and second plurality of data storage devices from the mirrored video media data based upon the first and second FATs. Moreover, the at least one memory controller may write the video media data in each of the first and second pluralities of data storage devices as striped video media data, and generate parity data from the striped video media. The at least one memory controller may therefore also perform data recovery within a given one of the first and second pluralities of data storage devices based upon the parity data. Additionally, the at least one memory controller may also advantageously select between using the mirrored video media data and the parity data for performing data recovery based upon a number of data storage devices having faults within the given one of the first and second pluralities of data storage devices.
  • the validity information may include data storage device fault information, for example.
  • the at least one memory controller may give reading preference to data storage devices without a fault and with a most recent video media data time stamp.
  • the at least one memory controller may also perform load balancing for reading the video media data from the first and second pluralities of data storage devices.
  • the first and second pluralities of data storage devices may each be arranged in a RAID 3 or higher configuration.
  • the at least one memory controller may include a first memory controller for the first plurality of data storage devices, and a second memory controller for the second plurality of data storage devices.
  • the system may further include first and second fibre channel (FC) switches respectively coupling the first and second memory controllers to the first and second pluralities of data storage devices.
  • the first memory controller may be coupled to the second FC switch, and the second memory controller may be coupled to the first FC switch.
  • the first and second pluralities of data storage devices and the at least one memory controller may be connected together in a storage area network (SAN), for example. Further, the at least one memory controller may be at least one broadcast video server.
  • SAN storage area network
  • the at least one memory controller may be at least one broadcast video server.
  • a video media data storage method aspect may include providing first and second pluralities of data storage devices each arranged in a redundant array of independent drives (RAID) configuration for permitting writing and reading of video media data, and performing mirrored writing of video media data to both the first and second pluralities of data storage devices.
  • the method may further include generating respective first and second file allocation tables (FATs) including video media data time stamps and validity information for both of the first and second pluralities of data storage devices.
  • the method may also include selectively reading video media data from the first and second pluralities of data storage devices based upon the first and second FATs.
  • FATs file allocation tables
  • FIG. 1 is a schematic block diagram of a video media data storage system in accordance with the invention.
  • FIG. 2 is a schematic block diagram of an alternative embodiment of the video media data storage system of FIG. 1 .
  • FIG. 3 is a RAID map of the mirrored RAID sets of FIG. 1 illustrating mirrored writing thereto.
  • FIG. 4 is a flow diagram illustrating video media data writing operations of the system of FIG. 1 .
  • FIG. 5 is a schematic diagram of the first and second pluralities of data storage devices from the system of FIG. 1 with accompanying file allocation tables (FATS) and data stripes.
  • FATS file allocation tables
  • FIG. 6 is a flow diagram illustrating video media data reading operations of the system of FIG. 1 .
  • FIG. 7 is a RAID map of the first and second pluralities of data storage devices of the system of FIG. 1 and their associated FAT tables.
  • FIGS. 8A and 8B are respective FAT tables for the first and second pluralities of data storage devices of the system of FIG. 1
  • FIG. 8C is a corresponding operation table showing read operations by the memory controller(s) of the system of FIG. 1 based upon the information in the FAT tables of FIGS. 8A and 8B .
  • FIGS. 9-13 are RAID drive maps illustrating various video media data recovery scenarios of the system of FIG. 1 .
  • FIG. 14 is a flow diagram generally illustrating reading and writing operations in accordance with the invention.
  • FIG. 15 is flow diagram illustrating data recovery operations in accordance with another advantageous aspect of the invention.
  • FIG. 16 is a graph comparing RAID 3 vs. ECC overhead for parity configurations that may be implemented in the system of FIG. 1 .
  • the system 30 illustratively includes first and second pluralities or sets of data storage devices D 1 and D 2 each arranged in a redundant array of independent drives (RAID) configuration for permitting writing and reading of video media data.
  • video media data includes video, graphics, automation and/or animation, and associated audio data, for example.
  • one or more memory controllers 31 are illustratively coupled to the first and second RAID sets D 1 , D 2 .
  • the data storage devices may be magnetic or optical disks, although other suitable data storage devices (e.g., FLASH drives, etc.), may also be used.
  • the system 30 may be particularly well suited for applications where a large bandwidth is required, such as in broadcast or streaming video data applications.
  • the video data feed is generated by video cameras 32
  • video media data from the system 30 may be communicated to recipients by one or more communications mediums, such as a satellite link (illustrated by a satellite station 33 in FIG. 1 ), co-axial/fiber connections, wide area networks (WANs) such as the Internet, etc., as will be appreciated by those skilled in the art.
  • WANs wide area networks
  • the memory controller(s) 31 performs mirrored writing of video media data to both the first and second RAID sets D 1 , D 2 (Blocks 141 - 142 ).
  • the data in a given RAID set D 1 , D 2 is written in data stripes (i.e., divided in blocks or sections each written to a different storage device), and the striped data is mirrored in both of the RAID sets D 1 , D 2 , as will be discussed further below. This mirrored data writing is illustrated in FIG. 3 .
  • the memory controller(s) 31 also generates respective first and second file allocation tables (FATs) including video media data time stamps and validity information (e.g., drive/sector fault information, etc.) for both of the first and second RAID sets D 1 , D 2 , at Block 143 . Since both of the FATs are available to the memory controller(s) 31 , it can advantageously selectively read video media data from the first and second RAID sets D 1 , D 2 based upon the first and second FATs, at Block 144 , thus concluding the method illustrated in FIG. 14 (Block 145 ). That is, the memory controller(s) 31 has more flexibility in performing read operations, which may be particularly important where there are multiple drive faults. For example, the memory controller(s) 31 may give reading preference to data storage devices without a fault and with a most recent video media data time stamp. Moreover, this also advantageously allows load balancing to be used to reduce read times and increase data throughput, for example.
  • FATs file allocation tables
  • this configuration also allows the memory controller(s) 31 to perform data recovery between the first and second RAID sets D 1 , D 2 from the mirrored video media data based upon the first and second FATs. Stated alternatively, this allows inter-RAID data recovery (i.e., from RAID set D 1 to D 2 , or vice-versa). More particularly, the memory controller(s) 31 may write the video media data in each of the first and second RAID sets D 1 , D 2 as striped video media data, and generate parity data from the striped video media data to perform data recovery within a given RAID set based thereon (i.e., intra-RAID data recovery), as will be discussed further below.
  • the memory controller(s) 31 will, generally speaking, select between using the mirrored video media data (inter-RAID recovery) and the parity data (intra-RAID recovery) for performing data recovery based upon a number of data storage devices having faults within the first and second RAID sets D 1 , D 2 , as will also be discussed further below.
  • the system 30 ′ is implemented as a storage area network (SAN) and includes respective broadcast video servers as the memory controllers 31 a ′, 31 b ′.
  • SAN storage area network
  • the memory controllers 31 a ′, 31 b ′ are suitable broadcast video servers.
  • AMP Nexio Advanced Media Platform
  • other suitable memory controllers/servers may also be used in different embodiments, as will be appreciated by those skilled in the art.
  • the first and second RAID sets D 1 ′, D 2 ′ are arranged in a RAID level 3 (or higher) configuration and implement data striping and parity (e.g., error correction code (ECC) parity), as will be appreciated by those skilled in the art.
  • the parity data is stored in one or more dedicated parity data storage devices 40 a ′, 40 b ′,
  • first and second fibre channel (FC) switches 36 a ′, 36 b ′ are respectively coupled to the first and second memory controllers 31 a ′, 31 b ′ and the first and second RAID sets D 1 ′, D 2 ′. That is, the first and second RAID sets D 1 ′, D 2 ′ are on separate FC domains to advantageously provide high redundancy and availability, as will be appreciated by those skilled in the art.
  • FC fibre channel
  • first memory controller 31 a ′ is illustratively coupled to the second FC switch 36 b ′
  • second memory controller 31 b ′ is illustratively coupled to the first FC switch 36 a ′.
  • additional FC switches may also be used to provide additional access points to the RAID sets D 1 ′, D 2 ′, for example.
  • a SAN and/or FC network configuration is not required in all embodiments.
  • parity information is simply used to reconstruct errant data after the problem is discovered.
  • parity information may be continuously read and decoded, improving the capability of error detection as well as data performance during error correction.
  • intra-RAID set parity may be implemented in the system 30 or 30 ′ as follows. Data is organized into buffers or stripes and is stored on respective drives 1 -P, and each partial data block size is defined as:
  • the memory controllers 31 a ′, 31 b ′ advantageously read the parity data and use it to verify data integrity, rather than simply using this information for after-the-fact data recovery.
  • error handling can take the form of either detection or correction.
  • the parity equation serves three purposes depending on the error cases.
  • a first error case is when a partial data block is not returned, or is reported in error. The defective partial is still reconstructed by the XOR resultant of the good partials and the parity. This case is known as single-error correcting (SEC).
  • SEC single-error correcting
  • a second case is when all partial data and parity is returned and no errors are reported by the drive. If all the data and parity is in fact good, the equation simply rebuilds the buffer. If any partial is detective, the error is detected and the damaged buffer is discarded. For the sake of video (and audio) information, data may be represented as black and silence can be substituted. This case is known as single-error detecting (SED). Still another case is where the partial data set has multiple defective elements. In this eventuality the data cannot be corrected, and again for the sake of video (and audio) information, data may be represented as black and silence can be substituted. This case is known as multiple-error detecting (MED).
  • SED single-error detecting
  • a failing partial usually represents a specific failing drive (resolving the issue of knowing which partial is in error), which reduces the model to the first and third cases noted above.
  • a failing drive By relying on the drive to know when it is having a problem and not always running data through XOR verification, most RAID systems typically ignore the second case, leaving the system vulnerable or erroneous unreported data.
  • the present approach implemented by the memory controllers 31 a ′, 31 b ′ of using parity data for error detection may potentially detect this erroneous data and replace it with black to maintain a continuous uninterrupted video stream. This same approach may also be used for audio to reduce “pops” and “clicks”.
  • Another advantage of always keeping a running, on the fly, parity count is an overall performance improvement in the case of single corrected errors. If a drive fails, there is no need to go back and re-read the previous partials. Rather, the memory controllers 31 a ′, 31 b ′ may skip over the failed device and reconstruct its data contribution using the accrued parity information. This advantageously results in no net decrease in system performance during a “degraded” mode of operation.
  • a single parity drive RAID can only protect a data stripe from a single data element failure, the odds of an uncorrectable failure increase with the amount of partials involved in the stripe. As more data elements (drives in this case) are added, the greater the likelihood of an uncorrectable double data failure.
  • the primary tradeoff of using a single parity RAID is that as sets grow larger they become much more susceptible to uncorrectable double errors.
  • the system 30 ′ therefore advantageously utilizes multiple, mirrored RAID sets D 1 ′, D 2 ′ to provide further error recovery flexibility (in addition to the additional data reading flexibility noted above), as will be discussed further below.
  • a third case involves a combination of multiple unknown partials.
  • the data cannot be corrected.
  • data representing black and/or silence can be inserted.
  • This case is multiple-error detecting (MED).
  • Comparing the relative cost of single parity and ECC in terms of additional overhead can be useful in determining which strategy is best suited for the given application. This can be done by comparing the parity overhead of a drive population containing a multiple RAID 3 set to the same population arranged in a single ECC set.
  • the graph of FIG. 16 examines both approaches for an example in which the maximum RAID 3 stripe length is set to 9 drives (8 data+1 parity).
  • ECC systems begin to become advantageous at around 25 drives (data and parity) with an overhead less than twice that of the multiple RAID 3 set. As systems begin to reach 50 drives, the overhead of ECC actually becomes less than that of RAID 3, while continuing a higher level of protection. It should be noted that either of these approaches (or other suitable approaches) may be used at the RAID set level in the system of 30 ′.
  • RAID 3 and ECC protect against drive related media and data errors within a given RAID
  • data mirroring between different RAIDS affords protection against not only multiple drive errors, but also drive enclosure failures.
  • this added layer of redundancy increases the storage system costs (i.e., because it uses an additional RAID set), it significantly increases data survivability and on-air availability.
  • mirroring can be done between two identical RAID 3 or ECC drive sets. Writing is accomplished by sending the data partials and parities to two identical sets of drives D 1 ′, D 2 ′, which results in the creation of a fully redundant data set, as seen in FIG. 3 .
  • identical it is meant that the RAID sets D 1 ′, D 2 ′ have at least some corresponding drives that are the same, although these RAID sets could have other or extra drives which are not part of the mirror configuration in some embodiments.
  • the memory controller(s) 31 initially attempts to write the video media data stripe to one of the RAID data sets (D 1 in the present example), at Block 41 . If writing to the RAID set D 1 cannot be completed successfully, such as due to a drive fault, etc., at Block 42 , then the memory controller(s) 31 attempts to write the video media data stripe to the second RAID set D 2 , at Block 43 .
  • the memory controller(s) 31 updates the FAT tables to indicate the RAID data set D 1 storage device(s) that is bad, and that the RAID data set D 2 is ON, at Block 45 , and to write a time stamp TS indicating the time of writing. Otherwise, both of the RAID sets D 1 and D 2 have one or more bad storage devices/sectors, and the first and second FATS are updated accordingly, at Block 46 .
  • the memory controller(s) 31 determines whether this writing operation is successful, at Block 45 .
  • the first and second FAT tables are updated accordingly for the case where the write operation to the RAID set D 2 was successful (Block 49 ), and where it was not (Block 50 ), as described above.
  • the memory controller(s) 31 writes the first and second FAT tables to their respective first and second RAID sets D 1 , D 2 , at Blocks 50 - 51 , thus concluding the illustrated writing operations (Block 52 ).
  • Exemplary FATs for a mirrored RAID set configuration are shown in FIG. 5 . Since a likely failure scenario involves not only write failures but also the possibility of loss of an entire RAID set, it is desirable to protect against these failure modes.
  • the FATs for each RAID set D 1 , D 2 advantageously include sector indexing information, timestamps, and data OK flags for both of the RAID sets, as seen in FIG. 5 . If a write fails on either side of the mirror (i.e., in either RAID set), the corresponding data OK flags in both FATS are cleared. This way subsequent reads will know to ignore these data elements, as discussed further below.
  • One particularly advantageous feature of the writing/reading/recovery operations described herein is that they may be implemented using a software application running at the memory controller/sever.
  • suitable software application that may be used to implement the approach described herein is the RAIDSoft application from Harris Corporation.
  • the parity operations/calculations are performed by the memory controller(s) 31 , and the data and localized parity are then written in a mirrored fashion to both of the RAID sets D 1 , D 2 , as discussed above. All of the information is mirrored at the block level, with the exception of the FAT tables.
  • the FAT tables differ between the two RAID sets D 1 , D 2 to track bad sectors/drives and offline conditions.
  • the FAT information for both RAID sets D 1 , D 2 may also be stored in a memory such as a random access memory (RAM) in some embodiments as well to provide faster operation by the memory controller(s) 31 .
  • RAM random access memory
  • data reads are performed once and load balanced between the first and second RAID sets D 1 , D 2 if both sets contain good (i.e., without fault and/or up-to-date) data. Unrecorded or faulty sectors/drives are noted in the FATS and not used in read operations. If the load balanced read returns an error, then the other copy of the data is immediately read with little or no disturbance to air playout in broadcast scenarios, for example. In the case of a RAID set being down, the down set will not be accessed again until the problem is cleared (e.g., drive replaced, etc.). Probe commands may be periodically sent to determine when the down set returns to operations, for example, as will be appreciated by those skilled in the art.
  • the memory controller(s) 31 first checks the first and second FAT tables to see if the data is good for a given block of data in both of the first and second RAID sets D 1 , D 2 (Blocks 61 , 62 ), as illustrated in FIG. 7 .
  • the results of the FAT tables are compared, such as through a logical OR operation (Block 63 ), to determine the four possible fault conditions for the drives in questions, namely: (a) the RAID set D 1 is OK but RAID set D 2 is BAD (Block 64 ); (b) both RAID sets are OK (Block 65 ); (c) RAID set D 1 is BAD but RAID set D 2 is OK (Block 66 ); and (d) both RAID sets are bad for the given block of data (Block 67 ).
  • the memory controller(s) 31 attempts to read the first RAID set D 1 , at Block 68 , and if the read is completed successfully (Block 69 ) then the data is output accordingly, at Block 70 . If the read operation is not successful for whatever reason, then black and/or silence may be output, at Block 71 .
  • the memory controller(s) 31 checks to see which RAID set D 1 , D 2 has the most up-to-date data (i.e., compares the time stamps), at Block 72 . If the time stamp for the RAID set D 1 time stamp is more recent, then the memory controller 31 reads the first RAID set as described above at Blocks 68 - 71 .
  • the memory controller(s) 31 may read from either RAID set D 1 , D 2 , and the choice between the two may be based upon a load balancing algorithm (i.e., checks which one is busiest at the time and uses the other), and/or a preferential (i.e., default) scheme may be used. For example, in the illustrated embodiment an affinity is given to odd or even RAID sets, such that odd-numbered memory controllers (e.g., controller # 1 ) would first access an odd-numbered RAID set (i.e., RAID set D 1 ), and vice-versa.
  • a load balancing algorithm i.e., checks which one is busiest at the time and uses the other
  • a preferential (i.e., default) scheme may be used. For example, in the illustrated embodiment an affinity is given to odd or even RAID sets, such that odd-numbered memory controllers (e.g., controller # 1 ) would first access an odd-numbered RAID set (i.e., RAID set D
  • the affinity is for the first RAID set D 1 , which the memory controller(s) 31 attempts to read at Block 73 . If the read is successful, at Block 74 , then the data is output (Block 75 ). If not, the memory controller(s) 31 still has the option of reverting back to the second RAID set D 2 . If the attempted read from the second RAID set D 2 is successful, at Blocks 76 - 77 , then the data is output, at Block 78 . Otherwise, black and/or silence data is output, at Block 79 .
  • the memory controller(s) 31 attempts to read from the second RAID set D 2 in the same manner described above with reference to Blocks 76 - 79 .
  • black and/or silence data is output, at Block 79 .
  • all of the drives are without fault and have equal time stamps (meaning both sets have the freshest or most up-to-date data). It should be noted that in some embodiments reading may also be performed from a faulty drive using the parity data described above if the mirrored data is bad or “stale”, as will be appreciated by those skilled in the art, although this is not illustrated in FIG. 6 for clarity of explanation.
  • both FATs reflect that drive d 1 is OK in both RAID sets, and that their time stamps are equal.
  • both data stripes have “good” data and either can be successfully read based upon affinity, load balancing, etc.
  • both FATS indicate that the drive d 2 is OK in the first RAID set D 1 , but the second FAT indicates that the drive d 2 in the second RAID set D 2 is bad, and that the time stamp for drive d 1 in RAID set D 1 is newer than for the drive d 2 of the second RAID set D 2 .
  • the first data stripe i.e., RAID set D 1
  • the drive d 4 is listed as bad in the second FAT table, and the time stamp for the drive d 4 in the second FAT is also newer.
  • the first data stripe is again considered bad, and the second data strip (i.e., RAID set D 2 ) is read.
  • both FAT tables indicate that the RAID sets D 1 and D 2 are OK, but the time stamp for the drive d 7 of the second RAID set is newer, indicating that the data on the drive d 7 of the first RAID set is stale.
  • the memory controller(s) 31 considers the first data stripe to be old or stale, and reads from the drive d 7 of the second RAID set.
  • both FAT tables indicate that the RAID sets D 1 and D 2 are OK, but the time stamp for the drive d 7 of the first RAID set is newer, indicating that the data on the drive d 7 of the second RAID set is stale. Accordingly, the memory controller(s) 31 considers the second data stripe to be old or stale, and reads from the drive d 7 of the first RAID set.
  • the data recovery steps performed by the memory controller(s) 31 are now further described.
  • the recovered data is preferably always the “freshest” copy, taken from whichever RAID set D 1 , D 2 is appropriate. If the freshest data had a read fault from an individual disk, then the inter-RAID set mirror data is used to recover the data.
  • the parity data is generated for the first and second RAID sets D 1 , D 2 as discussed above (Block 150 ).
  • the parity data may be generated for one of the RAID sets D 1 , D 2 , and then copied over as part of the mirrored data set, although in some embodiments parity data could be generated independently for each of the RAID sets, if desired.
  • the memory controller(s) 31 determines which of the storage devices in the first and second RAID sets require recovery or data freshening, at Block 152 , based upon the OK flags and the time stamps, for example.
  • the way in which the memory controller(s) will perform data recovery will depend upon the number of storage devices/drives that are affected. More particularly, if only a single drive in one of the RAID sets D 1 or D 2 is affected, the data is recovered on the fly from the intra-RAID set parity data, at Block 154 , thus concluding the illustrated example (Block 155 ).
  • the drive d 4 in the RAID set D 1 is bad (or has stale date), as indicated by the dashed markings.
  • the data may be recovered using the parity data stored in the drive P in the first RAID set, and thus it is not necessary to go to the mirrored data in the second RAID set D 2 for recovery (although this could be done, if desired).
  • the user would replace the bad drive and the recovery operations may advantageously be performed automatically by the memory controller(s) 31 once it detects that the drive is no longer faulty, for example. It should be noted that even if the corresponding partials from both RAID sets D 1 , D 2 are damaged, parity data from either set can be used to reconstruct the data buffer.
  • inter-RAID recovery will be used in addition to, or instead of, the intra-RAID parity-based recovery, at Blocks 156 - 157 .
  • two drives in the first RAID set D 1 are bad or have stale data, namely the drives d 4 and d 9 .
  • the memory controller(s) 31 uses the mirrored data from the corresponding drives d 4 and d 9 in the second RAID set D 2 to recover the data to the drives d 4 and d 9 of the first RAID set D 1 , which again may advantageously be done on the fly without disruption to air time.
  • FIG. 11 A more extended disk failure scenario is shown in FIG. 11 , where multiple drives in the first RAID set D 1 have failed (i.e., the drives d 4 , d 9 , d 13 , and P), and the corresponding disk d 9 in the second RAID set also fails.
  • the data in the drive d 9 of the second RAID set D 2 is recovered using the inter-RAID parity data from its drive P, and the faulty drives in the first RAID set D 2 (i.e., the drives d 4 , d 9 , d 13 , and P) may then be recovered using the data from the corresponding drives in the second RAID set.
  • This can still advantageously be performed by the memory controller(s) 31 on the fly and without disruption to air.
  • FIG. 12 Still another fault scenario is illustrated in FIG. 12 , where the entire RAID set D 1 goes down for some period of time.
  • this could happen when an FC cable is unplugged from a memory controller/server, drive chassis, or switch.
  • This could also happen where an FC switch is accidentally unplugged or fails, or where a drive chassis loses power or fails, as will be appreciated by those skilled in the art.
  • the memory controller(s) 31 will lose access to the first RAID set D 1 , but it will still advantageously be able to read and write from the second RAID set D 2 with little or no effect to the output video stream.
  • sectors written by the affected servers e.g., the server 31 a ′ in FIG.
  • the memory controller(s) performs data recovery from the mirrored data in the RAID set D 2 to the corresponding drives in the RAID set D 1 .
  • mirrored RAID 3 (or higher) set as discussed above, loss of all the partial elements (i.e., individual drives) in one of the RAID sets as well as the loss of an additional single drive in the other RAID set can be recovered.
  • Mirrored ECC RAID sets can advantageously recover from loss of an entire RAID set as well as the loss of two additional elements on the other RAID sets in some situations. If an entire RAID set is not available, as in the prior example, when it is returned to service it is likely that it will contain stale data elements. By comparing timestamps during reading, stale data elements can likewise be ignored. This may be done by ORing together the corresponding data values and comparing the timestamps, for example.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Library & Information Science (AREA)
  • Techniques For Improving Reliability Of Storages (AREA)

Abstract

A video media data storage system may include first and second pluralities of data storage devices each arranged in a redundant array of independent drives (RAID) configuration for permitting writing and reading of video media data. The system may further include at least one memory controller coupled to the first and second pluralities of data storage devices for performing mirrored writing of video media data to both the first and second pluralities of data storage devices. The at least one memory controller may also be for generating respective first and second file allocation tables (FATs) including video media data time stamps and validity information for both of the first and second pluralities of data storage devices, and selectively reading video media data from the first and second pluralities of data storage devices based upon the first and second FATs.

Description

FIELD OF THE INVENTION
The present invention relates to the field of data storage, and, more particularly, to video data storage networks and related methods.
BACKGROUND OF THE INVENTION
In the computer industry the term “crash” is defined as the sudden failure of a software application or operating system, or of a hardware device such as a hard disk. While a software crash usually creates an interruption in service, the dreaded hard disk crash also tends to cause the loss of data. Due to the mechanical nature of a fast-spinning disk drive (e.g., 10,000 RPM), crashes are usually permanent and the need for data protection becomes critical.
Along with data errors that occur due to outright drive failure, expressed as a mean time between failure (MTBF), drives also experience bit errors over the amount of data read, expressed as a bit error rate (BER). Other errors that can result in seek, read and write failures are usually masked by successful retries. Since many drives now come with MTBF's upward of one million hours, BER's on the order of 1 in 1×1015 and warranties up to 5 years, it makes economic sense for the vendors to be able to distinguish between a drive that has failed from one that is merely “having occasional trouble.” To reduce warranty replacement costs, drives hedge against excessive “failures” by employing many internal data recovery mechanisms, including error correction, retries, and sector remapping. Only when a drive exceeds its retry count and runs out of sectors for remapping is it considered “failed.”
Because of the need for deterministic data performance, video server designs usually cannot afford to allow drives the luxury to attempt all of the internal data correction mechanisms designed to conceal errors. Retries need to be limited and managed at a systemic level, and problematic drives generally are not acceptable.
Drive specifications also typically specify an annualized failure rate (AFR), which is equal to the operational duty cycle multiplied by the number of hours in a year and divided by the MTBF. Drives are usually divided into 3 classes, namely desktop, notebook and enterprise. Although more costly, only enterprise or server class drives are specified around a 100% duty cycle. So, a server drive with an MTBF of 1,000,000 hours would have an AFR of 0.876%, while a drive with an MTBF of 1,500,000 hours would have an AFR of 0.584%.
As systems increase in performance, size, and complexity, drive failure and error handling become a critical issue to video server design. Assuming a drive has a BER of 1 per 1×1015 (errors per bits read), as data rates increase, the error frequency approaches 1 every few hours. Since even a single uncorrected bit error in a critical data location may result in an unacceptable video anomaly, it is typically necessary to implement some form of data protection.
One disk drive configuration that is used to help guard against data loss is the redundant array of independent drives (RAID) configuration. Generally speaking, in a RAID data is divided and/or replicated among multiple hard disk drives, which can lead to increased data reliability as well as increased input/output (I/O) performance.
Various levels of RAID configurations have been developed, and different RAID levels take advantage of different storage/data protection techniques. For example, some RAIDs employ “striping,” meaning that the data stream is divided into segments or blocks that are stored on separate drives in the RAID. Parity data may be used with data striping, which allows data from a faulty drive or sector(s) to be reconstructed from data on the other data drives in the RAID and the parity data. Another RAID technique is mirroring data on multiple drives within a same RAID set. That is, a copy of a data set is stored/maintained on a separate drive so that if one of the drives goes down, the duplicate data set is immediately available from the mirror drive.
Various prior art RAID implementations have been used to provide increased read performance and data redundancy. By way of example, U.S. Pat. No. 7,225,315 is directed to a file system including a storage system having a plurality of volumes, a volume allocation table adapted to set the plurality of volumes for each directory, a file allocation table that stores attributes and divided block information of the file, a block reading table in which numbers of blocks read out in one reading operation for each volume are respectively set, and a read control module that controls data read from the volume. A read control module, when a read command is received, determines a volume to be read from the volume allocation table. The read control module further determines the number of blocks read for each volume by referring to the block reading table, determines the blocks read for each volume based on the volume, the number of blocks, and the block information, and reads from each volume in parallel.
Despite the advantages that such configurations provide in certain applications, further data reading and recovery features may be desirable in high-bandwidth, high-reliability data storage applications, such as broadcast video applications, for example.
SUMMARY OF THE INVENTION
In view of the foregoing background, it is therefore an object of the present invention to provide a system and related methods for enhanced video media data storage and recovery.
This and other objects, features, and advantages are provided by a video media data storage system which may include first and second pluralities of data storage devices each arranged in a redundant array of independent drives (RAID) configuration for permitting writing and reading of video media data. The system may further include at least one memory controller coupled to the first and second pluralities of data storage devices for performing mirrored writing of video media data to both the first and second pluralities of data storage devices. The at least one memory controller may also be for generating respective first and second file allocation tables (FATs) including video media data time stamps and validity information for both of the first and second pluralities of data storage devices, and selectively reading video media data from the first and second pluralities of data storage devices based upon the first and second FATs. As such, more flexibility in video media data reading is provided to therefore enhance performance.
In addition, the at least one memory controller may also be for performing data recovery between the first and second plurality of data storage devices from the mirrored video media data based upon the first and second FATs. Moreover, the at least one memory controller may write the video media data in each of the first and second pluralities of data storage devices as striped video media data, and generate parity data from the striped video media. The at least one memory controller may therefore also perform data recovery within a given one of the first and second pluralities of data storage devices based upon the parity data. Additionally, the at least one memory controller may also advantageously select between using the mirrored video media data and the parity data for performing data recovery based upon a number of data storage devices having faults within the given one of the first and second pluralities of data storage devices.
The validity information may include data storage device fault information, for example. As such, the at least one memory controller may give reading preference to data storage devices without a fault and with a most recent video media data time stamp. The at least one memory controller may also perform load balancing for reading the video media data from the first and second pluralities of data storage devices.
By way of example, the first and second pluralities of data storage devices may each be arranged in a RAID 3 or higher configuration. Furthermore, the at least one memory controller may include a first memory controller for the first plurality of data storage devices, and a second memory controller for the second plurality of data storage devices. Additionally, the system may further include first and second fibre channel (FC) switches respectively coupling the first and second memory controllers to the first and second pluralities of data storage devices. Also, the first memory controller may be coupled to the second FC switch, and the second memory controller may be coupled to the first FC switch.
The first and second pluralities of data storage devices and the at least one memory controller may be connected together in a storage area network (SAN), for example. Further, the at least one memory controller may be at least one broadcast video server.
A video media data storage method aspect may include providing first and second pluralities of data storage devices each arranged in a redundant array of independent drives (RAID) configuration for permitting writing and reading of video media data, and performing mirrored writing of video media data to both the first and second pluralities of data storage devices. The method may further include generating respective first and second file allocation tables (FATs) including video media data time stamps and validity information for both of the first and second pluralities of data storage devices. The method may also include selectively reading video media data from the first and second pluralities of data storage devices based upon the first and second FATs.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a schematic block diagram of a video media data storage system in accordance with the invention.
FIG. 2 is a schematic block diagram of an alternative embodiment of the video media data storage system of FIG. 1.
FIG. 3 is a RAID map of the mirrored RAID sets of FIG. 1 illustrating mirrored writing thereto.
FIG. 4 is a flow diagram illustrating video media data writing operations of the system of FIG. 1.
FIG. 5 is a schematic diagram of the first and second pluralities of data storage devices from the system of FIG. 1 with accompanying file allocation tables (FATS) and data stripes.
FIG. 6 is a flow diagram illustrating video media data reading operations of the system of FIG. 1.
FIG. 7 is a RAID map of the first and second pluralities of data storage devices of the system of FIG. 1 and their associated FAT tables.
FIGS. 8A and 8B are respective FAT tables for the first and second pluralities of data storage devices of the system of FIG. 1, and FIG. 8C is a corresponding operation table showing read operations by the memory controller(s) of the system of FIG. 1 based upon the information in the FAT tables of FIGS. 8A and 8B.
FIGS. 9-13 are RAID drive maps illustrating various video media data recovery scenarios of the system of FIG. 1.
FIG. 14 is a flow diagram generally illustrating reading and writing operations in accordance with the invention.
FIG. 15 is flow diagram illustrating data recovery operations in accordance with another advantageous aspect of the invention.
FIG. 16 is a graph comparing RAID 3 vs. ECC overhead for parity configurations that may be implemented in the system of FIG. 1.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
The present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which preferred embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Like numbers refer to like elements throughout, and prime notation is used to indicate similar elements in alternate embodiments.
Referring initially to FIG. 1, a video media data storage system 30 and associated video data storage method are first described. The system 30 illustratively includes first and second pluralities or sets of data storage devices D1 and D2 each arranged in a redundant array of independent drives (RAID) configuration for permitting writing and reading of video media data. As used herein, “video media data” includes video, graphics, automation and/or animation, and associated audio data, for example. Furthermore, one or more memory controllers 31 are illustratively coupled to the first and second RAID sets D1, D2. In a typical implementation, the data storage devices may be magnetic or optical disks, although other suitable data storage devices (e.g., FLASH drives, etc.), may also be used.
The system 30 may be particularly well suited for applications where a large bandwidth is required, such as in broadcast or streaming video data applications. In the illustrated example, the video data feed is generated by video cameras 32, and video media data from the system 30 may be communicated to recipients by one or more communications mediums, such as a satellite link (illustrated by a satellite station 33 in FIG. 1), co-axial/fiber connections, wide area networks (WANs) such as the Internet, etc., as will be appreciated by those skilled in the art.
Turning now additionally to FIGS. 3 and 14, beginning at Block 140, writing and reading operations of the system 30 will first be generally described, and further detail will be provided below. The memory controller(s) 31 performs mirrored writing of video media data to both the first and second RAID sets D1, D2 (Blocks 141-142). In one exemplary implementation, the data in a given RAID set D1, D2 is written in data stripes (i.e., divided in blocks or sections each written to a different storage device), and the striped data is mirrored in both of the RAID sets D1, D2, as will be discussed further below. This mirrored data writing is illustrated in FIG. 3.
The memory controller(s) 31 also generates respective first and second file allocation tables (FATs) including video media data time stamps and validity information (e.g., drive/sector fault information, etc.) for both of the first and second RAID sets D1, D2, at Block 143. Since both of the FATs are available to the memory controller(s) 31, it can advantageously selectively read video media data from the first and second RAID sets D1, D2 based upon the first and second FATs, at Block 144, thus concluding the method illustrated in FIG. 14 (Block 145). That is, the memory controller(s) 31 has more flexibility in performing read operations, which may be particularly important where there are multiple drive faults. For example, the memory controller(s) 31 may give reading preference to data storage devices without a fault and with a most recent video media data time stamp. Moreover, this also advantageously allows load balancing to be used to reduce read times and increase data throughput, for example.
In addition, this configuration also allows the memory controller(s) 31 to perform data recovery between the first and second RAID sets D1, D2 from the mirrored video media data based upon the first and second FATs. Stated alternatively, this allows inter-RAID data recovery (i.e., from RAID set D1 to D2, or vice-versa). More particularly, the memory controller(s) 31 may write the video media data in each of the first and second RAID sets D1, D2 as striped video media data, and generate parity data from the striped video media data to perform data recovery within a given RAID set based thereon (i.e., intra-RAID data recovery), as will be discussed further below. The memory controller(s) 31 will, generally speaking, select between using the mirrored video media data (inter-RAID recovery) and the parity data (intra-RAID recovery) for performing data recovery based upon a number of data storage devices having faults within the first and second RAID sets D1, D2, as will also be discussed further below.
Turning now additionally to FIG. 2, in an alternative embodiment the system 30′ is implemented as a storage area network (SAN) and includes respective broadcast video servers as the memory controllers 31 a′, 31 b′. By way of example, one suitable broadcast video server that may be used is the Nexio Advanced Media Platform (AMP) from Harris Corporation, the present Assignee. However, other suitable memory controllers/servers may also be used in different embodiments, as will be appreciated by those skilled in the art.
In the SAN system 30′, the first and second RAID sets D1′, D2′ are arranged in a RAID level 3 (or higher) configuration and implement data striping and parity (e.g., error correction code (ECC) parity), as will be appreciated by those skilled in the art. The parity data is stored in one or more dedicated parity data storage devices 40 a′, 40 b′, Furthermore, first and second fibre channel (FC) switches 36 a′, 36 b′ are respectively coupled to the first and second memory controllers 31 a′, 31 b′ and the first and second RAID sets D1′, D2′. That is, the first and second RAID sets D1′, D2′ are on separate FC domains to advantageously provide high redundancy and availability, as will be appreciated by those skilled in the art.
Moreover, the first memory controller 31 a′ is illustratively coupled to the second FC switch 36 b′, and the second memory controller 31 b′ is illustratively coupled to the first FC switch 36 a′. This advantageously gives the first memory controller 31 a′ access to the second RAID set D1′, and the second memory controller 31 b′ access to the second RAID set D2′. In some embodiments, additional FC switches may also be used to provide additional access points to the RAID sets D1′, D2′, for example. However, it should be noted that a SAN and/or FC network configuration is not required in all embodiments.
As noted above, since even a single uncorrected bit error in a critical data location may result in an unacceptable video anomaly, it is generally desired to implement some form of data protection. In a typical RAID 3 implementation, the parity information is simply used to reconstruct errant data after the problem is discovered. However, in the system 30 parity information may be continuously read and decoded, improving the capability of error detection as well as data performance during error correction.
More particularly, intra-RAID set parity may be implemented in the system 30 or 30′ as follows. Data is organized into buffers or stripes and is stored on respective drives 1-P, and each partial data block size is defined as:
Data Buffer: D
Partial: d
Parity: P
Buffer size=Sd
Partial size: Sd
Number of drives=n
Partial size (sd)=Sd/n, where Sd is divisible n, and
D=Σi=n i=0di
Representing the XOR contribution of each partial as d, the parity equation is defined as:
P=xorΣi=n i=0di
A single failing data block derr can be reconstructed via the inverse operation:
derr=Pxor(xorΣ i<err i=0 di+xorΣ i=n i<err di)
Unlike many RAID 3 implementations, in read mode the memory controllers 31 a′, 31 b′ advantageously read the parity data and use it to verify data integrity, rather than simply using this information for after-the-fact data recovery. Since the parity data is typically based on XOR contributions from each partial, error handling can take the form of either detection or correction. In this application, the parity equation serves three purposes depending on the error cases. A first error case is when a partial data block is not returned, or is reported in error. The defective partial is still reconstructed by the XOR resultant of the good partials and the parity. This case is known as single-error correcting (SEC).
A second case is when all partial data and parity is returned and no errors are reported by the drive. If all the data and parity is in fact good, the equation simply rebuilds the buffer. If any partial is detective, the error is detected and the damaged buffer is discarded. For the sake of video (and audio) information, data may be represented as black and silence can be substituted. This case is known as single-error detecting (SED). Still another case is where the partial data set has multiple defective elements. In this eventuality the data cannot be corrected, and again for the sake of video (and audio) information, data may be represented as black and silence can be substituted. This case is known as multiple-error detecting (MED).
In the case of a RAID disk storage sub-system, a failing partial usually represents a specific failing drive (resolving the issue of knowing which partial is in error), which reduces the model to the first and third cases noted above. By relying on the drive to know when it is having a problem and not always running data through XOR verification, most RAID systems typically ignore the second case, leaving the system vulnerable or erroneous unreported data. In the application of a video server, the present approach implemented by the memory controllers 31 a′, 31 b′ of using parity data for error detection may potentially detect this erroneous data and replace it with black to maintain a continuous uninterrupted video stream. This same approach may also be used for audio to reduce “pops” and “clicks”.
Another advantage of always keeping a running, on the fly, parity count is an overall performance improvement in the case of single corrected errors. If a drive fails, there is no need to go back and re-read the previous partials. Rather, the memory controllers 31 a′, 31 b′ may skip over the failed device and reconstruct its data contribution using the accrued parity information. This advantageously results in no net decrease in system performance during a “degraded” mode of operation.
Since a single parity drive RAID can only protect a data stripe from a single data element failure, the odds of an uncorrectable failure increase with the amount of partials involved in the stripe. As more data elements (drives in this case) are added, the greater the likelihood of an uncorrectable double data failure. The primary tradeoff of using a single parity RAID is that as sets grow larger they become much more susceptible to uncorrectable double errors. The system 30′ therefore advantageously utilizes multiple, mirrored RAID sets D1′, D2′ to provide further error recovery flexibility (in addition to the additional data reading flexibility noted above), as will be discussed further below.
An improved approach toward overcoming the deficiencies of single parity block per data stripe can be realized using a multi-parity block approach. The first part of this approach is defining a set of equations to represent parity information of a given data set. Hamming codes are used to define these equations. By using Hamming distance-3 codes to define a set of parity equations, desired data protection of the following nature may be achieved:
Number of parity equations (parity drives): r
Number of partial data blocks (data drives): n=2r−1−r
Examining the example of a typical (15,11,3) code SEC set of Hamming equations, the standard parity matrix for the given distance-3 code is defined as follows:
Position
p1 p2 d3 p4 d5 d6 d7 p8 d9 d10 d11 d12 d13 d14 d15
H= 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1
0 0 0 1 1 1 1 0 0 0 0 1 1 1 1
0 1 1 0 0 1 1 0 0 1 1 0 0 1 1
1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

Columns having only a single entry determine parity positions. This approach assures the independence of the parity equations, yielding the following set:
p1=d3+d5+d7+d9+d11+d13+d15
p2=d3+d6+d7+d10+d11+d14+d15
p4=d5+d6+d7+d12+d13+d14+d15
p8=d9+d10+d11+d12+d13+d14+d15
The decoding of a received data stripe in the vector form r=(r1, r2, . . . r15) uses the following syndrome equations derived from the corresponding data equations above:
s1=r1+r3+r5+r7+r9+r11+r13+r15
s2=r2+r3+r6+r7+r10+r11+r14+r15
s4=r4+r5+r6+r7+r12+r13+r14+r15
s8=r5+r9+r10+r11+r12+r13+r14+r15
The syndrome vector obtained from the preceding equation set, written as s=(s4+s3+s2+s1), identifies the partial data position of a single error. Once the error is located, the failing data can be reproduced by solving the original parity equations. This is the standard application of a distance-3 Hamming SEC decoder.
In a RAID application it is possible (and likely) that the location of a failing partial (or partials) will be known due to the previously mentioned internal drive error reporting mechanisms. Knowing the location of the failing data increases the performance of the distance-3 Hamming code in different ways depending upon the error scenario. For example, if a single partial data block is not returned or reported in error, the data can be reconstructed as described above (without the need of solving the syndrome equations). This case is single-error correcting (SEC). If two partial data blocks are not returned or reported in error, the data can be also be reconstructed. Based upon the above matrix above, for any two known bad data partials there exists at least one equation that only contains one of the failing vectors. Solving the corresponding data equation reconstructs that partial, and reduces the syndrome to the preceding case. This case is double-error correcting (DEC).
A third case involves a combination of multiple unknown partials. In this event, the data cannot be corrected. For the case of video (and audio) information, data representing black and/or silence can be inserted. This case is multiple-error detecting (MED). Notwithstanding this last case, advantages of the above-described approach are that double data errors can be corrected, and single, unknown location data errors can also be corrected.
Comparing the relative cost of single parity and ECC in terms of additional overhead can be useful in determining which strategy is best suited for the given application. This can be done by comparing the parity overhead of a drive population containing a multiple RAID 3 set to the same population arranged in a single ECC set. The graph of FIG. 16 examines both approaches for an example in which the maximum RAID 3 stripe length is set to 9 drives (8 data+1 parity). As can be seen, ECC systems begin to become advantageous at around 25 drives (data and parity) with an overhead less than twice that of the multiple RAID 3 set. As systems begin to reach 50 drives, the overhead of ECC actually becomes less than that of RAID 3, while continuing a higher level of protection. It should be noted that either of these approaches (or other suitable approaches) may be used at the RAID set level in the system of 30′.
Nonetheless, while localized RAID 3 and ECC protect against drive related media and data errors within a given RAID, data mirroring between different RAIDS affords protection against not only multiple drive errors, but also drive enclosure failures. Although this added layer of redundancy increases the storage system costs (i.e., because it uses an additional RAID set), it significantly increases data survivability and on-air availability. With respect to writing operations, mirroring can be done between two identical RAID 3 or ECC drive sets. Writing is accomplished by sending the data partials and parities to two identical sets of drives D1′, D2′, which results in the creation of a fully redundant data set, as seen in FIG. 3. It should be noted that by “identical” it is meant that the RAID sets D1′, D2′ have at least some corresponding drives that are the same, although these RAID sets could have other or extra drives which are not part of the mirror configuration in some embodiments.
Turning now to FIGS. 4 and 5, an exemplary write flow process by the memory controller(s) 31 is described. Beginning at Block 40, the memory controller(s) 31, initially attempts to write the video media data stripe to one of the RAID data sets (D1 in the present example), at Block 41. If writing to the RAID set D1 cannot be completed successfully, such as due to a drive fault, etc., at Block 42, then the memory controller(s) 31 attempts to write the video media data stripe to the second RAID set D2, at Block 43. If this is successful, then the memory controller(s) 31 updates the FAT tables to indicate the RAID data set D1 storage device(s) that is bad, and that the RAID data set D2 is ON, at Block 45, and to write a time stamp TS indicating the time of writing. Otherwise, both of the RAID sets D1 and D2 have one or more bad storage devices/sectors, and the first and second FATS are updated accordingly, at Block 46.
If writing to the RAID data set D1 is initially successful (Block 42), then the mirrored writing to the RAID data set D2 is attempted, at Block 47, and the memory controller(s) 31 determines whether this writing operation is successful, at Block 45. The first and second FAT tables are updated accordingly for the case where the write operation to the RAID set D2 was successful (Block 49), and where it was not (Block 50), as described above. At this point, the memory controller(s) 31 writes the first and second FAT tables to their respective first and second RAID sets D1, D2, at Blocks 50-51, thus concluding the illustrated writing operations (Block 52).
Exemplary FATs for a mirrored RAID set configuration are shown in FIG. 5. Since a likely failure scenario involves not only write failures but also the possibility of loss of an entire RAID set, it is desirable to protect against these failure modes. The FATs for each RAID set D1, D2 advantageously include sector indexing information, timestamps, and data OK flags for both of the RAID sets, as seen in FIG. 5. If a write fails on either side of the mirror (i.e., in either RAID set), the corresponding data OK flags in both FATS are cleared. This way subsequent reads will know to ignore these data elements, as discussed further below.
One particularly advantageous feature of the writing/reading/recovery operations described herein is that they may be implemented using a software application running at the memory controller/sever. Once suitable software application that may be used to implement the approach described herein is the RAIDSoft application from Harris Corporation. The parity operations/calculations are performed by the memory controller(s) 31, and the data and localized parity are then written in a mirrored fashion to both of the RAID sets D1, D2, as discussed above. All of the information is mirrored at the block level, with the exception of the FAT tables. The FAT tables differ between the two RAID sets D1, D2 to track bad sectors/drives and offline conditions. The FAT information for both RAID sets D1, D2 may also be stored in a memory such as a random access memory (RAM) in some embodiments as well to provide faster operation by the memory controller(s) 31.
Turning now additionally to FIGS. 6 and 7, the reading process will now be described. Generally speaking, data reads are performed once and load balanced between the first and second RAID sets D1, D2 if both sets contain good (i.e., without fault and/or up-to-date) data. Unrecorded or faulty sectors/drives are noted in the FATS and not used in read operations. If the load balanced read returns an error, then the other copy of the data is immediately read with little or no disturbance to air playout in broadcast scenarios, for example. In the case of a RAID set being down, the down set will not be accessed again until the problem is cleared (e.g., drive replaced, etc.). Probe commands may be periodically sent to determine when the down set returns to operations, for example, as will be appreciated by those skilled in the art.
More particularly, beginning at Block 60, when performing a read operation the memory controller(s) 31 first checks the first and second FAT tables to see if the data is good for a given block of data in both of the first and second RAID sets D1, D2 (Blocks 61, 62), as illustrated in FIG. 7. The results of the FAT tables are compared, such as through a logical OR operation (Block 63), to determine the four possible fault conditions for the drives in questions, namely: (a) the RAID set D1 is OK but RAID set D2 is BAD (Block 64); (b) both RAID sets are OK (Block 65); (c) RAID set D1 is BAD but RAID set D2 is OK (Block 66); and (d) both RAID sets are bad for the given block of data (Block 67).
In the first case (a), the memory controller(s) 31 attempts to read the first RAID set D1, at Block 68, and if the read is completed successfully (Block 69) then the data is output accordingly, at Block 70. If the read operation is not successful for whatever reason, then black and/or silence may be output, at Block 71. In the second case (b), since both RAID sets are OK (i.e., without fault), then the memory controller(s) 31 checks to see which RAID set D1, D2 has the most up-to-date data (i.e., compares the time stamps), at Block 72. If the time stamp for the RAID set D1 time stamp is more recent, then the memory controller 31 reads the first RAID set as described above at Blocks 68-71.
If the time stamps are the same, then the memory controller(s) 31 may read from either RAID set D1, D2, and the choice between the two may be based upon a load balancing algorithm (i.e., checks which one is busiest at the time and uses the other), and/or a preferential (i.e., default) scheme may be used. For example, in the illustrated embodiment an affinity is given to odd or even RAID sets, such that odd-numbered memory controllers (e.g., controller #1) would first access an odd-numbered RAID set (i.e., RAID set D1), and vice-versa. In the present example, the affinity is for the first RAID set D1, which the memory controller(s) 31 attempts to read at Block 73. If the read is successful, at Block 74, then the data is output (Block 75). If not, the memory controller(s) 31 still has the option of reverting back to the second RAID set D2. If the attempted read from the second RAID set D2 is successful, at Blocks 76-77, then the data is output, at Block 78. Otherwise, black and/or silence data is output, at Block 79.
For the third case (c), the memory controller(s) 31 attempts to read from the second RAID set D2 in the same manner described above with reference to Blocks 76-79. For the final case (d) where both RAID sets D1, D2 have faults, etc., then black and/or silence data is output, at Block 79. In the exemplary embodiment shown in FIG. 7, all of the drives are without fault and have equal time stamps (meaning both sets have the freshest or most up-to-date data). It should be noted that in some embodiments reading may also be performed from a faulty drive using the parity data described above if the mirrored data is bad or “stale”, as will be appreciated by those skilled in the art, although this is not illustrated in FIG. 6 for clarity of explanation. Typically this option is available if a single drive within a RAID set is faulty but the rest are not. That is, as the number of faulty drives increases past one, the less likely it will be that intra-RAID set parity can be used to read and/or recover this data, as will be appreciated by those skilled in the art. This is why the mirrored writing/reading/recovery operations are particularly advantageous where high throughput and availability are required of the system 30.
Referring now additionally to FIGS. 8A-8C, the above-described reading operations will be further understood with reference to an exemplary read scenario. For the first block of data (stored on drive d1 in both RAID sets D1, D2), it can be seen in the operation table of FIG. 8 that both FATs reflect that drive d1 is OK in both RAID sets, and that their time stamps are equal. Thus, both data stripes have “good” data and either can be successfully read based upon affinity, load balancing, etc. In the case of the second data block, both FATS indicate that the drive d2 is OK in the first RAID set D1, but the second FAT indicates that the drive d2 in the second RAID set D2 is bad, and that the time stamp for drive d1 in RAID set D1 is newer than for the drive d2 of the second RAID set D2. As such, only the first data stripe (i.e., RAID set D1) is considered valid for reading.
For the fourth data block, the drive d4 is listed as bad in the second FAT table, and the time stamp for the drive d4 in the second FAT is also newer. As such, the first data stripe is again considered bad, and the second data strip (i.e., RAID set D2) is read. For the seventh data block, both FAT tables indicate that the RAID sets D1 and D2 are OK, but the time stamp for the drive d7 of the second RAID set is newer, indicating that the data on the drive d7 of the first RAID set is stale. Accordingly, the memory controller(s) 31 considers the first data stripe to be old or stale, and reads from the drive d7 of the second RAID set. In the last illustrated case for the ninth data block, both FAT tables indicate that the RAID sets D1 and D2 are OK, but the time stamp for the drive d7 of the first RAID set is newer, indicating that the data on the drive d7 of the second RAID set is stale. Accordingly, the memory controller(s) 31 considers the second data stripe to be old or stale, and reads from the drive d7 of the first RAID set.
Referring now to FIGS. 9-13 and FIG. 15, the data recovery steps performed by the memory controller(s) 31 are now further described. Generally speaking, in cases where both RAID sets are available, all unused, bad, or out of sync sectors on all disks will be written with recovered data. The recovered data is preferably always the “freshest” copy, taken from whichever RAID set D1, D2 is appropriate. If the freshest data had a read fault from an individual disk, then the inter-RAID set mirror data is used to recover the data.
More particularly, beginning at Block 150, the parity data is generated for the first and second RAID sets D1, D2 as discussed above (Block 150). In particular, the parity data may be generated for one of the RAID sets D1, D2, and then copied over as part of the mirrored data set, although in some embodiments parity data could be generated independently for each of the RAID sets, if desired. The memory controller(s) 31 determines which of the storage devices in the first and second RAID sets require recovery or data freshening, at Block 152, based upon the OK flags and the time stamps, for example.
The way in which the memory controller(s) will perform data recovery will depend upon the number of storage devices/drives that are affected. More particularly, if only a single drive in one of the RAID sets D1 or D2 is affected, the data is recovered on the fly from the intra-RAID set parity data, at Block 154, thus concluding the illustrated example (Block 155). In the example shown in FIG. 9, the drive d4 in the RAID set D1 is bad (or has stale date), as indicated by the dashed markings. Here, the data may be recovered using the parity data stored in the drive P in the first RAID set, and thus it is not necessary to go to the mirrored data in the second RAID set D2 for recovery (although this could be done, if desired).
In the case of a bad drive, the user would replace the bad drive and the recovery operations may advantageously be performed automatically by the memory controller(s) 31 once it detects that the drive is no longer faulty, for example. It should be noted that even if the corresponding partials from both RAID sets D1, D2 are damaged, parity data from either set can be used to reconstruct the data buffer.
However, if more than a single drive is affected, depending upon the particular scenario, inter-RAID recovery will be used in addition to, or instead of, the intra-RAID parity-based recovery, at Blocks 156-157. In the example illustrated in FIG. 10, two drives in the first RAID set D1 are bad or have stale data, namely the drives d4 and d9. Here, the memory controller(s) 31 uses the mirrored data from the corresponding drives d4 and d9 in the second RAID set D2 to recover the data to the drives d4 and d9 of the first RAID set D1, which again may advantageously be done on the fly without disruption to air time.
A more extended disk failure scenario is shown in FIG. 11, where multiple drives in the first RAID set D1 have failed (i.e., the drives d4, d9, d13, and P), and the corresponding disk d9 in the second RAID set also fails. Here, the data in the drive d9 of the second RAID set D2 is recovered using the inter-RAID parity data from its drive P, and the faulty drives in the first RAID set D2 (i.e., the drives d4, d9, d13, and P) may then be recovered using the data from the corresponding drives in the second RAID set. This can still advantageously be performed by the memory controller(s) 31 on the fly and without disruption to air.
Still another fault scenario is illustrated in FIG. 12, where the entire RAID set D1 goes down for some period of time. By way of example, this could happen when an FC cable is unplugged from a memory controller/server, drive chassis, or switch. This could also happen where an FC switch is accidentally unplugged or fails, or where a drive chassis loses power or fails, as will be appreciated by those skilled in the art. In such case, the memory controller(s) 31 will lose access to the first RAID set D1, but it will still advantageously be able to read and write from the second RAID set D2 with little or no effect to the output video stream. In this situation, sectors written by the affected servers (e.g., the server 31 a′ in FIG. 2) are remembered as bad in the FAT of the opposite RAID set, and therefore will not be used in subsequent read operations by any server. Once power/connection to the affected RAID set is restored, then the memory controller(s) performs data recovery from the mirrored data in the RAID set D2 to the corresponding drives in the RAID set D1.
Thus, it will be appreciated that with a mirrored RAID 3 (or higher) set as discussed above, loss of all the partial elements (i.e., individual drives) in one of the RAID sets as well as the loss of an additional single drive in the other RAID set can be recovered. Mirrored ECC RAID sets can advantageously recover from loss of an entire RAID set as well as the loss of two additional elements on the other RAID sets in some situations. If an entire RAID set is not available, as in the prior example, when it is returned to service it is likely that it will contain stale data elements. By comparing timestamps during reading, stale data elements can likewise be ignored. This may be done by ORing together the corresponding data values and comparing the timestamps, for example.
Many modifications and other embodiments of the invention will come to the mind of one skilled in the art having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. Therefore, it is understood that the invention is not to be limited to the specific embodiments disclosed, and that modifications and embodiments are intended to be included within the scope of the appended claims.

Claims (23)

1. A video media data storage system comprising:
first and second pluralities of data storage devices each arranged in a redundant array of independent drives (RAID) configuration and configured to write and read video media data corresponding to a video stream; and
at least one memory controller coupled to said first and second pluralities of data storage devices and configured to
perform mirrored writing of video media data to both said first and second pluralities of data storage devices,
generate respective first and second file allocation tables (FATS) including video media data time stamps and validity information for both of said first and second pluralities of data storage devices, and
selectively read video media data from both of said first and second pluralities of data storage devices during a playback of the video stream to avoid faulty or outdated video media data on said first and second pluralities of data storage devices based upon the first and second FATs.
2. The video media data storage system of claim 1 wherein said at least one memory controller is also configured to perform data recovery between said first and second pluralities of data storage devices from the mirrored video media data based upon the first and second FATs.
3. The video media data storage system of claim 2 wherein said at least one memory controller is configured to write the video media data in each of said first and second pluralities of data storage devices as striped video media data; wherein said at least one memory controller also is configured to generate parity data from the striped video media; and wherein said at least one memory controller also is also configured to perform data recovery within a given one of the first and second pluralities of data storage devices based upon the parity data.
4. The video media data storage system of claim 3 wherein said at least one memory controller is configured to select between using the mirrored video media data and the parity data for performing data recovery based upon a number of data storage devices having faults within the given one of said first and second pluralities of data storage devices.
5. The video media data storage system of claim 1 wherein said at least one memory controller is configured to perform load balancing to read the video media data from said first and second pluralities of data storage devices.
6. The video media data storage system of claim 1 wherein said first and second pluralities of data storage devices are each arranged in a RAID 3 or higher configuration.
7. The video media data storage system of claim 1 wherein said at least one memory controller comprises a first memory controller for the first plurality of data storage devices, and a second memory controller for the second plurality of data storage devices.
8. The video media data storage system of claim 7 further comprising first and second fibre channel (FC) switches respectively coupling said first and second memory controllers to said first and second pluralities of data storage devices; wherein the first memory controller is coupled to said second FC switch; and wherein the second memory controller is coupled to said first FC switch.
9. The video media data storage system of claim 1 wherein said first and second pluralities of data storage devices and said at least one memory controller are connected together in a storage area network (SAN).
10. The video media data storage system of claim 1 wherein said at least one memory controller comprises at least one broadcast video server.
11. A video media data storage system comprising:
first and second pluralities of data storage devices each arranged in a redundant array of independent drives (RAID) configuration and configured to write and read video media data corresponding to a video stream; and
at least one memory controller coupled to said first and second pluralities of data storage devices and configured to
perform mirrored writing of video media data to both said first and second pluralities of data storage devices,
generate respective first and second file allocation tables (FATs) including video media data time stamps and fault information for both of said first and second pluralities of data storage devices,
selectively read video media data from both of said first and second pluralities of data storage devices during a playback of the video stream to avoid faulty or outdated video media data on said first and second plurality of data storage devices based upon the first and second FATs, and
performing data recovery between said first and second pluralities of data storage devices from the mirrored video media data based upon the first and second FATs.
12. The video media data storage system of claim 11 wherein said at least one memory controller is configured to write the video media data in each of said first and second pluralities of data storage devices as striped video media data; wherein said at least one memory controller also is configured to generate parity data from the striped video media; and wherein said at least one memory controller also is configured to perform data recovery within a given one of the first and second pluralities of data storage devices based upon the parity data.
13. The video media data storage system of claim 12 wherein said at least one memory controller is configured to select between using the mirrored video media data and the parity data for performing data recovery based upon a number of data storage devices having faults within the given one of said first and second pluralities of data storage devices.
14. The video media data storage system of claim 12 wherein said at least one memory controller is configured to perform load balancing to read the video media data from said first and second pluralities of data storage devices.
15. The video media data storage system of claim 12 wherein said first and second pluralities of data storage devices are each arranged in a RAID 3 or higher configuration.
16. The video media data storage system of claim 12 wherein said at least one memory controller comprises a first memory controller for said first plurality of data storage devices, and a second memory controller for said second plurality of data storage devices.
17. The video media data storage system of claim 12 wherein said at least one memory controller comprises at least one broadcast video server.
18. A video media data storage method comprising:
providing first and second pluralities of data storage devices each arranged in a redundant array of independent drives (RAID) configuration for permitting writing and reading of video media data corresponding to a video stream;
performing mirrored writing of video media data to both the first and second pluralities of data storage devices;
generating respective first and second file allocation tables (FATS) including video media data time stamps and validity information for both of the first and second pluralities of data storage devices; and
selectively reading video media data from both of the first and second pluralities of data storage devices during a playback of the video stream to avoid faulty or outdated video media data on the first and second pluralities of data storage devices based upon the first and second FATs.
19. The method of claim 18 further comprising performing data recovery between the first and second plurality of data storage devices from the mirrored video media data based upon the first and second FATs.
20. The method of claim 19 wherein performing mirrored writing comprises writing the video media data in each of the first and second pluralities of data storage devices as striped video media data; and further comprising:
generating parity data from the striped video media; and
performing data recovery within a given one of the first and second pluralities of data storage devices based upon the parity data.
21. The method of claim 20 further comprising selecting between using the mirrored video media data and the parity data for performing data recovery based upon a number of data storage devices having faults within the given one of the first and second pluralities of data storage devices.
22. The method of claim 18 wherein selectively reading comprises performing load balancing to read the video media data from the first and second pluralities of data storage devices.
23. The method of claim 18 wherein the first and second pluralities of data storage devices are each arranged in a RAID 3 or higher configuration.
US12/039,129 2008-02-28 2008-02-28 Video media data storage system and related methods Expired - Fee Related US7899988B2 (en)

Priority Applications (5)

Application Number Priority Date Filing Date Title
US12/039,129 US7899988B2 (en) 2008-02-28 2008-02-28 Video media data storage system and related methods
EP09715995.8A EP2271986B1 (en) 2008-02-28 2009-02-24 Video media data storage system and related methods
PCT/US2009/034942 WO2009108609A1 (en) 2008-02-28 2009-02-24 Video media data storage system and related methods
CN2009801067241A CN101960429B (en) 2008-02-28 2009-02-24 Video media data storage system and related methods
CA2715967A CA2715967C (en) 2008-02-28 2009-02-24 Video media data storage system and related methods

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US12/039,129 US7899988B2 (en) 2008-02-28 2008-02-28 Video media data storage system and related methods

Publications (2)

Publication Number Publication Date
US20090222622A1 US20090222622A1 (en) 2009-09-03
US7899988B2 true US7899988B2 (en) 2011-03-01

Family

ID=40433923

Family Applications (1)

Application Number Title Priority Date Filing Date
US12/039,129 Expired - Fee Related US7899988B2 (en) 2008-02-28 2008-02-28 Video media data storage system and related methods

Country Status (5)

Country Link
US (1) US7899988B2 (en)
EP (1) EP2271986B1 (en)
CN (1) CN101960429B (en)
CA (1) CA2715967C (en)
WO (1) WO2009108609A1 (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8874958B2 (en) 2010-11-09 2014-10-28 International Business Machines Corporation Error detection in a mirrored data storage system
CN102193857B (en) * 2011-05-21 2014-12-03 浙江工业大学 Method for quantitatively testing abnormal state of file allocation table (FAT) file system in embedded system
CN102226893B (en) * 2011-05-21 2013-01-23 浙江工业大学 FAT (file allocation table) file system repairing method in embedded system
JP5472947B2 (en) * 2012-03-08 2014-04-16 株式会社東芝 Video server apparatus and rebuild process control method thereof
CN102789370B (en) * 2012-06-29 2015-11-25 浙江宇视科技有限公司 A kind of RAID array synchronous method and device
TW201431355A (en) * 2013-01-25 2014-08-01 Elta Technology Co Ltd Expandable multimedia storage system, multimedia distribution device, and relevant computer program product
GB2519815A (en) * 2013-10-31 2015-05-06 Ibm Writing data cross storage devices in an erasure-coded system
CN103714022A (en) * 2014-01-13 2014-04-09 浪潮(北京)电子信息产业有限公司 Mixed storage system based on data block
US10878370B1 (en) * 2014-06-30 2020-12-29 EMC IP Holding Company LLC Conditional formating for display large scale information analytics of reliability data
GB2545221B (en) * 2015-12-09 2021-02-24 7Th Sense Design Ltd Video storage

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6160547A (en) 1995-10-12 2000-12-12 Asc Audio Video Corporation Shared video data storage system with separate video data and information buses
US6223252B1 (en) * 1998-05-04 2001-04-24 International Business Machines Corporation Hot spare light weight mirror for raid system
US6414725B1 (en) 1998-04-16 2002-07-02 Leitch Technology Corporation Method and apparatus for synchronized multiple format data storage
US6480970B1 (en) * 2000-05-17 2002-11-12 Lsi Logic Corporation Method of verifying data consistency between local and remote mirrored data storage systems
US20030084242A1 (en) * 2000-10-04 2003-05-01 Strange Stephen H. Resynchronization of mirrored storage devices
US6892167B2 (en) 2001-11-28 2005-05-10 Sypris Data Systems, Inc. Real-time data acquisition and storage network
US20060020664A1 (en) 2003-06-27 2006-01-26 Fujitsu Limited Method of managing storage capacity, server and recording medium therefor
US20060112219A1 (en) * 2004-11-19 2006-05-25 Gaurav Chawla Functional partitioning method for providing modular data storage systems
US20060129614A1 (en) 2004-12-14 2006-06-15 Kim Hong Y Crash recovery system and method for distributed file server using object based storage
US20060136778A1 (en) * 2002-10-25 2006-06-22 Graverand Philippe Y L Process for generating and reconstructing variable number of parity for byte streams independent of host block size
US20060215700A1 (en) * 2005-03-22 2006-09-28 Zayas Edward R Shared implementation for multiple system interfaces
US7225315B2 (en) 2003-11-11 2007-05-29 Hitachi, Ltd. High read performance file system and program
US20070153906A1 (en) * 2005-12-29 2007-07-05 Petrescu Mihai G Method and apparatus for compression of a video signal
US7266637B1 (en) 2002-05-07 2007-09-04 Veritas Operating Corporation Storage management system
US7299290B2 (en) 2000-03-22 2007-11-20 Yottayotta, Inc. Method and system for providing multimedia information on demand over wide area networks

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US1675556A (en) * 1927-05-21 1928-07-03 Wildman Mfg Co Yarn guide for knitting machines
US7024586B2 (en) * 2002-06-24 2006-04-04 Network Appliance, Inc. Using file system information in raid data reconstruction and migration

Patent Citations (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6160547A (en) 1995-10-12 2000-12-12 Asc Audio Video Corporation Shared video data storage system with separate video data and information buses
US6414725B1 (en) 1998-04-16 2002-07-02 Leitch Technology Corporation Method and apparatus for synchronized multiple format data storage
US6223252B1 (en) * 1998-05-04 2001-04-24 International Business Machines Corporation Hot spare light weight mirror for raid system
US7299290B2 (en) 2000-03-22 2007-11-20 Yottayotta, Inc. Method and system for providing multimedia information on demand over wide area networks
US6480970B1 (en) * 2000-05-17 2002-11-12 Lsi Logic Corporation Method of verifying data consistency between local and remote mirrored data storage systems
US7143249B2 (en) 2000-10-04 2006-11-28 Network Appliance, Inc. Resynchronization of mirrored storage devices
US20030084242A1 (en) * 2000-10-04 2003-05-01 Strange Stephen H. Resynchronization of mirrored storage devices
US6892167B2 (en) 2001-11-28 2005-05-10 Sypris Data Systems, Inc. Real-time data acquisition and storage network
US7266637B1 (en) 2002-05-07 2007-09-04 Veritas Operating Corporation Storage management system
US20060136778A1 (en) * 2002-10-25 2006-06-22 Graverand Philippe Y L Process for generating and reconstructing variable number of parity for byte streams independent of host block size
US20060020664A1 (en) 2003-06-27 2006-01-26 Fujitsu Limited Method of managing storage capacity, server and recording medium therefor
US7225315B2 (en) 2003-11-11 2007-05-29 Hitachi, Ltd. High read performance file system and program
US20060112219A1 (en) * 2004-11-19 2006-05-25 Gaurav Chawla Functional partitioning method for providing modular data storage systems
US20060129614A1 (en) 2004-12-14 2006-06-15 Kim Hong Y Crash recovery system and method for distributed file server using object based storage
US20060215700A1 (en) * 2005-03-22 2006-09-28 Zayas Edward R Shared implementation for multiple system interfaces
US20070153906A1 (en) * 2005-12-29 2007-07-05 Petrescu Mihai G Method and apparatus for compression of a video signal

Non-Patent Citations (15)

* Cited by examiner, † Cited by third party
Title
Definition of data; Free Online Dictionary of Computing. *
Definition of video; Free Online Dictionary of Computing. *
Harris Corporation Announces Launch of Leitch NEXIO XS SD/HD Server System; Harris Corporation; Apr. 4, 2006. *
Harris Corporation Launches Next-Generation NEXIO AMP Advanced Media Platform, News Release, Sep. 6, 2007.
Hybrid Redundancy Direct-Access Storage Device Array with Design Option NB9402141; IBM Technical Disclosure Bulletin; Feb. 1994. *
Leitch Introduces VR400 MPEG-2 Server; Business Wire; Mar. 17, 1999. *
Leitch Technology Corporation Investor Presentation; Aug. 2004. *
Nexio Amp, Advanced Media Platform, Harris Corporation, 2007.
NEXIO AMP; Harris Corporation; copyright 2007. *
NEXIO Server System for Your Integrated Content Environment; Harris Corporation; copyright 2003. *
Optimal Data Allotment to Build High Availability and High Performance Disk Arrays NN940575; IBM Technical Disclosure Bulletin; May 1, 1994. *
RAIDsoft: An Innovative Distributed Software RAID which Protects Broadcast Video Server Customers against Multiple Disk Drive Failures; Leitch Server Group; Sep. 1997. *
Raidsoft; Schultz, Fred; Jun. 20, 2000. *
The Nexio XS Server System; Harris Corporation; copyright 2007. *
U.S. Appl. No. 11/675,556, filed Feb. 15, 2007, Petrescu.

Also Published As

Publication number Publication date
CA2715967C (en) 2016-11-22
WO2009108609A1 (en) 2009-09-03
EP2271986A1 (en) 2011-01-12
US20090222622A1 (en) 2009-09-03
CN101960429A (en) 2011-01-26
EP2271986B1 (en) 2014-10-01
CA2715967A1 (en) 2009-09-03
CN101960429B (en) 2013-08-28

Similar Documents

Publication Publication Date Title
US7899988B2 (en) Video media data storage system and related methods
US6934904B2 (en) Data integrity error handling in a redundant storage array
US5513192A (en) Fault tolerant disk drive system with error detection and correction
US7640452B2 (en) Method for reconstructing data in case of two disk drives of RAID failure and system therefor
US7017107B2 (en) Storage array employing scrubbing operations at the disk-controller level
US7890815B2 (en) Detection and correction of dropped write errors in a data storage system
US6546499B1 (en) Redundant array of inexpensive platters (RAIP)
JP4815825B2 (en) Disk array device and method for reconstructing the same
CN100368976C (en) Disk array apparatus and backup method of data
US20020162076A1 (en) Storage array employing scrubbing operations using multiple levels of checksums
US7484137B2 (en) RAID system using regional error statistics for redundancy grouping
US20050283654A1 (en) Method and apparatus for decreasing failed disk reconstruction time in a raid data storage system
CN104484251A (en) Method and device for processing faults of hard disk
US8645622B2 (en) Method to protect data on a disk drive from uncorrectable media errors
US7793167B2 (en) Detection and correction of dropped write errors in a data storage system
US7730370B2 (en) Apparatus and method for disk read checking
KR101054814B1 (en) Method and data storage system to increase the error tolerance of the array of storage units
US20120304025A1 (en) Dual hard disk drive system and method for dropped write detection and recovery
CN110309012B (en) Data processing method and device
Iliadis Effect of lazy rebuild on reliability of erasure-coded storage systems
US20050102470A1 (en) Disk array device
US20180181467A1 (en) Hard disk array and method for reconstructing thereof
US7343519B2 (en) Disk drive power cycle screening method and apparatus for data storage system
Iliadis Reliability modeling of RAID storage systems with latent errors
JP4154776B2 (en) Data recording / reproducing apparatus and method, and server

Legal Events

Date Code Title Description
AS Assignment

Owner name: HARRIS CORPORATION, FLORIDA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:PETRESCU, MIHAI G.;CREVE, HILTON S.;TRAN, TUNG M.;AND OTHERS;REEL/FRAME:021251/0578;SIGNING DATES FROM 20080225 TO 20080301

Owner name: HARRIS CORPORATION, FLORIDA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:PETRESCU, MIHAI G.;CREVE, HILTON S.;TRAN, TUNG M.;AND OTHERS;SIGNING DATES FROM 20080225 TO 20080301;REEL/FRAME:021251/0578

AS Assignment

Owner name: HBC SOLUTIONS, INC., COLORADO

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HARRIS CORPORATION;EAGLE TECHNOLOGY INC.;REEL/FRAME:029759/0416

Effective date: 20130204

AS Assignment

Owner name: WILMINGTON TRUST, NATIONAL ASSOCIATION, MINNESOTA

Free format text: SECURITY AGREEMENT;ASSIGNOR:HB CANADA COMMUNICATIONS LTD;REEL/FRAME:030156/0751

Effective date: 20130329

Owner name: WILMINGTON TRUST, NATIONAL ASSOCIATION, MINNESOTA

Free format text: SECURITY AGREEMENT;ASSIGNOR:HBC SOLUTIONS, INC.;REEL/FRAME:030156/0636

Effective date: 20130204

AS Assignment

Owner name: PNC BANK, NATIONAL ASSOCIATION, AS AGENT, NEW JERS

Free format text: SECURITY AGREEMENT;ASSIGNOR:HBC SOLUTIONS, INC.;REEL/FRAME:030192/0355

Effective date: 20130204

AS Assignment

Owner name: HBC SOLUTIONS, INC., COLORADO

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HARRIS CORPORATION;EAGLE TECHNOLOGY, LLC;REEL/FRAME:030333/0671

Effective date: 20130204

REMI Maintenance fee reminder mailed
LAPS Lapse for failure to pay maintenance fees
STCH Information on status: patent discontinuation

Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362

FP Lapsed due to failure to pay maintenance fee

Effective date: 20150301