WO2008029099A1 - Distributed data allocation system - Google Patents
Distributed data allocation system Download PDFInfo
- Publication number
- WO2008029099A1 WO2008029099A1 PCT/GB2007/003306 GB2007003306W WO2008029099A1 WO 2008029099 A1 WO2008029099 A1 WO 2008029099A1 GB 2007003306 W GB2007003306 W GB 2007003306W WO 2008029099 A1 WO2008029099 A1 WO 2008029099A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- store
- storage locations
- unused
- data
- devices
- Prior art date
Links
- 238000000034 method Methods 0.000 claims description 15
- 239000000463 material Substances 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims 1
- 230000008901 benefit Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 238000007726 management method Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 230000008021 deposition Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0644—Management of space entities, e.g. partitions, extents, pools
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
- G06F12/023—Free address space management
- G06F12/0253—Garbage collection, i.e. reclamation of unreferenced memory
- G06F12/0261—Garbage collection, i.e. reclamation of unreferenced memory using reference counting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F12/0806—Multiuser, multiprocessor or multiprocessing cache systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
- G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
- G11B27/034—Electronic editing of digitised analogue information signals, e.g. audio or video signals on discs
Definitions
- This invention provides a system for the distribution of data from plural client locations into a shared electronic storage facility, from which facility stored data may be retrieved and used at any of the client locations.
- the invention also provides a method of operating such a system.
- Quantel currently uses a system, called "Frame Magic", to manage the allocation of disk space.
- This system requires a defined unit of minimum size, such that a composite data allocation, such as a video clip, can be divided into such units, each of which is stored at a respective and unique address in the store, which address is thereafter, for convenience, used as the identity of the unit.
- a clip comprises an assemblage of such units, and that parts of clips may be edited together, making new clips. The act of editing clips together can, and often does, create clips that share at least some of the units.
- Frame Magic uses a reference counting mechanism to manage the lifetime of shared units; each unit having associated therewith a reference count, indicating the number of clips currently including that unit.
- Frame Magic keeps a central map of disk space in the system, with contiguous disk addresses that have the same reference count maintained as a single run length coded entry.
- Frame Magic also keeps a map of free storage space, comprising those addresses of the central map that have reference counts of zero; in this respect it will be appreciated that some addresses may have reference counts of zero because they have never previously been allocated for data storage, whilst others may have been previously allocated for the storage of data that is no longer used.
- Frame Magic makes an allocation of storage addresses sufficient to accommodate all new units contained in the clip by selecting an appropriate number of storage addresses from the free space map, and updates the reference counts in the disk space map to reflect their usage.
- a data allocation system for allocating storage space in a store shared by a plurality of operating devices; the store having associated therewith distribution means for selectively distributing data units to respective storage locations of the store in dependence upon a determination of availability of said storage locations; wherein each said operating device includes a respective evaluating means for identifying storage locations of said store currently unused by that operating device and disclosure means for providing a listing of storage locations so identified, interrogation means for interrogating the disclosure means of all of the other operating devices in the system to obtain said listings therefrom, and combining means for logically combining the listings obtained from the other operating devices with the unused storage locations identified by its own evaluating means to determine a full listing of storage space in the shared store that is currently unused by any of said operating devices; said full listing constituting said determination and enabling said distribution means to distribute data units originating from any of said operating devices to respective unused storage locations of said store.
- the operating devices are video editing suites and the data units represent video information.
- the video information may comprise edited material and/or otherwise processed or unprocessed material. It is preferred, though not essential, that the storage of video information is implemented on a frame-by-frarne basis, with each said data unit comprising a frame of video information.
- the degree of use of data units held in the store is indicated by respective reference counts associated with the various storage locations; the reference count for a given storage location being increased each time a data unit stored at that location is accessed by an operating device and decremented each time a data unit stored at that location is discarded by an operating device.
- a storage location having a reference count of zero associated therewith is indicated as being available for data unit distribution thereto.
- a zero count in respect of any storage location indicates either that no data unit has previously been distributed to that storage location or that any data units so distributed have been discarded by all of the operating devices sharing the store.
- no distribution of data units to the store is considered complete until the reference counts associated with the subject storage locations are registered as non-zero. It is preferred that each operating device maintains a copy of the last known information identifying storage locations unused by each of the other operating devices. By this means it is ensured that non-availability of current information from any one operating device does not delay the distribution of data units originating from any of the other operating devices to respective unused storage locations of said store.
- the store may comprise a unitary store, for example a single disk or other electronic store of any convenient kind. Alternatively, an array of disks and/or other electronic stores can be used if capacity or convenience so dictates.
- a method of allocating storage space in a store shared by a plurality of operating devices comprising the steps of: selectively distributing data units to respective storage locations of the store in dependence upon a determination of availability of said storage locations; identifying storage locations of said store currently unused by each operating device individually and providing information listing said unused storage locations, interrogating all of the other operating devices to obtain listing information therefrom, and logically combining the listing information obtained from all operating devices to determine a listing of storage locations in the shared store currently unused by any operating device, thereby providing said determination, and utilising said determination to distribute data units originating from any of said operating devices to respective unused storage locations of said store.
- Figure 1 shows, in schematic and block-diagrammatic form, a system in accordance with one embodiment of the invention
- Figure 2 shows a flow diagram explanatory of the operation of the system shown in and described with reference to Figure 1, and an operational methodology therefor;
- Figure 3 shows a sequence diagram illustrative of the operation of the embodiment of the invention described with reference to Figures 1 and 2.
- FIG. 1 provides a distributed system providing storage allocation management which allows plural client devices 20, 30, 40 and 50 to share a common storage facility 10, in this case a disk, or an array of disks, such that each client device forfeits storage allocations of the other devices, even in the face of device failure.
- the system does not require a central authority (such as a central server), as it uses peer-to-peer communication to control allocations of storage space.
- Each client device such as 20 is in communication not only with the common storage facility 10 (as indicated by the connections 12, 13, 14 and 15 each of which, in practice, is constituted by a digital data bus), but also with its peers via an interconnecting bus 23, 34, 45 which links all of the client devices 20, 30, 40 and 50 together for digital communication.
- a request is generated which seeks from each peer client device 30, 40 and 50 a list of all the storage addresses on the disk 10, or in the disk array, which that peer is not currently using. That is, a list of all storage addresses which do not contain data units currently employed in clips maintained or in process on that client device.
- a client device (say 20) requesting an allocation of storage addresses may take the free space list of any of its peer devices 30, 40 or 50 and logically "NOT" it to infer all the space that the other device is currently using. However the requesting client device has no interest in the actual reference counts of the space in use by its peer. The free space list of the other device is never mutated by the device requesting an allocation of storage addresses, so no integer counts are required to pass between machines.
- each client device such as 20 keeps its own reference count based upon the clips in its own database.
- the reference count drops to zero, and the address of that unit is returned to the free space list on that device.
- freed-up disk addresses are not available for re-allocation until they are included in the free space lists of all peer devices.
- a main benefit of the distributed system described herein is that it takes advantage of the reference counting mechanism of Frame Magic without requiring any exposure, as between peers, of the highly volatile reference counts themselves.
- the structure exploits the fact that editing clips is performed relatively frequently, whereas allocation of new space is performed relatively infrequently, i.e. only as and when finished clips are accepted. Further, allocation implies a movement of some media which is orders of magnitude more costly than the allocation itself.
- Another benefit of the distributed system is that disk storage addresses, once allocated, are not left "orphaned" by client devices reserving them and then losing their reference to them (a situation known as "leaking"). Each client is asked what storage addresses it is actually using in a dynamic on-going exchange of communication with its peers.
- Each client device maintains a copy of the last known free space list for each of its peers, so that non-availability of a peer will not delay an allocation of storage addresses.
- An allocation is not treated as complete until the operational control system driving the central disk (or disk array) confirms the allocation by way of its reference counts.
- a central database would not have this property, and would require some sort of client reconcile process to recover such leaked space.
- FIG. 2 shows a flow diagram representative of the operation of the system just described.
- the system checks at 102 whether the allocation system has been temporarily locked in response to a precedent request issued by one of the other client devices. If the system is locked, the formal request is resubmitted after a delay imposed in a loop 103 and the process re-starts. If the system is not currently locked, a locking signal is broadcast at 104, to prevent clashes with subsequent requests issued by any of the other client devices, and the inter-client communication bus 23, 34, 45 is interrogated at step 105 to secure from each peer device a list of all storage locations in the common store 10 which, as far as that peer device is concerned, are free. A check is made at step 106 as to whether all peer devices have responded and, if any of the peer devices fails to provide a list at this point, its last-known list of free spaces is derived from the system at step 107.
- all free-space lists are merged at step 108 in accordance with an AND function, to generate the aforementioned true free space list, on the basis of which a self-allocation of the requisite number of free spaces in the common store 10 is made at step 109; this allocation being confirmed at 110 when the data are actually deposited into the storage addresses requested.
- FIG. 3 shows a sequence diagram illustrating the implementation of a specific algorithm for an allocation of video data to a store in accordance with the system and method described hereinbefore.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
In a system for allocating storage space for data shared between a plurality of data processing devices, each device provides an indication of storage locations unused by that device, receives corresponding indications from each of the other said devices, determines therefrom free locations not used by any of said devices, and allocates said free locations. non responder
Description
Distributed Data Allocation System
Field of the Invention
This invention provides a system for the distribution of data from plural client locations into a shared electronic storage facility, from which facility stored data may be retrieved and used at any of the client locations. The invention also provides a method of operating such a system.
Background of the Invention
Major electronic storage facilities, such as large disk stores used in association with video editing suites, demand powerful management of the distribution to and storage of data therein to ensure that efficient use is made of their extensive but expensive storage capacity.
Quantel currently uses a system, called "Frame Magic", to manage the allocation of disk space. This system requires a defined unit of minimum size, such that a composite data allocation, such as a video clip, can be divided into such units, each of which is stored at a respective and unique address in the store, which address is thereafter, for convenience, used as the identity of the unit. It will be appreciated that a clip comprises an assemblage of such units, and that parts of clips may be edited together, making new clips. The act of editing clips together can, and often does, create clips that share at least some of the units. Frame Magic uses a reference counting mechanism to manage the lifetime of shared units; each unit having associated therewith a reference count, indicating the number of clips currently including that unit.
Frame Magic keeps a central map of disk space in the system, with contiguous disk addresses that have the same reference count maintained as a single run length coded entry. Frame Magic also keeps a map of free storage space, comprising those addresses of the central map that have reference counts of zero; in this respect it will be appreciated that some addresses may have reference counts of zero because they have never previously been allocated for data storage, whilst others may have been previously allocated for the storage of data that is no longer used. When some disk space is requested by an editing suite, for a newly received and/or edited clip, for example, Frame Magic makes an allocation of storage addresses
sufficient to accommodate all new units contained in the clip by selecting an appropriate number of storage addresses from the free space map, and updates the reference counts in the disk space map to reflect their usage.
It will be appreciated that, whereas the creation of a new clip using any given unit already stored will cause the reference count for that particular stored unit to increase, the deletion of a clip incorporating any given stored unit will decrement the reference count for that particular stored unit. When the reference count for a stored unit decrements to zero, this indicates that no clips incorporating that unit are currently in use, so the storage location associated with that unit is "de-allocated", by returning its address back to the free space map.
Difficulties arise in adapting systems such as Frame Magic to accommodate shared usage of the storage facility by plural client devices (e.g. plural editing suites), and it is an object of the invention to address such difficulties.
Statement of the Invention According to one aspect of the invention, therefore, there is provided a data allocation system for allocating storage space in a store shared by a plurality of operating devices; the store having associated therewith distribution means for selectively distributing data units to respective storage locations of the store in dependence upon a determination of availability of said storage locations; wherein each said operating device includes a respective evaluating means for identifying storage locations of said store currently unused by that operating device and disclosure means for providing a listing of storage locations so identified, interrogation means for interrogating the disclosure means of all of the other operating devices in the system to obtain said listings therefrom, and combining means for logically combining the listings obtained from the other operating devices with the unused storage locations identified by its own evaluating means to determine a full listing of storage space in the shared store that is currently unused by any of said operating devices; said full listing constituting said determination and enabling said distribution means to distribute data units originating from any of said operating devices to respective unused storage locations of said store.
In one preferred embodiment of the invention, the operating devices are video editing suites and the data units represent video information. The video information may comprise edited material and/or otherwise processed or unprocessed material. It is preferred, though not essential, that the storage of video information is implemented on a frame-by-frarne basis, with each said data unit comprising a frame of video information.
In preferred embodiments of the invention, the degree of use of data units held in the store is indicated by respective reference counts associated with the various storage locations; the reference count for a given storage location being increased each time a data unit stored at that location is accessed by an operating device and decremented each time a data unit stored at that location is discarded by an operating device. In such preferred embodiments, a storage location having a reference count of zero associated therewith is indicated as being available for data unit distribution thereto. A zero count in respect of any storage location indicates either that no data unit has previously been distributed to that storage location or that any data units so distributed have been discarded by all of the operating devices sharing the store. Moreover, in some preferred embodiments of the invention, no distribution of data units to the store is considered complete until the reference counts associated with the subject storage locations are registered as non-zero. It is preferred that each operating device maintains a copy of the last known information identifying storage locations unused by each of the other operating devices. By this means it is ensured that non-availability of current information from any one operating device does not delay the distribution of data units originating from any of the other operating devices to respective unused storage locations of said store. The store may comprise a unitary store, for example a single disk or other electronic store of any convenient kind. Alternatively, an array of disks and/or other electronic stores can be used if capacity or convenience so dictates.
According to the invention from another aspect there is provided a method of allocating storage space in a store shared by a plurality of operating devices; the method comprising the steps of: selectively distributing data units to respective storage locations of the store in dependence upon a determination of availability of said storage locations; identifying storage locations of said store currently unused by
each operating device individually and providing information listing said unused storage locations, interrogating all of the other operating devices to obtain listing information therefrom, and logically combining the listing information obtained from all operating devices to determine a listing of storage locations in the shared store currently unused by any operating device, thereby providing said determination, and utilising said determination to distribute data units originating from any of said operating devices to respective unused storage locations of said store.
Brief Description of the Drawings
In order that the invention may be clearly understood and readily carried into effect, one embodiment thereof will now be described, by way of example only, with reference to the accompanying drawings, of which:
Figure 1 shows, in schematic and block-diagrammatic form, a system in accordance with one embodiment of the invention;
Figure 2 shows a flow diagram explanatory of the operation of the system shown in and described with reference to Figure 1, and an operational methodology therefor; and
Figure 3 shows a sequence diagram illustrative of the operation of the embodiment of the invention described with reference to Figures 1 and 2.
Description of Embodiments of the Invention The embodiment of the invention shown in Figure 1 provides a distributed system providing storage allocation management which allows plural client devices 20, 30, 40 and 50 to share a common storage facility 10, in this case a disk, or an array of disks, such that each client device honours storage allocations of the other devices, even in the face of device failure. The system does not require a central authority (such as a central server), as it uses peer-to-peer communication to control allocations of storage space. Each client device such as 20 is in communication not only with the common storage facility 10 (as indicated by the connections 12, 13, 14 and 15 each of which, in practice, is constituted by a digital data bus), but also with its peers via an interconnecting bus 23, 34, 45 which links all of the client devices 20, 30, 40 and 50 together for digital communication.
Thus, whenever one client device such as 20 requires an allocation of storage capacity, a request is generated which seeks from each peer client device 30, 40 and 50 a list of all the storage addresses on the disk 10, or in the disk array, which that peer is not currently using. That is, a list of all storage addresses which do not contain data units currently employed in clips maintained or in process on that client device. The client device 20 requesting the allocation of storage space on the shared disk 10 (or in the shared disk array) then logically "ANDS" together its free spaces and the free spaces listed by all of its peer devices 30, 40 and 50, producing a true free space list; i.e. a list of all disk addresses which are either empty (i.e. have not previously been allocated) or which contain data units that are no longer in use by any of the client devices (and can thus be safely over- written).
This procedure takes advantage of some of the properties of reference counting, since all storage addresses included in the true free space list have, by definition, reference counts of zero. A client device (say 20) requesting an allocation of storage addresses may take the free space list of any of its peer devices 30, 40 or 50 and logically "NOT" it to infer all the space that the other device is currently using. However the requesting client device has no interest in the actual reference counts of the space in use by its peer. The free space list of the other device is never mutated by the device requesting an allocation of storage addresses, so no integer counts are required to pass between machines.
As usual with Frame Magic, each client device such as 20 keeps its own reference count based upon the clips in its own database. When all clips containing a given unit are finished with, the reference count drops to zero, and the address of that unit is returned to the free space list on that device. However such freed-up disk addresses are not available for re-allocation until they are included in the free space lists of all peer devices.
A main benefit of the distributed system described herein is that it takes advantage of the reference counting mechanism of Frame Magic without requiring any exposure, as between peers, of the highly volatile reference counts themselves. The structure exploits the fact that editing clips is performed relatively frequently, whereas allocation of new space is performed relatively infrequently, i.e. only as and when
finished clips are accepted. Further, allocation implies a movement of some media which is orders of magnitude more costly than the allocation itself. Another benefit of the distributed system is that disk storage addresses, once allocated, are not left "orphaned" by client devices reserving them and then losing their reference to them (a situation known as "leaking"). Each client is asked what storage addresses it is actually using in a dynamic on-going exchange of communication with its peers.
Each client device maintains a copy of the last known free space list for each of its peers, so that non-availability of a peer will not delay an allocation of storage addresses. An allocation is not treated as complete until the operational control system driving the central disk (or disk array) confirms the allocation by way of its reference counts. As each peer restarts, it re-creates its own free space list out of the storage addresses that have been confirmed in a database. Therefore any allocations it may have requested, but which have not been confirmed, are automatically marked as free. A central database would not have this property, and would require some sort of client reconcile process to recover such leaked space.
Figure 2 shows a flow diagram representative of the operation of the system just described. Following a formal request for storage initiated at 101 by one of the client devices 20, 30, 40 or 50 the system checks at 102 whether the allocation system has been temporarily locked in response to a precedent request issued by one of the other client devices. If the system is locked, the formal request is resubmitted after a delay imposed in a loop 103 and the process re-starts. If the system is not currently locked, a locking signal is broadcast at 104, to prevent clashes with subsequent requests issued by any of the other client devices, and the inter-client communication bus 23, 34, 45 is interrogated at step 105 to secure from each peer device a list of all storage locations in the common store 10 which, as far as that peer device is concerned, are free. A check is made at step 106 as to whether all peer devices have responded and, if any of the peer devices fails to provide a list at this point, its last-known list of free spaces is derived from the system at step 107.
In any event, all free-space lists are merged at step 108 in accordance with an AND function, to generate the aforementioned true free space list, on the basis of which a
self-allocation of the requisite number of free spaces in the common store 10 is made at step 109; this allocation being confirmed at 110 when the data are actually deposited into the storage addresses requested.
If the deposition is confirmed, the reference counts for the relevant storage addresses in the common store 10 are up-dated at 111 following which step an unlocking signal is broadcast at 112 to release the system for further allocations. In the event that the allocation is not confirmed at step 110, the system is unlocked without any change to the reference counts via the connection 113. Figure 3 shows a sequence diagram illustrating the implementation of a specific algorithm for an allocation of video data to a store in accordance with the system and method described hereinbefore.
Claims
1. A data allocation system for allocating storage space in a store shared by a plurality of operating devices; the store having associated therewith distribution means for selectively distributing data units to respective storage locations of the store in dependence upon a determination of availability of said storage locations; wherein each said operating device includes a respective evaluating means for identifying storage locations of said store currently unused by that operating device and disclosure means for providing a listing of storage locations so identified, interrogation means for interrogating the disclosure means of all of the other operating devices in the system to obtain said listings therefrom, and combining means for logically combining the listings obtained from the other operating devices with the unused storage locations identified by its own evaluating means to determine a full listing of storage space in the shared store that is currently unused by any of said operating devices; said full listing constituting said determination and enabling said distribution means to distribute data units originating from any of said operating devices to respective unused storage locations of said store.
2. A system according to claim 1, wherein each of said operating devices comprises a video editing suite and the data units represent video information.
3. A system according to claim 2, wherein the video information comprises edited material and/or otherwise processed or unprocessed material.
4. A system according to claim 2 or claim 3, further including means for storing said video information on a frame-by-frame basis, with each said data unit comprising a frame of video information.
5. A system according to any preceding claim, further comprising means for indicating the degree of use of data units held in the store by respective reference counts associated with the various storage locations.
6. A system according to claim 5, wherein the means for indicating is configured such that the reference count for a given storage location is increased each time a data unit stored at that location is accessed by an operating device and decremented each time a data unit stored at that location is discarded by an operating device.
7. A system according to claim 5 or claim 6, wherein a storage location having a reference count of zero associated therewith is indicated as being available for data unit distribution thereto.
8. A system according to claim 7, comprising further means for confirming a distribution of data units to the store as complete once the reference counts associated with the subject storage locations are registered as non-zero.
9. A system according to any preceding claim, wherein each operating device comprises a memory adapted to maintain a copy of the last known information identifying storage locations unused by each of the other operating devices, whereby non-availability of current information from any one operating device does not delay the distribution of data units originating from any of the other operating devices to respective unused storage locations of said store.
10. A system according to any preceding claim, wherein said store comprises a unitary store in the form of a single disk or other electronic store.
11. A system according to any of claims 1 to 9, wherein said store comprises an array of disks and/or other electronic stores.
12. A method of allocating storage space in a store shared by a plurality of operating devices; the method comprising the steps of: selectively distributing data units to respective storage locations of the store in dependence upon a determination of availability of said storage locations; identifying storage locations of said store currently unused by each operating device individually and providing information listing said unused storage locations, interrogating all of the other operating devices to obtain listing information therefrom, and logically combining the listing information obtained from all operating devices to determine a listing of storage locations in the shared store currently unused by any operating device, thereby providing said determination, and utilising said determination to distribute data units originating from any of said operating devices to respective unused storage locations of said store.
13. A system for allocating storage space for data shared between a plurality of data processing devices, each being arranged to provide an indication of storage locations unused by that device, to receive corresponding said indications from each of the other said devices, to determine therefrom free locations not used by any of said devices, and to allocate said free locations as said storage space.
14. A method of storing data in a common storage space, the data being shared between a plurality of data processing devices, the method comprising, in one of said data processing devices, receiving an indication from each of the other processing devices of locations unused by the respective data processing device; determining therefrom free locations unused by any of the respective data processing devices, and allocating said data in one or more of said free locations.
15. A computer program comprising program code means arranged to perform the method of claim 14.
16. A data processing device arranged to perform the method of claim 14.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GB0617526A GB2444083B (en) | 2006-09-05 | 2006-09-05 | Distributed data allocation system |
GB0617526.9 | 2006-09-05 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2008029099A1 true WO2008029099A1 (en) | 2008-03-13 |
Family
ID=37232445
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/GB2007/003306 WO2008029099A1 (en) | 2006-09-05 | 2007-09-03 | Distributed data allocation system |
Country Status (2)
Country | Link |
---|---|
GB (1) | GB2444083B (en) |
WO (1) | WO2008029099A1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2010142956A2 (en) | 2009-06-09 | 2010-12-16 | University Court Of The University Of Dundee | Compounds |
US9668100B2 (en) | 2015-04-03 | 2017-05-30 | International Business Machines Corporation | Locating misplaced mobile computing devices |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2273220A (en) * | 1992-12-07 | 1994-06-08 | Quantel Ltd | A video editing system |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4912629A (en) * | 1986-06-26 | 1990-03-27 | The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration | Real-time garbage collection for list processing using restructured cells for increased reference counter size |
US7474832B2 (en) * | 2002-11-21 | 2009-01-06 | International Business Machines Corporation | Personal video recording with storage space distributed among remote personal video recorders |
US8001088B2 (en) * | 2003-04-04 | 2011-08-16 | Avid Technology, Inc. | Indexing media files in a distributed, multi-user system for managing and editing digital media |
-
2006
- 2006-09-05 GB GB0617526A patent/GB2444083B/en not_active Expired - Fee Related
-
2007
- 2007-09-03 WO PCT/GB2007/003306 patent/WO2008029099A1/en active Application Filing
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2273220A (en) * | 1992-12-07 | 1994-06-08 | Quantel Ltd | A video editing system |
Non-Patent Citations (2)
Title |
---|
"Quantel White Paper. Broadcast News: How technology architecture choices impact business", INTERNET CITATION, April 2006 (2006-04-01), XP007903711, Retrieved from the Internet <URL:http://www.quantel.com/resource.nsf/Files/Broadcast+News+Architecture +White+Paper/$FILE/Broadcast+News+Architecture+White+Paper.pdf> [retrieved on 20071214] * |
MANCINI L ET AL: "FAULT-TOLERANT REFERENCE COUNTING FOR GARBAGE COLLECTION IN DISTRIBUTED SYSTEMS", COMPUTER JOURNAL, OXFORD UNIVERSITY PRESS, SURREY, GB, vol. 34, no. 6, 1 December 1991 (1991-12-01), pages 503 - 513, XP000330326, ISSN: 0010-4620 * |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2010142956A2 (en) | 2009-06-09 | 2010-12-16 | University Court Of The University Of Dundee | Compounds |
US9668100B2 (en) | 2015-04-03 | 2017-05-30 | International Business Machines Corporation | Locating misplaced mobile computing devices |
US10368194B2 (en) | 2015-04-03 | 2019-07-30 | International Business Machines Corporation | Locating misplaced mobile computing devices |
Also Published As
Publication number | Publication date |
---|---|
GB2444083A (en) | 2008-05-28 |
GB0617526D0 (en) | 2006-10-18 |
GB2444083B (en) | 2011-06-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP4049525B2 (en) | Distributed processing system | |
CN109597567B (en) | Data processing method and device | |
CN113806066A (en) | Big data resource scheduling method, system and storage medium | |
JP4197303B2 (en) | Computer resource management method, execution apparatus, and processing program | |
CN109525658A (en) | A kind of production method, server, equipment, storage medium and operation system | |
US20140040893A1 (en) | Selecting provisioning targets for new virtual machine instances | |
CN110244901B (en) | Task allocation method and device and distributed storage system | |
CN112153167B (en) | Internet interconnection protocol management method, device, electronic equipment and storage medium | |
CN110245129B (en) | Distributed global data deduplication method and device | |
CN109308269B (en) | Memory management method and device | |
US20100138627A1 (en) | Efficient alert mechanism for overutilization of storage resources | |
CN109543365B (en) | Authorization method and device | |
CN111399761B (en) | Storage resource allocation method, device and equipment, and storage medium | |
CN110012050A (en) | Message Processing, storage method, apparatus and system | |
CN111190537B (en) | Method and system for managing sequential storage disk in additional writing scene | |
CN110765094B (en) | File creation method, device, system and storage medium | |
CN112688799A (en) | Client number distribution method and device based on Redis cluster | |
CN105978744B (en) | A kind of resource allocation methods, apparatus and system | |
CN109669621A (en) | A kind of file management method, file management system, electronic equipment and storage medium | |
CN111163186A (en) | ID generation method, device, equipment and storage medium | |
CN110798507A (en) | Method and system for DHCP address allocation memory | |
WO2008029099A1 (en) | Distributed data allocation system | |
CN111800494B (en) | Equipment online method, system and device and electronic equipment | |
CN107122136B (en) | Capacity obtaining method and device | |
CN112527496B (en) | Sequence number distribution service system, sequence number distribution method, server, and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 07804112 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 07804112 Country of ref document: EP Kind code of ref document: A1 |