GB2504534A

GB2504534A - Determining whether a data block requires compression before being stored

Info

Publication number: GB2504534A
Application number: GB1213763.4A
Authority: GB
Inventors: Matthew James Jarvis; David Haikney; William Owen Lovett; James Walker
Original assignee: International Business Machines Corp
Current assignee: International Business Machines Corp
Priority date: 2012-08-02
Filing date: 2012-08-02
Publication date: 2014-02-05
Also published as: GB201213763D0

Abstract

A storage controller 210 receives data blocks 241, 242 to be stored in a storage system 211-213, and determines whether the block of data needs to be compressed 230 before being written to disk by means of an indication of the compression status provided with the data block, which may be a flag 251, 252. If the flag indicates that the block is already compressed, such as a ZIP file or an encoded media file, the system may pass it through for storage without causing the compression engine 220 to attempt compression of the block; otherwise the compression engine creates a compressed block 261. This prevents redundant work from being performed by the compression engine, and therefore reduces the workload of the system. The flag may also permit bypassing a decompression engine on read. The flag may be set by an application at the host 201 which opens the file the data block comprises before it is forwarded to the storage system.

Description

REDUCING WORKLOAD ON A STORAGE SYSTEM

FIELD OF INVENTION

This invention relates to the fie'd of storage systems. In particular, the invention relates to reducing workload on a storage system.

BACKGROUND OF INVENTION

Real time compression in storage systems is an increasingly used feature in customer enterprises, but represents a significant CPU and memory load when integrated into the underlying storage systems.

An alternative solution to this issue is to have an external inline compression engine using 1 5 separate hardware, but this creates increased cost, and further issues with manageability, space and energy usage.

Therefore, there is a need in the art to address the aforementioned problems.

BRIEF SUMMARY OF THE INVENTION

According to a first aspect of the present invention there is provided a method for reducing workload on a storage system, comprising: receiving a block of data at a storage system together with an indication of compression status of the block of data and determining from the indication of compression status if the block of data needs compressing for storagc.

If it is determined that a block of data needs compressing, the method may include compressing the block of data at a compression engine. However, if it is determined that a block of data does not need compressing, the method may pass the block of data through a compression engine without compressing or bypass the compression engine.

The compression status may be compressed or un-compressed. The indication of compression status of a block of data may be provided as a SCSI command. A single indication of compression status may be applied to multiple blocks of data.

An application may opcn a flic for writing and the applicatioll may provide an indication of compression status of the file to a file system. The indication of compression status of a file may be a flag indicating the file is compressed or un-compressed.

The method may indude a file system forwarding a received indication of compression status for a file as an indication of compression status of blocks of data of the file to a hardware device driver for transmitting to a storage system.

The method may include recording the indication of compression status for a block of data at metadata of a compression engine of a storage controller. The method may further include skipping dc-compression of a block of data on read if the metadata of the compression engine indicates that the block of data was not compressed on write.

According to a second aspect of the present invention there is provided a system for reducing workload on a storage system comprising: a storage controller enabled to receive a block of data for storage including an indication of compression status of the block of data; and a compression indication determining component for determining from the indication of compression status if the block of data needs compressing for storage.

The compression indication determining component may be for forwarding the block of data to a compression engine, if it is determined that a block of data needs compressing. The compression indication determining component may be for passing the block of data through a compression engine or bypassing the compression engine, if it is determined that a block of data does not need compressing.

The system may include a compression indication setting component for an application to provide an indication of compression status for a file.

The system may include: a file system layer of an operating system including a compression indication forwarding component for forwarding a received indication of compression status for a file to a block layer.

The system may include: a block layer with a block compression indication forwarding component for forwarding blocks of data for storage with an indication of compression status to a hardware device driver for transmitting to a storage system.

A compression engine of a storage controller may includes metadata at which the indication of compression status of a block of data is recorded.

According to a third aspect of the present invention there is provided a computer program product for reducing workload on a storage system, the computer program product comprising a computer readable storage medium having computer-readable program code embodied therewith, the computer-readable program code configures to: receive a block of data at a storage system together with an indication of compression status of the block of data and determine from the indication of compression status if the block of data needs compressing for storage.

According to a fourth aspect of the present invention there is provided a method substantially as described with reference to the figures.

According to an fifth aspect of the present invention there is provided a system substantially as described with reference to the figures.

The described aspects of the invention provide the advantage of reducing the load on a compression engine by avoiding compression of blocks of data that will gain little benefit from being compressed.

BRIEF DESCRIPTION OF THE DRAWINGS

The subject matter regarded as the invention is particularly pointed out and distinctly claimed in the concluding portion of the specification. The invention, both as to organization and method of operation, together with objects, features, and advantages thereof may best be understood by reference to the following detailed description when read with the accompanying drawings.

Preferred embodiments of the present invention will now be described, by way of example only, with reference to the following drawings in which: Figures IA to IC are block diagrams showing the use of a storage controller as known

in the prior art;

Figure 2 isa block diagram showing the use of a storage eontroHer in accordance with the present invention; Figure 3 is a flow diagram of a method of storing data as known in the prior art; Figure 4 is a flow diagram of an example embodiment of a method of storing data in accordance with the present invention; Figure 5 is block diagram of an example embodiment of a system in accordance with the present invention; and Figure 6 is a block diagram of an embodiment of a computer system in which the present invention may be implemented.

DETAILED DESCRIPTION OF THE DRAWINGS

It will be appreciated that for simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, the dimensions of some of the elements may be exaggerated relative to other elements for clarity. Further, where considered appropriate, reference numbers may be repeated among the figures to indicate corresponding or analogous features.

In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the present invention.

Method and system are provided for reducing the consumption of automated storage system compression resource in situations in which little or no compression is achievable or necessary.

A growing number of files in enterprise applications have little to no benefit from compression since they are already compressed data. This category may include video and audio formats and sofiwarc compressed files such as zips.

The described method and system add an indication of compression status of a block of data before sending to a storage system. The indication of compression status may relate to multiple blocks of data.

The indication of compression status may indicate that a block of data is compressed and does not need further compression. The indication of compression status may indicate that a block of data is un-compressed and needs compressing. In one embodiment, each block of data may have an indication of whether it is compressed or not. In another embodiment, only blocks of data that need compressing may have an indication. In a further embodiment, only blocks of data that do not need compressing may have an indication.

Referring to Figures IA to lB. block diagrams show transmission of data blocks for storage

as known in the prior art.

Figure 1A shows a storage system 100 in which a storage controller 110 provides storage across multiple disks 111-113 for a number of hosts 101-103 over a network.

As shown in Figure 1 B, the storage controller 110 may use a compression engine 120 and may compress blocks of data 131-132 to result in compressed blocks 141-142 for storage in order to minimise disk space used.

As shown in Figure IC, the compression engine 120 may expend wasted effort attempting to compress blocks of data 151-152 which are already compressed.

Referring to Figure 2, a storage system 200 in shown as an illustration of the described system. A host 201 may send blocks of data 241, 242 to be stored by a storage controller 210 across multiple disks 211-213.

The storage controller 210 may include a compression engine 220 as in the prior art for compressing the blocks of data 241, 242 to form compressed blocks of data 26lfor storage.

In the described system, the blocks of data 241, 242 may include indications of their compression status in the form of flags 251,252. An uncompressed block of data 24! may have an uncompressed flag 251; whereas, a block of data 242 which is already of compressed form may have a compressed flag 252. In one embodiment, only compressed or un-compressed data may be flagged.

The storage controller 210 may include a compression determining component 230 for taking note of the flags 251,252 and processing the blocks of data 241,242 accordingly. This may involve sending un-compressed blocks of data 241 to the compression engine 220 whilst allowing already compressed blocks of data 242 to pass through or bypass the compression engine 220.

In one embodiment, the compression determining component 230 may be provided as part of the compression engine 220.

The output of the storage controller 210 would be a compressed block of data 261 corresponding to the first block of data 241, but the second, already compressed, block of data 242 would remain the same for storage.

Referring to Figure 3, a flow diagram 300 shows a method of storing data in a storage system asknowinthepriorart An application may open a file for writing 301 (an open system call). The file system of the operating system may record 302 information about the file and return a handle to that file.

The application may write 303 to that file handle (a write system call).

The file system may send 304 data to be stored as blocks of data to a hardware device driver of the operating system. The hardware device driver may then transmit 305 blocks of data across a network to a storagc controller of a storage system.

The storage controller's compression engine may attempt to compress 306 the blocks of data.

The compressed blocks may then be written 307 to disk.

Referring to Figure 4, a flow diagram 400 shows an embodiment of the described method of reducing workload on a storage system. Some of the steps are the same as in the prior art method shown in Figure 3. The new steps are shown with a bold outline in Figure 4.

An application may open a file for writing 401 (an open system call). The file system of the operating system may record 402 information about the file and return a handle to that file.

In the described method, the application may indicate 403 to the file system whether or not this file is compressed. This may be done by setting a flag for the file. The application may write 404 to the file handle (a write system call) including the indication of compression status.

The file system may send 405 data blocks to be stored via a block layer to a hardware device driver along with the indication of compression status. The hardware device driver may transmit 406 data blocks across a network to a storage controller of a storage system along with the indication of compression status. A single indication of compression status may be sent for multiple blocks of data, for example, as a command to be applied to a number of blocks following the command.

The storage controller may determine 407 from the indication of compression status if a data block is compressed. If it is not compressed, the storage controller's compression engine may compress 408 the data block. Any data blocks determined as already being compressed and any data blocks compressed by the compression engine may then be written 409 to disk.

The storage controller's compression engine may add a flag to it's own metadata signaling that a block of data which did not need compressing does not need to be dc-compressed on access.

On read, this metadata may be checked by the compression engine as part of its normal function. If this flag is asserted for a given data block, the decompression step may be skipped Referring to Figure 5, a block diagram shows an example embodiment of the described system 500.

S

A host 510 may include a number of applications 520 and an operating system 530. The operating system 530 may include a tile system layer 531, a block layer 534, and a hardware device driver 536 The host 510 may use a storage system for storing data across multiple disks (not shown).

The host 510 may connect to the storage system via a network. The storage system may include a storage controller 540 for receiving and processing storage requests from the host 510, as well as other hosts using the storage system.

The described system may be implemented by modifications to the file system layer 531 and the block layer 534 interface in the operating system 530 kernel, and to the Small Computer System Interface (SCSI) protocol or other protocol used for storage commands, to enable the underlying block storage system to identi data blocks already containing compressed data.

Additional semantics may be provided in the file system layer 531, to allow end user applications 510 to tag file data as compressed. This may be stored in the metadata of the file system, for example, in a file header.

The described system may include a compression indication setting component 521 which may be used by applications 520 to set an indication of compression status for a file to be written to storage. For example, an application may wish to write a file in the form of video or audio formats or software compressed files such as zip files and may flag such as file with a compressed status. The compression indication setting component 521 may enable an application 510 to set a flag for a file indicating whether it is compressed or not.

The file system layer 531 of the operating system 530 may include a data sending component 532 for sending data via a block layer 534 to a hardware device driver 536 for transmission for storing.

The data sending component 532 of the file system layer 531 may include a compression indication forwarding component 533 for forwarding the indication of compression status associated with a file received from an application 520. The compression indication forwarding component 533 may convert the indication of a file to a per block basis to be processed by the block layer 534. The block layer 534 may include a block compression indication forwarding component 535 for forwarding the indication of compression on a per block basis.

The hardware device driver 536 of the operating system 530 may include a data transmitting component 537 for transmitting data to a storage controller 540 of an external storage system.

Thc data transmitting componcnt 537 may include a compression indication transmitting component 538 for transmitting the indication of compression status associated with blocks of data rcceivcd from the block layer 534.

The storage controller 540 of a storage system may include a compression indication determining component 541 for evaluating the indication of compression status received with blocks of data for storage.

This indication of compression status allows a compression engine 542 of a storage system to pass any data blocks indicated as compressed straight through without attempting to compress them further, which would have little to no benefit in tenis of space reduction.

Skipping data blocks that already contain compressed data has a number of benefits: the overall Cpu and memory usage required by the compression engine is reduced; overall latency of writes is reduced; and overall latency on reads is reduced since the dc-compression engine can also be skipped for these data blocks.

In one embodiment, when one or more pages of data from a file are flushed to the underlying storage, the indication of compression status may be communicated to the compression engine 542 via a new SCSI command. This would signal to the compression engine 542 that data blocks following are already compressed. The storage controller's compression engine 542 would pass the data blocks through without compression, and add a flag to it's own metadata 543 signaling that this data blocks do not need to be dc-compressed on access.

On read, this metadata 543 is checked by the compression engine 542 as part of its normal function. If this flag is asserted for a given data block, the decompression step is skipped Referring to Figure 6, an exemplary system for implementing aspects of the invention includes a data processing system 600 suitable for storing andior executing program code including at least one processor 601 coupled directly or indirectly to memory elements through a bus system 603. The memory elements may include local memory employed during actual execution of the program code, bulk storage, and cache memories which provide temporary storage of at least some program code in order to reduce the number of times code must be retrieved from bulk storage during execution.

The memory elements may include system memory 602 in the form of read only memory (ROM) 604 and random access memory (RAM) 605. A basic input/output system (BIOS) 606 may be stored in ROM 604. System software 607 may be stored in RAM 605 including operating system software 608. Software applications 610 may also be stored in RAM 605.

The system 600 may also include a primary storage means 611 such as a magnetic bath disk drive and secondary storage means 612 such as a magnetic disc drive and an optical disc drive. The drives and their associated computer-readable media provide non-volatile storage of computer-executable instructions, data structures, program modules and other data for the system 600. Software applications may be stored on the primary and secondary storage means6ll,6l2aswellasthesystemmemory602.

The computing system 600 may operate in a networked environment using logical connections to onc or more remote computers via a network adapter 616.

Input/output devices 613 may be coupled to the system either direetly or through intervening I/O controllers. A user may enter commands and information into the system 600 through input devices such as a keyboard, pointing device, or other input devices (for example, microphone, joy stick, game pad, satellite dish, scanner, or the like). Output devices may include speakers, printers, cte. A display device 614 is also connected to system bus 603 via an interfhce, such as video adapter 615.

The invention can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment containing both hardware and software elements. In a preferred embodiment, the invention is implemented in software, which includes but is not limited to firmware, resident software, microcode, etc. The invention can take the form of a computer program product aeeessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any rnstruction execution system. For the purposes of this description, a computer usable or computer readable medium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus or device.

The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read only memory (ROM), a rigid magnetic disk and an optical disk. Current examples of optical disks include compact disk read only memory (CD-ROM), compact disk readlwrite (CD-R/W), and DYD.

Improvements and modifications can be made to the foregoing without departing from the scope of the present invention.

Claims

(:LAIMs A method for reducing workload on a storage system, comprising: receiving (406) a block of data at a storage system together with an indication of compression status of the Mock of data; and determining (407) from the indication of compression status if the block of data needs compressing for storage.
2. The method as claimed in claim 1, including: if it is determined that a block of data needs compressing, compressing (408) the block of data at a compression engine.
3. The method as claimed in claim 1, including: if it is determined that a block of data does not need compressing, passing the block of 1 5 data through a compression engine without compressing or bypassing the compression engine.
4. The method as claimed in any one of claims 1 to 3, wherein the compression status is compressed or un-compressed.
5. The method as claimed in any one of the preceding claims, wherein the indication of compression status of a block of data is provided as a SCSI command.
6. The method as claimed in any one of the preceding claims, wherein a single indication of compression status is applied to multiple blocks of data.
7. The method as claimed in any one of the preceding claims, including: an application opening (403) a file for writing and the application providing an indication of compression status of the file to a file system.
8. The method as claimed in claim 7, wherein the indication of compression status ofa file is a flag indicating file is compressed or un-compressed.
9. The method as claimed in any one of the preceding claims, including: a file system forwarding (405) a received indication of compression status for a file as an indication of compression status of blocks of data of the file to a hardware device driver for transmitting to a storage system.
10. The method as claimed in any one of the preceding claims, including: recording the indication of compression status for a block of data at metadata of a compression engine of a storage controller.
11. The method as claimed in claim 10, including: skipping dc-compression of a block of data on read if the metadata of the compression engine indicates that the block of data was not compressed on write.
12. A system for reducing Drkload on a storage system comprising: a storage controller (540) enabled to receive a block of data for storage including an indication of compression status of the block of data; a compression indication determining component (541) for determining from the indication of compression status if the block of data needs compressing for storage.
13. The system as claimed in claim 12, wherein the compression indication determining component (541) is for forwarding the block of data to a compression engine (542), if it is determined that a block of data needs compressing.
14. The system as claimed in claim 12, wherein the compression indication determining component (541) is for passing the block of data through a compression engine (542) or bypassing the compression engine (542), if it is determined that a block of data does not need compressing.
15. The system as claimed in any one of claims 12 to 14, wherein the compression status is compressed or un-compressed.
16. The system as claimed in any one of claims 12 to 15, wherein the indication of compression status of a block of data is provided as a SCSI command.
17. The system as claimed in any one of claims 12 to 16, wherein a single indication of compression status is applied to multiple blocks of data.
18. The system as claimed in any one of claims 12 to 17, including: compression indication setting component (521) for an application (520) to provide an indication of compression status for a file.
19. The system as claimed in any one of claims 12 to 18, wherein the indication of compression status for a file is a flag indicating the file is compressed or un-compressed.
20. The system as claimed in any one of claims 12 to 19, including: a file system layer (531) of an operating system (530) including a compression indication forwarding component (533) for forwarding a received indication of compression status for blocks of data of a file to a block layer 534).
21. The system as claimed in claim 20, including: a block layer (534) with a block compression indication forwarding component (535) for forwarding blocks of data for storage with an indication of compression status to a hardware device driver (536) for transmitting to a storage system.
22. The system as claimed in any one of claims 12 to 21, wherein a compression engine (542) of a storage controller (540) includes metadata at which the indication of compression status of a block of data is recorded.
23. A computer program product for reducing workload on a storage system, the computer program product comprising a computer readable storage medium having computer-readable program code embodied therewith, the computer-readable program code configures to: receive (406) a block of data at a storage system together with an indication of compression status of the block of data; and determine (407) from the indication of compression status if the block of data needs compressing for storage.
24. A method substantially as described with reference to the figures.
25. A system substantially as described with reference to the figures.