GB2402766A - Storage Device with Optimal Compression Management Mechanism - Google Patents

Storage Device with Optimal Compression Management Mechanism Download PDF

Info

Publication number
GB2402766A
GB2402766A GB0321395A GB0321395A GB2402766A GB 2402766 A GB2402766 A GB 2402766A GB 0321395 A GB0321395 A GB 0321395A GB 0321395 A GB0321395 A GB 0321395A GB 2402766 A GB2402766 A GB 2402766A
Authority
GB
United Kingdom
Prior art keywords
data
compression
storage device
solid
storage medium
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
GB0321395A
Other versions
GB0321395D0 (en
Inventor
Chia-Li Chen
Hsiang-An Hsieh
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Carry Computer Engineering Co Ltd
Original Assignee
Carry Computer Engineering Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Carry Computer Engineering Co Ltd filed Critical Carry Computer Engineering Co Ltd
Publication of GB0321395D0 publication Critical patent/GB0321395D0/en
Publication of GB2402766A publication Critical patent/GB2402766A/en
Withdrawn legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B20/00—Signal processing not specific to the method of recording or reproducing; Circuits therefor
    • G11B20/10—Digital recording or reproducing
    • G11B20/12—Formatting, e.g. arrangement of data block or words on the record carriers
    • G11B20/1217—Formatting, e.g. arrangement of data block or words on the record carriers on discs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608—Saving storage space on storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0673—Single storage device
    • G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B20/00—Signal processing not specific to the method of recording or reproducing; Circuits therefor
    • G11B20/00007—Time or data compression or expansion
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B20/00—Signal processing not specific to the method of recording or reproducing; Circuits therefor
    • G11B20/10—Digital recording or reproducing
    • G11B20/12—Formatting, e.g. arrangement of data block or words on the record carriers
    • G11B2020/1264—Formatting, e.g. arrangement of data block or words on the record carriers wherein the formatting concerns a specific kind of data
    • G11B2020/1265—Control data, system data or management information, i.e. data used to access or process user data
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Signal Processing (AREA)
  • Memory System Of A Hierarchy Structure (AREA)

Abstract

The present invention discloses a storage device with an optimal compression management mechanism, mainly comprising a controller and at least a solid-state storage medium. Said controller has at least a system interface that may be connected to an external system end, a microprocessor processing system instructions, and a memory interface communicating with said solid-state storage medium. Said controller detects and distinguishes the type of raw data to be stored, chooses the compression algorithm and associated parameters most suitable for the raw data, compresses the raw data into minimized data at the optimal compression ratio and writes the compressed data onto said solid-state storage medium. In that way, the storage device may boost the storage capacity of said solid-state storage medium without increasing the physical memory of the solid-state storage medium.

Description

Storage Device with Optimal Compression Management Mechanism The present
invention relates to a storage device with an optimal compression management mechanism, in particular to a storage device that may choose the most suitable compression algorithm and automatically compress data to be stored in an optimal way.
Currently, solid-state storage media (e.g., flash memory) utilizing silicon wafers for memory have become more and more popular. Due to the benefits of silicon waters such as low power consumption, high reliability, high storage capacity, and high access speed, they are widely used in mini memory cards (e.g., OF cards, MS cards, SD cards, MMC cards, and SM cards) and USB U disks. Besides a solid-state storage medium, such a storage device A (see Fig.6) has a controller A1 in it. Said controller A1 has a system interface All that may be connected to an external system end B. a microprocessor A12 processing system instructions, and a memory interface A13 communicating with the solid-state storage medium A2. Thus said controller A1 may write data from the system end B onto said solid-state storage medium A2 or read data stored on said solid-state storage medium A2.
However, whether for memory cards or for USB portable disks, production costs and sales prices depend on the capacity of their embedded solidstate storage media, for example, there are 64MB, 128MB, and 256MB storage media currently available. The cost and sales price are proportionate to the capacity of the embedded storage media, i.e., the higher the capacity of the embedded storage medium is, the higher the price of the storage device is.
However, as hardware manufacturing technology has developed to a certain degree, solid-state storage media have encountered the same embarrassment as today's CD-R disks, i.e., the storage capacity per unit area of a silicon wafer cannot be increased any further. Though emerging nanometre technologies may further reduce the granularity of storage space to increase storage capacity, these technologies are in the primary stages of development and cannot currently be used to overcome the above embarrassment.
In practice, there is a way to solve the above problem, i.e., devise another socket at an appropriate position on the body of said storage device (memory card or USB portable disk) to insert an external memory card to expand the storage capacity of the memory device. Though that may solve the problem of insufficient storage capacity, it requires additional external memory cards, which lead to higher costs.
If an additional solid-state storage medium or external storage device is to be avoided, necessary compression measures have to be taken to reduce the storage volume required for the raw data, in order to boost the data storage capacity of existing solid-state storage media. However, data compression could only be done on computers until recently. That is to say, on computers, files may be compressed using appropriate compression software (e.g., Winrar, WinZip, etc.) and stored on internal storage devices (e.g., a hard disk) or external storage devices (e.g., CDs, diskettes, portable disks, or electronic memory cards), in order to save storage space and achieve higher transmission rates.
Therefore, there is an urgent need to develop a storage device that has not only a storage function, but also data compression capability to compress raw data before storage. Preferably, such a storage device can also choose the optimal compression algorithm needed to "minimize" raw data, in order to boost the storage capacity of existing storage media significantly without adding additional storage media or external storage devices.
The main purpose of the invention is to provide a storage device with an optimal compression management mechanism, which may boost the data storage capacity of the solid-state storage medium by compressing raw data to reduce data volume significantly using the internal compression mechanism. In that way, the storage device helps to increase data storage capacity, decrease product costs, and improve data access speed.
Another purpose of the invention is to provide a storage device with an optimal storage management mechanism, which may choose the optimal compression algorithm automatically to minimize the volume of raw data and boost the data storage capacity of the solid-state storage medium significantly.
To attain the above and other purposes and efficacies, the storage device with an optimal compression management mechanism mainly comprises a controller and at least a solid-state storage medium, wherein said controller has an internal system interface that may be connected to a system end, a processor that processes system instructions, and a memory interface that communicates with said solid-state storage medium. Said controller features: a data compression/decompression module between the system interface and the memory interface, wherein the data compression/decompression module may compress the raw data to be stored at an appropriate compression ratio and then store the compressed data onto the solid-state storage medium.
To attain an optimal compression effect, said data compression/decompression module has an internal data compression circuit and a plurality of data compression algorithms that are used with said data compression circuit. Said microprocessor distinguishes the type of raw data transmitted via the system interface and chooses the optimal data compression algorithm, instructs the data compression circuit to compress the raw data with said optimal data compression algorithm to minimize data volume, and stores the compressed data onto said solid- state storage medium via the memory interface.
To understand the above and other purposes, features, and benefits of the invention better, the invention is described in the following examples of preferred embodiments, with reference to the attached drawings.
Figure 1 is a sketch map of the circuit of a preferred embodiment in the present invention.
Figure 2 shows the content of the solid-state storage medium in Figure 1 in an uncompressed state.
Figure 3 is a sketch map of the circuit of another preferred embodiment in the present invention.
Figure 4 shows the content of the solid-state storage medium in Figure 3 in a compressed state.
Figure 5A shows the compression process of the embodiment shown in Figure 3.
Figure 5B shows the decompression process of the embodiment shown in Figure 3.
Figure 6 is a sketch map of a common circuit.
Figure 1 shows a sketch map of the internal circuit of said storage device with an optimal compression mechanism. The storage device 1 may be a memory card that may be widely used in various portable digital products, a USB U-disk that may be used in PCs, or a storage device with a solid-state storage medium (i.e., Flash Memory) under development.
The storage device 1 comprises a controller 10 and at least a solid-state storage medium 20; said controller 10 comprises a system interface 104, a microprocessor 102, and a memory interface 106. Said system interface 104 may be connected to an external system end 2 (i.e., a portable digital product or a PC); said memory interface 106 communicates with said solidstate storage medium 20; said microprocessor 102 is wired to said system interface 104 and said memory interface 106.
To boost the storage capacity of the solid-state storage medium, a data compression/decompression module 108 is located between the system interface 104 and the memory interface 106 in said storage device 1 and is wired to said system interface 104 and said memory interface 106. In addition, to adapt to the different transmission speeds of high speed and low speed interfaces, there is a first data cache 110 and a second data cache 120 in the controller. Said first data cache 110 is wired to said data compression/decompression module 108 and said system interface 104 and serves as the front-end cache of the data compression/decompression module 108; said second data cache 120 is wired to said data compression/decompression module 108 and said memory interface 106 and serves as the rear-end cache of the data compression/decompression module 108. Said caches 110 and 120 are used to store data temporarily.
When raw data is to be stored in the solid-state storage medium 20 in said storage device 1, the system interface 104 receives raw data transmitted from the external system end 2, the microprocessor 102 compresses the raw data at an appropriate compression ratio through the compression mechanism of the data compression/decompression module 108, and then stores the compressed data into said solid-state storage medium via the memory interface 106. Thus the invention enables the solid-state storage medium 20 to store data at a volume many times that of the raw data.
In the present invention, the system interface stores the raw data received in the first data cache 110 before the data is transmitted for compression. The data compression/decompression module 108 then retrieves raw data from the first data cache 110 at a certain transmission speed, compresses the raw data, and transfers the compressed data to the second data cache 120. Under the control of the microprocessor 102, the compressed data in the second data cache 120 is stored in the solid-state storage medium 20 via the memory interface 106.
During decompression, the data compression/decompression module 108 retrieves the compressed data from the solid-state storage medium 20 via the memory interface 106 and decompresses it. The second data cache 120 stores the compressed data to be decompressed, and the first data cache 110 stores the decompressed raw data, which is transferred to the external system end 2 via the system interface 104.
In reference to Figure 2, data transferred from the external system end 2 and stored in the solid-state storage medium 20 comprises not only raw data but also Control Information for the raw data. The solid-state storage medium 20 comprises several data storage blocks 4. In the present embodiment, it is supposed that each data storage block 4 stores 528 data bits. Each data storage block 4 comprises a data storage area 42 (occupying 512 data bits, similar to a sector of a hard disk) and a Control Information storage area 44 (occupying 16 bits). The Control Information in the Control Information storage area 44 comprises a Status Flag 441, an Error Correction Code 442, a Logical Address Record 443, and a reserved area as the reserved area 444 shown in Figure 2. In an optimal compression/decompression process, the present invention utilizes said reserved area 444 to store the compression record.
Figure1 - Figure 3 describe the optimal compression technology used in the present invention.
As shown in Figure 1, the data compression/decompression module 108 has a data compression circuit 1082 and a plurality of algorithm definitions 1083a 1083n and parameter lists 1084a 1084n used with said data compression circuit 1082. Each algorithm definition defines a compression/decompression algorithm, which may be used with different parameter lists in order to minimize the volume of the raw data through combinations of the compression algorithms.
The microprocessor 102 distinguishes the type of raw data transferred via the system interface 104 to determine the optimal compression combination.
The microprocessor 102 distinguishes the type of raw data by detecting the distribution of binary bits in the raw data, i.e., it determines the optimal algorithm according to the proportion, distribution, and repetition of "O" and "1" bits in raw data. When the data type has been distinguished, the microprocessor chooses the most suitable combination between the algorithm definition group 1083 and the parameter list group 1084 and hands it over to the data compression circuit 1082 to compress the raw data into the minimized data volume, and store the compressed data into the second data cache 120. As the memory interface 106 is triggered and stores the compressed data in the solid- state storage medium 20, the indexes of the corresponding optimal algorithm definition and parameter list are also stored in the solid-state storage medium 20. Compressed data is stored in the data storage area 42 in the data storage blocks 4, while the indexes are stored in the reserved area 444 in the data storage blocks 4.
In addition, there is a data decompression circuit 1085 in the data compression/decompression module 1082. When the external system end 2 retrieves data stored in the storage device 1, said data decompression circuit 1085 is triggered by the microprocessor 102, reads the indexes stored in the reserved areas 444 in the solid-state storage media via the memory interface 106, decompresses the compressed data into raw data according to the algorithm and parameter list referred to by the indexes, and then transfers the raw data to the external system end via the system interface 104.
Figure 4 and Figure 5A show flowcharts of the optimal compression management mechanism used in a preferred embodiment of the invention.
When raw data transferred from the external system end is loaded into the first data cache 110, the microprocessor 102 detects the distribution of binary bits in the raw data and then chooses the optimal combination between the algorithm definition 1083 and the parameter lists 1084. In the present embodiment, the first algorithm definition 1083a and the second parameter list 1084b are selected to constitute the compression combination (1, 2); next, the data compression circuit 1082 in the data compression/decompression module 108 is triggered and the compression combination (1, 2) is handed over to the compression circuit 1082 as the basis for raw data compression. In the present embodiment, the compression combination (1, 2) indicates that the raw data is compressed at a 1/2 compression ratio i.e., suppose the raw data occupied 512 bytes, the compressed data would only occupy 256 bytes. Thus the data storage area 42 in a data storage block 4, which could only store a batch of raw data originally, may now store 2 batches of compressed data. Thus the storage capacity of the solid-state storage medium is doubled.
As the compressed data is stored, two indexes (1, 2) are added to the reserved area 444 of the Control Information storage area 44, while the Status Flag 441, Error Correction Code 442, and Logical Address Record 443 remain constant. The first and second number in parentheses (i.e., (1, 2)) indicate the first algorithm definition and second parameter list, respectively. Therefore, the indexes (1, 2) may facilitate data decompression.
Figure 4 and Figure 5B show flowcharts of an optimal decompression management mechanism used in a preferred embodiment in the present invention.
When the controller receives a data retrieval request from the system end, it locates the logical address of the data according to the Logical Address Record 443 of the data and the corresponding data storage block 4 in the solid state storage medium, and then reads the data stored in the data storage block 4 in the solid-state storage medium to the second data cache 120. Next, the microprocessor triggers the data decompression circuit 1085 to read the index (1, 2) stored in the reserve area 444 in the same data storage block 4. The data decompression circuit then reads the first algorithm definition and the second parameter and decompresses the compressed data into raw data and transfers the raw data to the first data cache 110. Finally, the raw data in the first data cache 110 is transferred to the external system end 2.
In conclusion, the present invention is disclosed as above with examples of preferred embodiments. However, it is noted that the above embodiments I shall not constitute any limitation to the invention. Any person familiar with the technologies may carry out modifications or embellishments to the embodiments without deviating from the concept and scope of the invention. Therefore, the scope of the invention is solely defined by the attached claims. Any embodiment implemented with equivalent modifications or embellishments to the invention (e.g., replacing the microprocessor distinguishing the type of raw data with a circuit, such as a data compression circuit) shall fall within the scope of the invention. I

Claims (7)

  1. What is claimed is: 1. A storage device with an optimal compression
    management mechanism, comprising a controller and at least a solid-state storage medium; said controller has an internal system interface that may be connected to an external system end, a microprocessor that processes system instructions, I and a memory interface that communicates with said solid-state storage medium; wherein said storage device features: a data compression/decompression module located between said system interface and said memory interface; said data compression/decompression module is wired to said microprocessor and has a data compression circuit and a plurality of algorithm definitions and parameter lists that are used with said data compression circuit; said microprocessor distinguishes the type of raw data transferred from the I system interface and chooses the most suitable compression combination from said algorithm definitions and parameters; said data compression circuit I compresses the raw data into a minimized data volume according to said compression combination and stores the compressed data onto said solid state storage medium.
  2. 2. The storage device with an optimal compression management mechanism as in claim 1, wherein said solid-state storage medium stores indexes referring to the optimal algorithm definition and parameter list.
  3. 3. The storage device with an optimal compression management mechanism as in claim 2, wherein said data compression/decompression module further has a data decompression circuit, which is triggered by the microprocessor to read said indexes from the solid-state storage medium and decompress the compressed data into raw data according to the algorithm definition and I parameter list referred by the indexes.
  4. 4. The storage device with an optimal compression management mechanism as in claim 1, wherein said storage device has the first data cache wired to I said system interface, said microprocessor, and said data compression/decompression module.
  5. 5. The storage device with an optimal compression management mechanism as in claim 1, wherein said microprocessor has the second data cache wired to said memory interface, said microprocessor, and said data compression/decompression module.
  6. 6. The storage device with an optimal compression management mechanism as in claim 1, wherein said data compression/decompression module is located in said controller.
  7. 7. The storage device with an optimal compression management mechanism as in claim 1, wherein said microprocessor distinguishes the type of raw data on the basis of the distribution of binary bits in the raw data.
GB0321395A 2003-06-05 2003-09-12 Storage Device with Optimal Compression Management Mechanism Withdrawn GB2402766A (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
TW92115319 2003-06-05

Publications (2)

Publication Number Publication Date
GB0321395D0 GB0321395D0 (en) 2003-10-15
GB2402766A true GB2402766A (en) 2004-12-15

Family

ID=29247726

Family Applications (1)

Application Number Title Priority Date Filing Date
GB0321395A Withdrawn GB2402766A (en) 2003-06-05 2003-09-12 Storage Device with Optimal Compression Management Mechanism

Country Status (1)

Country Link
GB (1) GB2402766A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
NO20064962L (en) * 2006-10-30 2008-05-02 Visitech As Procedure for selecting the compression algorithm for data compression
US8595199B2 (en) 2012-01-06 2013-11-26 International Business Machines Corporation Real-time selection of compression operations

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5467087A (en) * 1992-12-18 1995-11-14 Apple Computer, Inc. High speed lossless data compression system
US20020097172A1 (en) * 1998-12-11 2002-07-25 Fallon James J. Content independent data compression method and system
WO2003102788A1 (en) * 2002-05-13 2003-12-11 Trek 2000 International Ltd. System and apparatus for compressing and decompressing data stored to a portable data storage device

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5467087A (en) * 1992-12-18 1995-11-14 Apple Computer, Inc. High speed lossless data compression system
US20020097172A1 (en) * 1998-12-11 2002-07-25 Fallon James J. Content independent data compression method and system
WO2003102788A1 (en) * 2002-05-13 2003-12-11 Trek 2000 International Ltd. System and apparatus for compressing and decompressing data stored to a portable data storage device

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
NO20064962L (en) * 2006-10-30 2008-05-02 Visitech As Procedure for selecting the compression algorithm for data compression
US8595199B2 (en) 2012-01-06 2013-11-26 International Business Machines Corporation Real-time selection of compression operations
US8903781B2 (en) 2012-01-06 2014-12-02 International Business Machines Corporation Real-time selection of compression operations

Also Published As

Publication number Publication date
GB0321395D0 (en) 2003-10-15

Similar Documents

Publication Publication Date Title
US20040250009A1 (en) Storage device with optimal compression management mechanism
KR101150162B1 (en) Symbol frequency leveling in a data storage system
US5640349A (en) Flash memory system
US6883079B1 (en) Method and apparatus for using data compression as a means of increasing buffer bandwidth
US7606954B2 (en) Data storage using compression
JP2006518897A (en) Multi-protocol memory card
US10678443B2 (en) Method and system for high-density converged storage via memory bus
US10866746B2 (en) Memory addressing methods and associated controller, memory device and host
US20180210858A1 (en) Bus encoding using on-chip memory
CN109582508B (en) Data backup and recovery method for NVDIMM, NVDIMM controller and NVDIMM
US7689807B2 (en) Mass storage device, mass storage controller and methods for use therewith
WO2011048400A1 (en) Memory interface compression
EP3040867B1 (en) A method for addressing a memory card, a system using a memory card, and a memory card
US7062603B2 (en) External storage device for selectively storing data between first and second recording media
US6266671B1 (en) Data storage apparatus, method, and medium with variable data storage structure
GB2402766A (en) Storage Device with Optimal Compression Management Mechanism
US20040250010A1 (en) Storage device available for increasing storage capacity
JP3100146U (en) Storage device with compression management mechanism
CN209803776U (en) NVDIMM controller and NVDIMM
KR200335514Y1 (en) Storage device comprising optimization compression management mechanism
EP0784325B1 (en) Flash memory system
US11847325B2 (en) Semiconductor integrated apparatus, operating method thereof and data processing apparatus including the same
TWI841113B (en) Memory addressing methods and associated controller
US20240232010A9 (en) Storage device for providing event data and operation method of storage device
US20240134745A1 (en) Storage device for providing event data and operation method of storage device

Legal Events

Date Code Title Description
WAP Application withdrawn, taken to be withdrawn or refused ** after publication under section 16(1)