CN103957013A - Gzip hardware compression method - Google Patents

Abstract

The invention discloses a Gzip hardware compression method. A DMA module and a Gzip compression module are involved, after DMA is configured, the DMA sends request data information to an external CPU, the external CPU selects data from a corresponding address and sends the data to the DMA, the DMA sends the received data to the Gzip compression module, and after data compression, the Gzip module launches a writing request to the DMA, and the DMA transmits data to a PC. By means of the method, CPU time can be effectively saved, the data rate is improved, compression efficiency is improved, and the method has great practical value.

Description

A kind of Gzip hardware-compressed method

Technical field

The present invention relates to the technology of data channel in data compression.Relate in particular to a kind of Gzip hardware-compressed system, method.

Background technology

Development along with modern science and technology, particularly in the research of cloud computing massesization more and more, the transmission of large data more and more becomes focus with processing, a kind of effective method is to use compress technique after data compression, transmitting, and traditional compression is all software realization, the maximum problem that software is realized is wasted CPU precious resources exactly.And solve this class problem effective method, be with hardware resource, to realize the function of Software Compression, this can replace CPU to carry out data processing, thereby makes CPU process other programs.

At present, data compression is divided into two kinds: lossy compression method and Lossless Compression.Lossy compression method comprises pulse code modulation, predictive coding, transition coding, wavelet coding, and Lossless Compression comprises LZW, LZO, LZMA, Gzip.Although lossy compression method is initial data, significantly compressed, the data and the initial data that decompress are inconsistent, unrepairable, and there is loss in data, and Lossless Compression can finely make up this shortcoming of lossy compression method.Lossless Compression is reversible, and data are not lost in compression process.Wherein Gzip algorithm compression speed and and compression ratio all relatively good, still, when FPGA tests, run into the problem of transfer of data on the memory of PC and FPGA.Conventional data transfer mode comprises program control, interrupts control, DMA(Direct Memory Access) mode.It is to carry out under the control of CPU (Central Processing Unit) that general data transmit (comprising program controlled mode, interrupt mode), be that CPU sends both sides' address and control information to address bus and control bus, then the data that will transmit are delivered to data/address bus and are transferred to memory cell or port for peripheral equipment through the accumulator of CPU.And each limited bits (being no more than at most cpu data line width) transmitting, its data transfer rate is more much lower than the storage speed of internal memory.It is very uneconomic through CPU, carrying out large batch of data transmission, is especially not suitable for requiring the scene of transmitting data in real time.And DMA technology only needs CPU to complete that it(?) just can be automatically transmit data according to configuration information to its configuration, thereby CPU is freed, process other processes.

The characteristic that the present invention is directed to Gzip hardware-compressed has customized the method for transfer of data, thereby the efficiency of Gzip hardware-compressed is got a promotion.

Summary of the invention

Technical problem to be solved by this invention is for the defect in background technology, and a kind of effective Gzip hardware-compressed method of saving the CPU time, data rate and raising compression efficiency being provided is provided.

The present invention is for solving the problems of the technologies described above by the following technical solutions:

A Gzip hardware-compressed method, comprises dma module and Gzip compression module, and concrete steps are as follows:

Step 1), is configured the register of the register of dma module and Gzip compression module;

Step 2), dma module sends data request packet to outer CPU;

Step 3), dma module receives the packet that CPU sends, and resolves this packet;

Step 4), the transfer of data of the needs compression that dma module obtains resolution data bag is to Gzip compression module;

Step 5), Gzip compression module sends write data requests to dma module after the data of needs compression are compressed;

Step 6), dma module receives the data of compression, and is packaged into DMA write request bag and sends to outer CPU.

As the further prioritization scheme of a kind of Gzip hardware-compressed of the present invention method, the register of dma module described in step 1) comprises the first reseting register, DMA writes register group, DMA read register group, OIER and interrupt mask register, and the register of described Gzip compression module comprises the second reseting register, Gzip compact model register and Gzip compression enable register.

As the further prioritization scheme of a kind of Gzip hardware-compressed of the present invention method, described DMA writes register group and comprises DMA and write status register, DMA writing address register and DMA and write data length register, and described DMA read register group comprises DMA read states register, DMA reads address register and DMA read data length register.

As the further prioritization scheme of a kind of Gzip hardware-compressed of the present invention method, the detailed step described in step 1), the register of dma module being configured is as follows:

Step 1.1), dma module receives the memory write request of outer CPU, and resolving memory write request by the data in memory write request bag and address resolution out, and is write data in corresponding address;

Step 1.2), dma module receives the read request of outer CPU, and the data in memory read request packet are sent to outer CPU, for outer CPU, confirms that memory write request is performed.

As the further prioritization scheme of a kind of Gzip hardware-compressed of the present invention method, described memory write request bag, memory read request packet adopt PCIe protocol format.

As the further prioritization scheme of a kind of Gzip hardware-compressed of the present invention method, step 2) described in dma module concrete steps from data request packet to outer CPU that send be:

Dma module is according to the address in memory write request and data, and the structure PCIe memory literary composition of reading the newspaper, to outer CPU request for data.

The present invention adopts above technical scheme compared with prior art, has following technique effect:

1. not only realize and do not taking data-transformation facility in processor resource situation, and solved Gzip hardware-compressed system data transfer function;

2. meet PCI Express bus protocol.Support is transmitted interruption, mistake interrupt mechanism;

3. pair transmission read request transaction bag can out of orderly reclaim;

4. make Gzip compression module always in running order, improve the efficiency of Gzip hardware-compressed.

Accompanying drawing explanation

Fig. 1 is implementation step of the present invention;

Fig. 2 is second step flow chart of the present invention;

Fig. 3 is data encapsulation form in second step of the present invention;

Fig. 4 is data encapsulation form in the present invention's the 4th step.

Embodiment

Below in conjunction with accompanying drawing, technical scheme of the present invention is described in further detail:

As shown in Figure 1, the invention discloses a kind of Gzip hardware-compressed method, comprise dma module and Gzip compression module, concrete steps are as follows:

The first step, PC is configured DMA internal register and control Gzip compression module register.DMA, receiving the memory write request bag of PC transmission, extracts configuration address and configuration data in write request bag.Then data are write in corresponding address.

Wherein, the register of configuration comprises that reseting register, Gzip compact model register, DMA write register, DMA read register etc.

Second step, the first address of request msg length and request in reading DMA read register, starts to organize DMA read request packet.The form of read request packet is read transaction packet form according to PCIe and is encapsulated.Fig. 2 is data encapsulation process.

First, judge whether read request data length is greater than 128 bytes, if be greater than, carry out burst processing, each sheet bag wraps except last, and the request msg length of all the other sheet bags is 128 bytes.The data length of last sheet bag request is that total data length deducts, and sheet bag number subtracts 1. acquired results and takes advantage of and 128 bytes.

Then, No. tag, application.No. tag, first to file before every transmission sheet bag, when defining tag and can use, starts to prepare encapsulation when anter bag packet header.The every transmission of value of tag is once from adding 1.

Finally, according to the length when the request of anter bag, determine the value of length in packet header, FBE, LBE field.Length represents the length of sheet bag request msg, and bit wide is 10bit; FBE represents first count data effective byte enable bit in packet, and bit wide is 4bit, and whether the corresponding respective byte of each bit is effective; LBE represents last beat of data effective byte enable bit in packet, and bit wide is 4bit, and whether each corresponding respective byte is effective.

Length field is except last sheet bag, and in all the other sheet bags, this field is 32 always.For last sheet bag, total length is removed in 128, the result obtaining.If result is 4 integral multiple, the length of length is that result is removed in 4; If result is not 4 integral multiple, the length of length is that result is removed in 4 value and added 1.

FBE field and LBE are except last sheet bag, and in all the other sheet bags, this field is 4`b1111 always, represent that first count data byte is all effective.For last sheet bag, the value of this field is relevant with this sheet bag request msg length.If this sheet request msg length surpasses 32 bytes, FBE field is 4`b1111, and LBE field is determined according to the low 2bit of data length.If this sheet request msg length is less than 32 bytes, FBE field is determined according to the low 2bit of data length, and LBE field is 4`b0000.

Address field is determined the first address of read data, and DMA internal register is deposited this address, when carrying out burst.First bag Addres field is the value of DMA internal register, and the value of second bag Address adds 128 certainly, the like, until the request of all read tablet bags is all sent completely.

Complete after aforesaid operations, according to Fig. 3 data format, encapsulate.

The 3rd step, receives the bag that completes that PC sends, and this completes bag is that response second step sends the transaction packet request of reading.This packet packet header length is 12 bytes, in the data of second count, in low 32, will occur for the first time data.Tag field in first count is parsed, by this tag value, determine the base address that this bag is deposited.This is due to the corresponding read request of PCIe equipment time not to be sequenced, and this is not order by the bag that completes that causes receiving.Meanwhile, for the data that receive, need to reconfigure and re-send to Gzip compression module.Low 32 of last beat of data are placed on highly 32, when high 32 of beat of data are placed on lowly 32, after having organized, by 64bit data/address bus, send to Gzip compression module.

The 4th step, handles after data at Gzip compression module, to DMA, sends write data requests, and DMA becomes DMA write request bag to send to PC data encapsulation.Detailed process is as follows.

Gzip compression module will be notified DMA after completing data compression, have data to prepare to upload, and now judge that whether DMA is idle.If idle, respond the request of Gzip compression module, reading out data size is write to DMA simultaneously and write in data length register, start read data.When if reading out data length is greater than 128 byte, repeatedly writes transaction packet and send until all data are all sent completely.Except last a slice bag, the data length that all the other sheet bags carry is 128 bytes.The form that each sheet bag is write transaction packet according to DMA encapsulates.As shown in Figure 4.Fmt, the type field write 2`b10,5`b0; TC, TD, EP, Attr, Requester ID, Tag insert fixed value.Below Length, LBE, FBE, Address field are produced and set forth.

Length field is except last sheet bag, and in all the other sheet bags, this field is 32 always.Detailed process is consistent with the 3rd step Length field generation, repeats no more here.

FBE field and LBE are except last sheet bag, and in all the other sheet bags, this field is 4`b1111 always.Detailed process and the 3rd step FBE, LBE, field produce consistent, repeat no more here.

Addess field produces consistent with the 3rd step Address field generation, repeats no more here.

After DMA write request completes, trigger and write interruption, PC responds in this and has no progeny, and reads interrupt register, then gets back to the first step, and so circulation is gone down.

Claims (6)

1. a Gzip hardware-compressed method, is characterized in that, comprises dma module and Gzip compression module, and concrete steps are as follows:

Step 1), is configured the register of the register of dma module and Gzip compression module;

Step 2), dma module sends data request packet to outer CPU;

Step 3), dma module receives the packet that outer CPU sends, and resolves this packet;

Step 4), the transfer of data of the needs compression that dma module obtains resolution data bag is to Gzip compression module;

Step 5), Gzip compression module sends write data requests to dma module after the data of needs compression are compressed;

Step 6), dma module receives the data of compression, and is packaged into DMA write request bag and sends to outer CPU.

2. a kind of Gzip hardware-compressed method according to claim 1, it is characterized in that, the register of dma module described in step 1) comprises the first reseting register, DMA writes register group, DMA read register group, OIER and interrupt mask register, and the register of described Gzip compression module comprises the second reseting register, Gzip compact model register and Gzip compression enable register.

3. a kind of Gzip hardware-compressed method according to claim 2, it is characterized in that, described DMA writes register group and comprises DMA and write status register, DMA writing address register and DMA and write data length register, and described DMA read register group comprises DMA read states register, DMA reads address register and DMA read data length register.

4. a kind of Gzip hardware-compressed method according to claim 2, is characterized in that, the detailed step described in step 1), the register of dma module being configured is as follows:

Step 1.1), dma module receives the memory write request of outer CPU, and resolving memory write request by the data in memory write request bag and address resolution out, and is write data in corresponding address;

Step 1.2), dma module receives the read request of outer CPU, and the data in memory read request packet are sent to outer CPU, for outer CPU, confirms that memory write request is performed.

5. a kind of Gzip hardware-compressed method according to claim 4, is characterized in that, described memory write request bag, memory read request packet adopt PCIe protocol format.

6. a kind of Gzip hardware-compressed method according to claim 4, is characterized in that step 2) described in dma module concrete steps from data request packet to outer CPU that send be:

Dma module is according to the address in memory write request and data, and the structure PCIe memory literary composition of reading the newspaper, to outer CPU request for data.