WO2015070539A1

WO2015070539A1 - Dma-based data compression chip structure and implementation method therefor

Info

Publication number: WO2015070539A1
Application number: PCT/CN2014/071244
Authority: WO
Inventors: 王恩东; 胡雷钧; 李仁刚
Original assignee: 浪潮电子信息产业股份有限公司
Priority date: 2013-11-14
Filing date: 2014-01-23
Publication date: 2015-05-21
Also published as: CN103593437A

Abstract

A DMA-based data compression chip structure and an implementation method therefor. The data compression chip structure comprises a system to device DMA module, a device to system DMA module, a sending-end mediation module, a data storage module interface module and a compression algorithm core module. The DMA-based data compression chip and the DMA technology with matched bandwidth in combination with a plurality of compression algorithm cores are used together to realize the compression processing and storage of large data, thereby increasing the utilization rate of system storage resources.

Description

Data compression chip structure based on DMA and implementation method thereof

Technical field

The present invention relates to the field of computer system design and integrated circuit design, and in particular to a chip structure and an implementation method thereof, and more particularly to a data compression chip structure based on DMA and an implementation method thereof. Background technique

With the rapid development of computer technology, in order to meet the application needs of various fields, high performance and high storage capacity have become the basic characteristics of computer systems. Increasing the storage equipment will make the equipment cost extremely high and occupy a large amount of equipment room area. , the cost is very high, which requires the use of new processing equipment, efficient implementation of big data compression processing. However, on the other hand, it suffers from performance loss and CPU computing resource loss caused by additional data processing. Therefore, DMA (Direct Memory Access) technology that satisfies bandwidth matching can effectively improve system performance, and Save CPU computing resources as much as possible to ensure minimal system performance impact caused by data processing. Compressing the stored data can save more than 1 time of storage device resources. The effect of device saving is different according to different application environments. In an application environment with high data redundancy, storage space savings can reach tens of times. Additional data compression processing will inevitably lead to system performance degradation. Without performance analysis optimization, the performance loss cost is unacceptable. Therefore, the compression chip uses DMA technology to improve the processing efficiency of system data, and the DMA technology uses bandwidth matching design. The method works together with a plurality of compression algorithm cores to complete data compression processing and storage operations. Summary of the invention

In order to solve the problems of the prior art, the present invention provides a data compression chip structure based on DMA and an implementation method thereof, which adopts a data compression chip based on the DMA principle, compresses large data, and adopts a bandwidth matching DMA technology. With multiple compression algorithm cores, it can realize the compression processing and storage of big data together, and improve the utilization of system storage resources. The technical solutions adopted by the present invention are as follows:

A data compression chip structure based on DMA, including system to device DMA module, device to system DMA module, transmitting end arbitration module, data storage module interface module and compression algorithm core module, Among them, the system-to-device DMA module uses descriptor DMA and data DMA to improve system efficiency, that is, descriptor information and transmission data are all DMA transmission; device-to-system DMA module adopts descriptor DMA and data DMA. Improve system efficiency, that is, descriptor information and transmission data are all DMA transmission; the arbitration module at the sending end controls the arbitration mechanism for sending packets by giving priority analysis, judgment, and generation, which is used to solve multi-message transmission and transmission. Conflicts; Data storage module interface module uses high parallel multi-group storage array to achieve data caching; compression algorithm core module uses data path bandwidth matching method to configure multiple compression algorithm cores. A DMA-based data compression chip structure implementation method is to communicate with a system through a PCI-E interface, realize system and device data transmission through DMA technology, respectively design a system to device and device to system DMA engine, and send The end design arbitration logic realizes the efficient transmission of the transmitted data. The descriptor DMA and the data DMA are respectively implemented in the DMA module in two directions to improve the data transmission efficiency, and the received data is connected to the data compression algorithm module through the interface storage module, and the data is connected. The compression module performs compression processing, and the compressed data is then transferred to the system via DMA. The implementation method of the present invention specifically includes the following steps:

A, the system initiates a read request, the data compression chip sends a read request and a read command through its sender, and sends it to the system through the PCI-E port, and at the receiving end, receives descriptor information sent by the system, and the information passes through the system to the device. The descriptor DMA mode controls the transmission, that is, the start and end address information of the descriptor, and is transmitted by DMA, and finally written to the descriptor register, located in the system-to-device descriptor, and the data transmission of the corresponding descriptor information is controlled by the system-to-device DMA engine. ;

B. The received DMA data is transmitted to the data storage interface module through the system-to-device interface, so that the compression algorithm core module performs data compression application, and the data storage module is a highly parallel packet memory array, which can support multiple data compression cores to read in parallel. Write operation

C. The core module of the compression algorithm reads data from the data storage interface module, performs data compression operations, and collects data information for compression control to improve compression efficiency. The data matching information in the data compression process is saved by the matching information module for compression. Module use; D. Determine the core number of the core module of the compression algorithm according to the PCI-E interface, the DMA transmission capability, the data storage interface, and the bandwidth of each interface and module of the compressed core module capability system; the core modules of each compression algorithm are completely independent and parallel work;

E. The compressed data after compression is controlled by the device to the system DMA module to complete the data transmission from the device to the system; the device to system descriptor DMA mode controls the transmission of multiple descriptor information to improve system performance. The transmitted descriptor information is registered in the device to system descriptor register, and is transmitted by the device to the system DMA mode;

F. When the data is transmitted at the transmitting end, the arbitrating module of the transmitting end controls the priority control when multiple message data are transmitted, thereby avoiding the conflict problem when multiple messages are simultaneously transmitted, and finally the data is completed by the PCI-E interface to the system. Send transmission.

During system operation, internal and external interrupts are controlled by the terminal module and transmitted to the system.

In the technical solution of the present invention, the system-to-device DMA feature mainly refers to the compression chip receiving data from the system by means of DMA, including descriptor DMA and data DMA, wherein the data DMA includes a DMA descriptor and a DMA engine, and the description DMA is to transfer the descriptor information of the system by DMA to improve performance. The DMA descriptor refers to the descriptor register, and the DMA engine refers to the DMA method for data transfer. The device-to-system DMA feature mainly refers to the compression chip transmitting compressed data to the system by means of DMA, including descriptor DMA and data DMA, wherein the data DMA includes a DMA descriptor and a DMA engine, and the descriptor DMA is a system. The descriptor information is transmitted by DMA to improve performance. The DMA descriptor refers to the descriptor register, the DMA engine refers to the data transfer by DMA, and the characteristics of the sender arbitration mainly refers to the priority. The generated arbitration method controls and resolves conflicts when multiple messages are sent simultaneously. The characteristics of the data storage module interface mainly refer to the use of a configurable packet storage array to form a data buffer with high parallel storage for multiple compression. The algorithm core stores system-to-device data and device-to-system compressed data;

The core characteristics of the compression algorithm mainly refer to the core of multiple sets of compression algorithms, so that each module and interface The bandwidth matching ensures the utilization of resources, thereby achieving the purpose of improving system performance. The core module of the compression algorithm includes a data compression module, a compression control module, and a matching information module, and the data compression module mainly implements data compression function, and the compression control module mainly The control information is received, and the compression control information is generated, and the compression module is controlled to work. The matching information module mainly stores the matching information in the compression process for use by the compression module.

The DMA-based data compression chip structure and the implementation method thereof have the above advantages, which make up for the deficiency of the traditional software data compression design structure seriously affecting the system CPU computing resources, and does not significantly affect the system. Under the condition of performance, it has many features such as configurability, hardware resource saving, and flexible expansion of the core of the compression algorithm, making it more suitable for big data storage and analysis systems, so it has a very broad development prospect. High technical value. DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly described. It is obvious that the drawings in the following description are only some embodiments of the present invention. Other drawings may also be obtained from those of ordinary skill in the art in view of the drawings.

1 is a logic structural diagram of a DMA-based data compression chip structure and an implementation method thereof according to the present invention;

2 is a schematic diagram of a logic structure of a transmitting end arbitration of a DMA-based data compression chip structure and an implementation method thereof according to the present invention.

detailed description

The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Embodiment 1

A data compression chip structure based on DMA, comprising a system to device DMA module, a device to system DMA module, a transmitting end arbitration module, a data storage module interface module, and a compression algorithm core module, wherein The system-to-device DMA module uses descriptor DMA and data DMA to improve system efficiency, that is, descriptor information and transmission data are all DMA transmission; device-to-system DMA module uses descriptor DMA and data DMA to improve the system. The efficiency, that is, the descriptor information and the transmission data are all DMA transmission; the arbitrating module at the transmitting end controls the arbitration mechanism for sending packets by giving priority analysis, determination, and generation, and is used to solve the conflict problem of multi-message transmission and transmission. The data storage module interface module uses a high parallel multi-group storage array to implement data caching; the compression algorithm core module adopts a data path bandwidth matching method to configure multiple compression algorithm cores. Embodiment 2

A DMA-based data compression chip structure implementation method is to communicate with a system through a PCI-E interface, realize system and device data transmission through DMA technology, respectively design a system to device and device to system DMA engine, and send The end design arbitration logic realizes the efficient transmission of the transmitted data. The descriptor DMA and the data DMA are respectively implemented in the DMA module in two directions to improve the data transmission efficiency, and the received data is connected to the data compression algorithm module through the interface storage module, and the data is connected. The compression module performs compression processing, and the compressed data is then transferred to the system via DMA. The implementation method of the present invention specifically includes the following steps:

The above is only the preferred embodiment of the present invention, and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., which are within the spirit and scope of the present invention, should be included in the protection of the present invention. Within the scope.

Claims

claims

1. A data compression chip structure based on DMA, including system-to-device DMA module, device-to-system DMA module, sender arbitration module, data storage module interface module and compression algorithm core module, where,

The system-to-device DMA module uses descriptor DMA and data DMA to improve system efficiency, that is, both descriptor information and transmission data use DMA transmission;

The device-to-system DMA module uses descriptor DMA and data DMA to improve system efficiency, that is, descriptor information and transmission data are both transmitted using DMA; the sending-end arbitration module uses priority analysis, judgment, and generation to control sending. The message arbitration mechanism is used to solve the conflict problem of multiple message sending and transmission; the data storage module interface module uses highly parallel multi-group storage arrays to implement data caching; the compression algorithm core module uses the data path bandwidth matching method, with multiple configurations. A compression algorithm core.

2. An implementation method of a DMA-based data compression chip structure is to communicate with the system through the PCI-E interface, realize data transmission between the system and the device through DMA technology, and design system-to-device and device-to-system DMA engines respectively. Arbitration logic is designed at the sending end to achieve efficient transmission of sent data. Descriptor DMA and data DMA are implemented in the DMA modules in both directions to improve data transmission efficiency. The received data is connected to the data compression algorithm module through the interface storage module. , the data is compressed in the compression module, and the compressed data is then transmitted to the system via DMA.

3. A method for implementing a DMA-based data compression chip structure according to claim 2, which specifically includes the following steps:

A. The system initiates a read request. The data compression chip sends a read request and a read command through its sending end and sends them to the system through the PCI-E port. At the receiving end, the descriptor information sent by the system is received. This information is sent to the device through the system. The descriptor DMA method controls the transmission, that is, the start and end address information of the descriptor, and is transmitted through DMA, and finally written to the descriptor register, which is located in the system to device descriptor, and controls the data transmission of the corresponding descriptor information through the system to device DMA engine. ;

B. The received DMA data is transmitted to the data storage interface module through the system to device interface, so that the compression algorithm core module can perform data compression applications. The data storage module is a highly parallel group memory. c. The compression algorithm core module reads data from the data storage interface module, performs data compression operations, and collects data information for compression control to improve compression efficiency. The data matching information during the data compression process is saved by the matching information module for compression. module usage;

D. Determine the number of cores of the compression algorithm core module based on the PCI-E interface, DMA transmission capability, data storage interface, compression core module capability, each interface of the system and the bandwidth of the module;

E. The compressed data passes through the data storage interface module and is controlled by the device-to-system DMA module to complete device-to-system data transmission; the device-to-system descriptor DMA method also controls the transmission of multiple descriptor information to improve system performance. , the transferred descriptor information is stored in the device to system descriptor register, and is transmitted from the device to the system in DMA mode;

F. When the sending end data is sent, the sending end arbitration module controls the priority control when sending multiple message data, avoiding the conflict problem when multiple messages are sent at the same time. Finally, the data is transferred to the system through the PCI-E interface. Send transmission.

4. A method for implementing a DMA-based data compression chip structure according to claim 3, in which during system operation, internal interrupts and external interrupts are controlled by the terminal module and transmitted to the system.

5. A method for implementing a DMA-based data compression chip structure according to claim 3, in step D, each compression algorithm core module works completely independently in parallel.