CN106817216B

CN106817216B - ZIP (ZIP packet decompression) method based on ZLib library and AES (advanced encryption Standard) algorithm

Info

Publication number: CN106817216B
Application number: CN201611071964.0A
Authority: CN
Inventors: 孙永超
Original assignee: Beijing Ruian Technology Co Ltd
Current assignee: Beijing Ruian Technology Co Ltd
Priority date: 2016-11-29
Filing date: 2016-11-29
Publication date: 2022-02-22
Anticipated expiration: 2036-11-29
Also published as: CN106817216A

Abstract

The invention relates to a ZIP packet decompression method based on a ZLib library and an AES algorithm. The method comprises the following steps: 1) acquiring ZIP packet data of a website, reading a file from the ZIP packet and judging whether the file is encrypted by adopting an AES algorithm or not; 2) for ZIP packet data encrypted by adopting an AES algorithm, the AES algorithm is added on the basis of a ZLib library to decrypt and decompress the ZIP packet. Further, step 2) encapsulates the AES algorithm in libaes. In the invention, the open source compression ZLib project is improved and optimized, and the AES decryption algorithm is added, so that the YCL can accurately and quickly process the data reported by the website operator, and the method has the advantages of high efficiency and less CPU occupation compared with the 7-zip algorithm.

Description

ZIP (ZIP packet decompression) method based on ZLib library and AES (advanced encryption Standard) algorithm

Technical Field

The invention relates to the technical field of computer information security, in particular to a Zlib-based open source compression and decompression item, which is optimized and added with an AES decryption algorithm. A method for decrypting and decompressing an AES encrypted ZIP packet by calling a dynamic link library of a ZLib by an application.

Background

The development of technology makes computers go deep into the aspects of our lives, and compression is a very typical technology and is applied to many occasions, such as file systems, databases, message transmission and other occasions. Compression can be divided into lossless compression and lossy compression; lossy, which means that the original information cannot be completely restored after compression, but the compression rate is high; lossless compression is used for situations such as files and the like where information must be completely restored, and the ZIP file mentioned herein is a lossless compression.

In practical application, it is often necessary to read and analyze the internet data of each website to extract interesting information. But the content of the internet data file of each website is very huge, and the data needs to be compressed; in order to prevent information leakage, data is also subjected to encryption processing.

Aes (the Advanced encryption standard) is a specification used by the national institute of standards and technology for encrypting electronic data. It is widely used in the fields of finance, telecommunications, government digital information, etc. AES is an iterative, symmetric key block cipher that may use 128, 192, 256 bit keys and encrypt and decrypt data in 128 bit (16 byte) blocks.

Zlib is a free, universal, non-intrusive, lossless, data compression development library, and is also an open source project across platforms. This means that anyone can modify and use without paying any fee, which is also the main reason for developing the Zlib library in general when reading and analyzing the internet data of each website, and the ZIP format belongs to one of several mainstream compression formats.

When a Zlib library is used for reading and analyzing the internet data of each website, the data is compressed and decompressed by adopting a Zlib library function carried by Linux, data reported by some websites are encrypted and compressed into ZIP packets by adopting 7-ZIP software, and the data is encrypted by adopting an AES algorithm through the 7-ZIP software. However, the Zlib only supports a simple encryption algorithm and does not support an AES encryption and decryption algorithm; the problem that comes with it is that the Zlib library cannot normally handle data reported by the website that is encrypted with AES.

Disclosure of Invention

The invention aims to provide a ZIP package which is based on a ZLib open source library, can automatically and accurately decrypt and decompress a file encrypted and compressed by 7-ZIP software by combining an AES decryption algorithm, and can process the encrypted ZIP package reported by a website by only adding a libz.

The technical scheme adopted by the invention is as follows:

a method for decompressing ZIP packets based on a ZLib library and an AES algorithm comprises the following steps:

1) acquiring ZIP packet data of a website, reading a file from the ZIP packet and judging whether the file is encrypted by adopting an AES algorithm or not;

2) for ZIP packet data encrypted by adopting an AES algorithm, the AES algorithm is added on the basis of a ZLib library to decrypt and decompress the ZIP packet.

Further, step 2) encapsulates the AES algorithm in a static library file, and encapsulates the ZLib library in a libz.

Further, the AES encryption and decryption algorithm includes:

AES encryption process: SubBytes, ShiftRows, MixColumns,

AddRoundKey (round key addition);

AES decryption process: InvSubBytes (reverse byte substitution), InvShiftRows (reverse shift), InvMixColumns

(reverse rank obfuscation) AddRoundKey (round key addition).

The invention has the following advantages and positive effects:

in the invention, the open source compression ZLib project is improved and optimized, and an AES decryption algorithm is added; this is exactly a 7-zip encryption algorithm, so that the data reported by the website operator can be processed accurately and quickly. In addition, the ZLib decompression algorithm adopts a libz.so dynamic library mode, while the AES algorithm adopts a static library mode packaged, and can be automatically installed when processing data reported by a website without upgrading the ZLib library of the system, so that the additional problem caused by too low version of an operating system is avoided; finally, the ZLib decompression algorithm adopted by the invention has the advantages of higher efficiency and less CPU occupation than the 7-zip algorithm.

Drawings

FIG. 1 is a flow chart of processing data reported by a website;

FIG. 2 is a flow diagram of a call to ZLib to decompress a ZIP file;

FIG. 3 is a flow chart of AES encryption decryption;

FIG. 4 is a schematic representation of a SubBytes (byte alternate) matrix;

FIG. 5 is a schematic diagram of ShiftRows (row shifting);

FIG. 6 is a graph of 7zip versus ZLib decompression time.

Detailed Description

The invention is further illustrated by the following specific examples and the accompanying drawings.

Fig. 1 shows the whole process of processing the reported data of the website in the prior art. As shown in the figure, firstly, ZIP packet data is received from a field carrying program, a corresponding directory is scanned, then a ZIP packet file is decompressed, a bcp file and an entity file after the ZIP packet file is decompressed are scanned, finally, format conversion is carried out on the bcp file, namely invalid data is obtained, valid data is verified, and specific information is extracted from the valid data; and the entity file is put into a full-text database. The invention relates to a method for decompressing a ZIP packet by adding an AES algorithm on the basis of an original ZLib library, which is positioned in the third link of figure 1.

Decompression Module design

Although the compression rate of 7ZIP is high and AES encryption and decryption are supported, 7ZIP occupies a high cpu, which greatly affects the real-time performance of the method when the data is processed by a website, so that the ZLib library is adopted in the method to decompress the ZIP file.

The ZLib library belongs to an open source library, adopts a DEFLATE compression algorithm and only supports a simple encryption and decryption function; but the method does not support the AES encryption and decryption algorithm, so that the open source library ZLib code must be modified to add the AES encryption and decryption algorithm to decompress ZIP packets encrypted by the AES algorithm. In order to not affect the interface called before, the AES algorithm is packaged in a library.a static library file (the library.a is the static library name defined by the invention); so dynamic library file encapsulated Zlib library in libz. This has the advantage that no other program has to be modified by only updating the two library files, no matter how the code is modified.

Modify unzip.c files, add

Modifying an undincurrentcurrentFilePassdord function, wherein the function mainly has the function of obtaining ZIP file information; judging whether HEADID is 0x9901, and encrypting the ZIP packet by adopting AES (advanced encryption Standard); reading the version number; reading two bytes to judge whether the two bytes are 'A' and 'E'; read ZIP packet encryption mode, here AES; read ZIP packet compression mode.

Modifying an untReadCurrentFile function, wherein the function mainly has the functions of reading the file content in the ZIP package and judging the previously read ZIP encryption mode; if the key is AES, the libaes.a static library is called to decrypt, and the plaintext is obtained by decrypting with a known key.

And (3) decompression flow: as shown in fig. 2, includes: opening a ZIP stream, acquiring the information of a ZIP file, opening the currently positioned file, circularly reading the file (if AES encryption is adopted, AES decryption is carried out on the read data), closing the current file, and closing the ZIP stream. The method comprises the following specific steps:

1. opening a ZIP package in a file stream mode;

2. locating a first file in the compressed files;

3. obtaining file information related to ZIP, continuing recursion if the file information is a directory, and creating a file if the file information is a file;

4. opening a currently positioned file, acquiring ZIP information, judging whether AES encryption is performed or not, and initializing an AES information structure body if the AES encryption is performed, wherein the structure body is called AES _ cox;

5. reading a file from the current ZIP packet, circularly reading, judging whether the file is AES encrypted, if so, decrypting and reading the content, and writing the read content into the previously created file;

6. setting a file time stamp;

7. closing the currently read file, and releasing the memory of the AES information structure which is dynamically allocated before;

8. if other files exist in the ZIP package, positioning the next file;

9. the open flow is closed.

AES encryption and decryption algorithm constitution

As shown in fig. 3, the core part of the AES encryption algorithm includes: SubBytes, ShiftRows, MixColumns, AddRoundKey, round key plus. The AES decryption algorithm is the inverse of encryption, so except AddRoundKey (round key addition), the rest need to be inverse transformed, InvSubBytes (inverse byte substitution), InvShiftRows (inverse shift), InvMixColumns (inverse column confusion).

1. Byte substitution

The main function of byte substitution is to perform a mapping of one byte to another through the S-box. It is a non-linear transformation on bytes that non-linearly transforms each byte in the state to another byte, and the substitution table S-box is invertible; as shown in fig. 4.

The byte substitution is divided into: forward byte substitution and reverse byte substitution.

2. Line shifting

The function of the row shift is to implement permutations between bytes within a 4 x 4 matrix.

The line shifting is divided into: forward row shifting and reverse row shifting.

The operation of the forward row shift is: the first row is left unchanged, the second row is left shifted cyclically by 1 byte, the third row is left shifted cyclically by 2 bytes and the fourth row is left shifted cyclically by 3 bytes as shown in fig. 5.

Reverse row shifting is the opposite operation.

3. Column obfuscation

Column confusion: it is a column-by-column transformation of a state, i.e. a linear transformation is used to mix the four bytes of each column.

Column confusion is divided into: forward column confusion and reverse column confusion.

4. Round key adding

In the round key adding process, simple bitwise XOR operation is used, the key is expanded through a key scheduling process to obtain a round key, and then a plaintext is XOR-ed with a corresponding sub-key, namely the input of each round is XOR-ed with the round key once; therefore, the key of the round can be recovered when the key is decrypted or not.

5. Key scheduling

The key scheduling includes two parts: key expansion and round key selection. The key expansion is used to obtain the round keys used in the round transformation before data encryption/decryption.

In summary, it can be known from the AES encryption and decryption flowchart that both AES encryption and decryption are iterative processes; and repeatedly replacing the data in the iteration process until the final plaintext or ciphertext is obtained.

API called in program

API 1: decompressing files

void unzip_extract(char*pZipFile,char*pExtractDir)

Inputting parameters:

1) decompressing file names, 2) decompressing directory names;

and returning a value: none.

API 2: opening a file stream

void*unzOpen(char*pZipFile)

Inputting parameters:

1) decompressing the file name;

and returning a value:

1) converted to a file stream pointer in void.

API 3: go to the first file

int unzGoToFirstFile(void*uf)

Inputting parameters:

1) a file stream pointer;

and returning a value:

1)0 succeeded, 2) -1 failed.

API 4: opening a current file

int unzOpenCurrentFilePassword(void*uf,char*pwd)

Inputting parameters:

1) file stream pointer, 2) key (key when website encrypts ZIP),

and returning a value:

1)0 succeeded, 2) -1 failed.

API 5: reading current file content

int unzReadCurrentFile(void*uf,char*buf,size_t size)

Inputting parameters:

1) file stream pointer, 2) read content, 3) content size;

and returning a value:

1)0 succeeded, 2) -1 failed.

API 6: closing a current file

int unzCloseCurrentFile(void*uf)

Inputting parameters:

1) a file stream pointer;

and returning a value:

1)0 succeeded, 2) -1 failed.

API 7: go to the next file

int unzGoToNextFile(void*uf)

Inputting parameters:

1) a file stream pointer;

and returning a value:

1)0 succeeded, 2) -1 failed.

API 8: closing a file stream

int unzClose(void*uf)

Inputting parameters:

1) a file stream pointer;

and returning a value:

1)0 succeeded, 2) -1 failed.

API 9: inverse nematic aliasing

void containment _ mixColumns (unidentified char · col) input parameters:

1) processing data (content of files in ZIP package);

and returning a value:

1) the processed data is obfuscated.

API 10: reverse byte replacement

void contrary_subBytes(unsigned char*col)

Inputting parameters:

1) processing data (content of files in ZIP package);

and returning a value:

1) and replacing the processed data by the reverse byte.

API 11: reverse row shift

void contrary_shiftRows(unsigned char*col)

Inputting parameters:

1) processing data (content of files in ZIP package);

and returning a value:

1) and shifting the processed data in reverse rows.

API 12: round key adding

void add_roundKey(unsigned char*col,unsigned char*key,int round)

Inputting parameters:

1) processing data (content of a file in a ZIP package), 2) a key (key when a website encrypts and compresses ZIP), and 3) polling times;

and returning a value:

1) the round key adds the processed data.

The method comprises the steps of respectively using 7zip and ZLib to conduct decompression test on reported data of a website, wherein the time unit obtained by the test is microsecond, and is shown in the following table 1; and the decompression results are analyzed and compared, as shown in fig. 6, the abscissa of the graph represents different files, and the ordinate represents the time taken for decompression.

TABLE 1 test results List

File name/decompression method	7z ip(us)	Zl ib(us)
			data_1471449694.zip	617700	2922
data_1471453015.zip	256364	3209
			data_1471453608.zip	183181	5564
data_1471451806.zip	193180	6083
			data_1471450007.zip	169391	18622
data_1471450907.zip	250134	15472
			data_1471452105.zip	180231	36974
data_1471451507.zip	184553	15633
			data_1471450307.zip	206964	7315

From fig. 6 it can be concluded that decompression with the method of the invention using the Zlib library is far superior in temporal level to decompression with the 7zip library.

The above embodiments are only intended to illustrate the technical solution of the present invention and not to limit the same, and a person skilled in the art can modify the technical solution of the present invention or substitute the same without departing from the spirit and scope of the present invention, and the scope of the present invention should be determined by the claims.

Claims

1. A ZIP packet decompression method based on a ZLib library and an AES algorithm is characterized by comprising the following steps:

2) for ZIP packet data encrypted by adopting an AES algorithm, the AES algorithm is added on the basis of a Zlib library to decrypt and decompress the ZIP packet, wherein the AES algorithm is packaged in a static library file, and the Zlib library is packaged in a libz.

The method for decrypting and decompressing the ZIP packet by adding the AES algorithm on the basis of the ZLib library comprises the following steps:

c, modifying the unzip.c file, and adding the following contents:

uLong encryption _ mode, which represents an encryption mode;

uLong comp _ method, indicating compressed mode;

uLong aes _ version, representing aes version number;

aes _ condex aes _ cox, representing aes information, including key, key length, plaintext, file decryption mode;

modifying an undincurrentcurrentFilePassdord function, wherein the function is used for obtaining ZIP file information; judging whether HEADID is 0x9901, and encrypting the ZIP packet by adopting AES (advanced encryption Standard); reading the version number; reading two bytes to judge whether the two bytes are 'A' and 'E'; reading a ZIP packet encryption mode; reading a ZIP packet compression mode;

modifying an untReadCurrentFile function, wherein the function is to read the file content in the ZIP package and judge the previously read ZIP encryption mode; if the key is AES, the libaes.a static library is called to decrypt, and the plaintext is obtained by decrypting with a known key.

2. The method of claim 1, wherein the AES algorithm comprises:

a) an AES encryption process, comprising: byte substitution, row shift, column obfuscation, round key addition;

b) an AES decryption process, comprising: reverse byte substitution, reverse shift, reverse column obfuscation, round key addition.

3. The method of claim 2, wherein said byte substitution performs a byte-to-byte mapping through an S-box, the mapping being a non-linear transformation of bytes.

4. The method of claim 2, wherein the row shifting implements permutations between bytes within a 4 x 4 matrix, including forward row shifting and reverse row shifting; the operation of the forward row shift is: the first row is left unchanged, the second row is left shifted cyclically by 1 byte, the third row is left shifted cyclically by 2 bytes and the fourth row is left shifted cyclically by 3 bytes.

5. The method of claim 2, wherein the column obfuscation transforms one state column by column, i.e., a linear transformation is used to mix the four bytes of each column.

6. The method of claim 2, wherein the round key addition uses a bitwise xor operation, the round key is obtained by expanding the key through a key scheduling process, then the plaintext is xored with the corresponding subkey, i.e., the input of each round is xored with the round key once, and the input can be recovered by xoring the key of the round again during decryption.

7. The method according to claim 1, wherein after decompressing the ZIP package file in step 2), scanning the acquired bcp file and the entity file; format conversion is carried out on the bcp file, effective data are obtained and verified, and specific information is extracted from the effective data; and putting the entity file into a full-text database.