UA64907A - Adaptive device for compressing text messages - Google Patents

Abstract

The proposed adaptive device for compressing text messages contains an input data register, a register for forming the prefix code of a text row, a comparator, a random-access memory unit, a timing pulse generator, and additionally, a unit for determining the text type, a unit for selecting a module of the random-access memory, a counter module, and a control unit. The input of the register for forming the prefix code is connected to the first output of the input data register. The second output of the input data register and the output of the register for forming the prefix code are connected to the corresponding inputs of the comparator. The unit for determining the text type and the module of the random-access memory are connected in series to each other and connected between the input of the proposed device and the input of the random-access memory unit. The first output of the comparator is connected to the corresponding input of the unit for determining the text type, and the second output is connected to the input of the control unit. The outputs of the control unit are connected to the corresponding control inputs of the input data register, the register for forming the prefix code, and the counter module.

Description

The invention relates to the field of effective coding of information and can be used in computer communication nodes and the Internet.

There is a device for compressing text messages using the G2M method/ V.S. Chernega. Compression of information in computer networks / V.S. Chernega - Sevastopol: SevGTU, 1997. - 214 p.), which contains a code table, a code counter, a block for forming a hash function, a comparison scheme and a prefix register. However, this scheme does not take into account the informational and statistical properties of the language of the text message. Therefore, at the initial stage of coding, when the code table of strings of variable length is being built, single symbols coded with code words longer than 8 bits are sent from the output of the device to the transmission channel, which leads to the expansion of the transmitted message. As the code table is filled with substrings of the compressed text and two or more symbols (substrings) are encoded with one code combination, the amount of information transmitted to the channel decreases due to the increase in the compression ratio.

The prototype was taken as a device for encoding lines of variable length (UmMeisp T.A. A iespppidne yTog podp-repotapvze data sotrgezviop / IEEE Sotrshiieg. - 1984. - MoI.17. -Mb. - R. 8-19.Y. This device contains a register of input characters, to which the hash function formation block and the subcode formation register formed from the characters of the compressible text are connected. The output of the substring code formation register is connected to the hash function formation block, which speeds up the procedure for finding a line in a table of variable-length lines; a comparison device , which searches the table of variable-length strings for substrings received in the input register from the input of the device; and the table of variable-length strings containing the substrings found by the device in the text. Moreover, the output of the substring code formation register is the output of the device. To the address selection block string tables are connected with an initial counter and a block for forming a hash function. A string table of variable length is connected to the output of the string table address selection block, to which in turn, the device for comparing the substring code generation register and the code counter is connected.

The invention is based on the task of increasing the degree of compression and reducing the time of compression of text information, especially at the initial stage of effective text encoding. The problem is solved in the following way. It is proposed to divide the text into different types: spoken language, fiction, business texts, etc. Due to the fact that for each type of text of the same language there is its own statistics of substrings, which can be collected in advance by processing several texts of the same type, that is, in each table there are substrings with the highest probability for each type of text. The use of these tables allows you to avoid building inefficient code combinations at the initial stage of coding. To implement this method, a text type evaluation block, a block of text type coding tables, implemented using a block of random access memory (RAM), strings of variable length corresponding to different types of text, and a switch of tables are introduced into the text information compression device. The drawing (Fig.) shows the structural diagram of this compression device.

The device includes a text type evaluation block 1, an input character register 2, a clock pulse generator (CTI) 3, a prefix register 4, an output register 5, a control device 6, a table switch 7, a RAM block 8, a comparison block 9, a block of code counters 10.

The first inputs of block 1 and block 2 are connected in parallel, and the point of their connection is the input of the device. The output of block 1 is connected to the input of block 7, the outputs of which are connected to block 8 and block 6. The outputs of block 6 are connected to the second inputs of blocks 2, 4 and 10, and the corresponding inputs to the outputs of blocks 3, 9 and 7 The outputs of block 2 are connected to the inputs of blocks 4 and 9, respectively. The outputs of block 10 are connected to block 8, the outputs of which are connected to block 9. The corresponding outputs of block 9 are connected to blocks 1 and 6. The outputs of the block 4, are respectively the inputs of blocks 9 and 5. The output of block 5 is the output of the compression device.

The device works as follows.

Encoding symbols are sequentially entered into input symbol register 2. Then, the symbol from block 2 is shifted into prefix register 4, and input register 2 is loaded with the next symbol of the input stream. The code combination located in block 4 is a prefix that, together with the character of the input register, will form a string. The text type evaluation unit 1 performs a sequential search for a line in each of the RAMs of the table unit. The table in which the largest number of substrings matching the substrings to be coded (in the experimental model that was created, through dths of characters) is determined to be the current one. The row index of the table is the row code and is written to the output register. At the initial stage of coding, a table generalized for all types of text is used.

The code of the found string is assigned a new prefix and entered in prefix register 4. Then the next symbol of the stream is entered into input register 2, which is connected to the prefix, and a matching substring is searched for again. This operation continues until it is noted that the newly created row is missing in each of the tables. At the same time, the number of substrings per unit of time is calculated.

If the frequency of the newly formed line exceeds the limit, it is entered in the current code table in the position of the first free line.

The code combination from the output register 5 is issued to the output of the device. The character from the input register 2 is moved to the prefix register and the coding procedure starts over.

After receiving at the input of the compressor M symbols, which are enough for a reliable assessment of the text class, the search for a comparable line in the RAM of the encoding tables is not carried out by sequentially viewing them, but in the table corresponding to the given text class.

Thus, the search time for a comparable line is significantly reduced and thus the speed of the compressor increases.

In the process of encoding incoming characters, the text class score may change. In this way, the order of viewing the RAM of the encoding tables is changed for the presence of a matching substring.

The control device, text evaluation unit, input and output registers are implemented on the basis of the GTI signal microprocessor - on the basis of the quartz clock pulse generator. The RAM block is based on memory chips.

Thus, the use of an adaptive compression device for text messages for systems of transmission and storage of text information allows to increase the degree of data compression due to the introduction of a text evaluation block and probability tables of strings of variable length for different types of text. This makes it possible to reduce the time of transmission of text data over communication channels, as well as to reduce the amount of memory that deals with texts when stored on magnetic and optical media. 4 babysitters nn nn babysitters for E nn U Eesennnnnntnnnnnu ! Tires and Tires. write ski, drink

E | day slddjkkya poten nn? Her 5: и Z ШЕ ЗИ: и и и сеххк жк шх и на

From her x. And this and more! in and I coffee and eat them.

What SHY Y y and 5 o chimney nn I Z skid. pi Z i MNN: z poni Z vi nnnn i ish: i | :

It's been a long time since I've been doing it:

SA; -

Fig.

Claims (1)

An adaptive text message compression device comprising an input character register to which a substring code formation prefix register is connected, the output of this register and a second output of the input character register are connected to corresponding inputs of a comparator unit which in turn is connected to by a block of operational storage devices, as well as a clock pulse generator, which differs in that between the input to the device and the block of operational storage devices, a serially connected text type evaluation block and a switching block of operational storage devices are introduced, the first output of the block the comparison is connected to the corresponding input of the text type evaluation block, and the second output is connected to the control block, which is connected to the input character register, the prefix register of the substring code formation, and also to the code counter block.