CN100474781C - Compression method of two-byte character data - Google Patents

Abstract

The present invention provides a method for compressing information in units of 2-byte characters (Korean characters, Chinese) before storing in an information processing module of a terminal, thereby reducing storage space for 2-byte character data. . The compression method of 2-byte character data of the present invention is characterized in that comprising: generate a plurality of compressible codewords according to the frequency number, store in the basic dictionary table, the step of the variable initialisation of the representation next codeword of registering; Identify input Whether the information data is a 2-byte character, and receive the input step; compare whether the input data is contained in the compressible code word, and when contained in the compressible code word, go through the mapping process from the dictionary table Searching for the matching code and outputting it, when there is no such matching code in the dictionary, registering it in the dictionary; judging whether it is the mantissa of the data, when the data has not been input, returning to the input step of inputting the information data in sequence; and when it is During the mantissa of data, carry out the step of clearing process, when the number of digits that conforms to the code obtained by encoding this compressible code word is smaller than the critical value that this compressible code word can reduce bit, take log ₂ (C1+1)-1 bit Output, when the matching code word is larger than the critical value, it is output with log ₂ (C1+1) bits, where C1 is the number of code words currently assigned.

Description

Compression method for 2-byte character data

technical field

The present invention relates to a kind of compression method of 2-byte character data, relate to a kind of in order to reduce the information storage space of SMS (Short Message Service) and EMS (Enhanced Messaging Service) in mobile communication terminal, utilize 2 Compression method for 2-byte character data of byte character compression algorithm.

Background technique

Generally, customers use the information sending and receiving functions (SMS, EMS) of mobile communication terminals to exchange various information. Most mobile communication terminals hardly compress such information, and terminals that perform partial compression only use compression algorithms suitable for English letters.

However, when using this compression algorithm, languages such as Korean characters and Chinese are relatively redundant because most of them have the characteristics of redundancy, so the compression efficiency is relatively low, and more memory is required, and there is a problem that the storage space cannot be effectively reduced. .

[Patent Document 1] Japanese Patent Laid-Open No. 2-255977 (Japanese Patent Publication No. 1990-255977)

[Patent Document 2] Japanese Patent Laid-Open No. 9-069785 (Japanese Patent No. 1997-069785 publication)

Contents of the invention

The present invention overcomes the above-mentioned disadvantages, and its purpose is to provide a 2-byte character (Korean character, Chinese) to compress and store information in the information processing module of the terminal, thereby reducing storage space. Compression method for byte character data.

In order to achieve the above object, the compression method of the 2-byte character data of the present invention is characterized in that comprising: generating a plurality of compressible code words according to the frequency number, and storing them in the basic dictionary table, and registering the variable representing the next code word The step of initialization; referring to the initialized variable, storing the additional compressible codeword in the additional dictionary table including the basic dictionary table, and reinitializing the registered variable representing the next codeword; identifying Whether the input information data is a 2-byte character, and receive the input step; compare whether the input data is contained in the compressible code word, and when contained in the compressible code word, it is mapped from the dictionary table The process searches for the matching code and outputs it. When there is no such matching code in the dictionary, register it in the dictionary; judge whether it is the mantissa of the data, when the data has not been input, return to the input step of inputting the information data in sequence; and when When it is the mantissa of the data, the step of performing a clearing process, the clearing process refers to storing data in 8 bits or 16 bits in the memory storage method, but for the compressed data to have a variable length of bits, when the last When the stored data is not 8-bit or 16-bit, the process of filling the last remaining bit with 0; when the number of digits of the code obtained by encoding the compressible codeword is lower than the number of digits that can be reduced by the compressible codeword When the critical value is small, it will be output with log2(C1+1)-1 bits. When the number of digits corresponding to the code word is greater than the critical value, it will be output with log2(C1+1) bits. The C1 is the number of code words currently assigned .

The beneficial effect of the present invention is that, in the information processing module of the terminal, by compressing and storing information of 2-byte characters (Korean characters, Chinese, etc.), the storage space can be reduced. That is to say, when using the method of the present invention to compress a text file with mixed English and Korean characters, the average compression rate has an improvement effect of about 22% compared with the existing compression method.

Description of drawings

FIG. 1 is an operation flowchart of a method for compressing 2-byte character data in one embodiment of the present invention.

Fig. 2 is to in the compression method of 2 byte character data of an embodiment of the present invention, from this dictionary table through mapping process search accords with code and the step (compression step) that output is described in detail operation flowchart.

FIG. 3 is an operation flowchart explaining in detail the dictionary generation/management steps for managing the code-compliant dictionary in the compression method of 2-byte character data according to an embodiment of the present invention.

Detailed ways

For the convenience of description, the method for compressing 2-byte character data in the present invention is described by taking Korean as an example. But the same applies to languages marked in 2 bytes such as Chinese, Japanese, etc. Therefore, in this embodiment, only the compression of the Korean language is described, but the present invention is not limited to the Korean language, which is obvious to those skilled in the art.

Embodiments of the present invention will be described below with reference to the accompanying drawings.

FIG. 1 is an operation flow chart of a method for compressing 2-byte character data in an embodiment of the present invention, and the related situation will be described below.

First, initialize the maximum number of character strings (N7), the number of code words (N2), the initial dictionary entry number (N5), etc., store the characters with high frequency numbers in the basic dictionary table, and store the registered variable representing the next code word C1 initialization (S101), the composition of the code word used for character compression is described in the following table. Here, in order to find the code words needed for character compression, after finding out the occurrence frequency of 2350 words of complete Korean characters from the Korean character and English mixed file, they are arranged and observed, and 470 words that are 2% of them are frequently used Register as a code word. In this case, the overall occurrence frequency of the 2% of the 470 characters reaches more than 85%. Therefore, the initialization value of the variable C1 can be 471.

Table 1:

0～255 ASCII (American Standard Code for Information Interchange) 256～725 Korean character code (470 characters) 726～1023 10-bit encoding 1024～2047 11-bit encoding 2048～4095 12-bit encoding

Next, an additional compressible codeword is stored in the additional dictionary table including the basic dictionary table in comparison with the initialized variable, and the registered variable C1 indicating the next codeword is reinitialized (S102). Here, the number of bits of the conforming code encoding the compressible codeword depends on the following formula.

Formula 1: (C1+lim)≤2 ^log(C1+1) -1

Formula 2: lim=C3-C1-1

Formula 3: C3=2 ^log(C1+1)

Here, the C1 refers to the number of code words currently assigned, and lim refers to the critical value of a code word that can reduce bits. Therefore, when the code word is converted into a bit sequence, if the code word is smaller than the determined critical value (lim), it will be output in log ₂ (C1+1)-1 bits, and if the code word is larger than the critical value, then Output in log ₂ (C1+1) bits.

For example, when the C1 is 750, lim=(1024-750-1)=273, so the code word is between 0 and 273 during compression and output with 9-bit encoding, if the code word is between 274 and 749 during compression , add 274 to each code word, and output in 10-bit code

When uncompressing, read out the code word bit with 9 bits, if the read value is smaller than 274, then read its value as the code word code, if the read value is greater than 274, then read it again with 10 bits out, the value subtracted by 274 is read out as a code word code. The following Table 2 shows the structure of the dictionary table of the present invention as described above.

Table 2:

compressible codeword encoded code 10 hex 0 000000000 0 1 000000001 1 2 000000010 2 . . . . . . 273 100010001 273 274 1000100100 548(274+274) 275 1000100101 549(274+275) . . . . . . 749 1111111111 1023(274+749)

Thereafter, information data are input sequentially. Compare whether the input data is included in the compressible code word, if it is included in the compressible code word, go through the mapping process from the dictionary table, search for the matching code and output it (S103). Then, it is checked whether the matching code exists in the dictionary, and if it is not in the dictionary, a dictionary creation step of registering in the dictionary is performed (S104).

Afterwards, it is judged whether it is the end of the data, and if it is not the end of the data, it returns to the step of sequentially inputting information data (S105).

If it is the mantissa of the data, a flushing process (Flush) is performed (S106). Here, the clearing process refers to storing data in 8-bit or 16-bit in the memory storage method, but for the compressed data to have a variable-length number of bits, when the last stored data is not 8-bit or 16-bit For 16 bits, the process of filling the last remaining bits with 0s.

Fig. 2 is to in the compression method of 2 byte character data in one embodiment of the present invention, from this dictionary table, through mapping process, the operation flow diagram that searches the step (compression step) that matches code and outputs in detail , instructions for this are described below.

First, the first byte of input data is read (S201).

Then judge whether the first byte is in the first assignment range (S202). Here, when it is a complete Korean character, since the first byte is assigned 25 numbers from B0 to C8 in hexadecimal, the first assignment range can be from B0 to C8 in hexadecimal.

If the first byte is within the first assigned range, the second byte of the input data is read (S203).

On the other hand, if the first byte is not within the first assignment range, it is determined to be a character in the American Standard Code for Information Interchange (S207) because it is not a complete Korean character.

Then judge whether the second byte is in the second assignment range (S204). Here, when it is a complete Korean character, since the second byte is assigned 94 numbers from A1 to FE in hexadecimal, the second assignment range can be from A1 to FE in hexadecimal.

If the second byte is within the second assignment range, it is judged whether the input data is included in the dictionary table (S205).

On the other hand, if the second byte is not within the range of the second assignment, because it is not a complete Korean character, it is determined to be a character in the American Standard Code for Information Interchange

(S207).

If the input data is contained in the dictionary table, it is determined that the code value is met (S206).

On the other hand, if the input data is not included in the dictionary table, it is determined to be a character in ASI since it is not a Korean character with a high frequency of appearance (S207).

Fig. 3 checks whether this conforming code exists in the dictionary in the compression method of 2 byte character data of an embodiment of the present invention, if not just register in the dictionary in the dictionary, and remove the infrequent ones registered in the dictionary The operation flow chart that explains the dictionary management procedure of the code used in detail, and the explanation related to it is as follows.

First judge whether the string (length) of this code word exceeds maximum string number (N7), if the string of this code word exceeds maximum string number (N7), then terminate dictionary management step (S301).

If the character string of this codeword does not exceed maximum character string number (N7), then judge whether to exist in this dictionary table, when exist in this dictionary table, then terminate the dictionary management step (S302).

If it does not exist in the dictionary table, assign the character string to the new variable C1 (S303).

Next, the value of the new variable C1 is increased to be assigned to the code word of the next generated character string (S304).

Next, it is judged whether the increased variable C1 is greater than the number of code words (N2) (S305).

If the increased variable C1 is greater than the number of code words (N2), the dictionary entry number (N5) is assigned to the increased variable C1, if the increased variable C1 is less than the number of code words (N2), the dictionary entry number (N5) is not assigned to it (S306 ).

Then, judge whether the node assigned to the increased new variable C1 is the leaf (leaf) node as the node representing the end character of the character string or whether it is an unused node (C1==NULL), when assigned to the increased new variable When the node is not the leaf node of the node representing the end character of the character string in the dictionary entry or is not an unused node, return to the step of increasing the value of the new variable C1 in order to be assigned a value by the code word of the next generated character string (S307).

If the node assigned to the increased variable C1 is a leaf (leaf) node representing a character string end character or an unused node, the variable C1 is removed from the dictionary entry, and the code word of a new character string is ready to be assigned (S308).

The present invention is not limited to the scope disclosed in the above embodiments. Various improvements and changes can be made within the technical subject matter of the present invention, and these improvements and changes also belong to the technical scope of the present invention and are protected by the present invention.

Claims (8)

1. a compression method of 2 byte character data, is characterized in that comprising: Generate a plurality of compressible codewords according to the frequency number, store in the basic dictionary table, the step of initializing the variable representing the next codeword registered; Referring to the initialized variable, storing the additional compressible codeword in the additional dictionary table including the basic dictionary table, and reinitializing the registered variable representing the next codeword; Identify whether the input information data is a 2-byte character, and receive the input step; Compare whether the input data is included in the compressible code word, if it is included in the compressible code word, search the matching code from the dictionary table through the mapping process and output it, if there is no matching code in the dictionary When code, the step of registering it in the dictionary; Judging whether it is the mantissa of the data, when the data has not been input, return to the input step of sequentially inputting information data; and When it is the mantissa of the data, the step of performing a clearing process, the clearing process refers to storing data with 8 bits or 16 bits in the memory storage method, but for the compressed data to have a variable length of bits, when When the last stored data is not 8-bit or 16-bit, the process of filling the last remaining bit with 0; When the number of digits of the code obtained by encoding the compressible codeword is smaller than the critical value that the compressible codeword can reduce, output with log ₂ (C1+1)-1 bits, when the number of digits of the codeword is met When it is greater than the critical value, it is output in log ₂ (C1+1) bits, where C1 is the number of code words currently assigned. 2. the compression method of 2 byte character data according to claim 1, is characterized in that: In order to find the compressible codeword, after finding the occurrence frequency of the 2-byte characters of the completion type from the mixed file of 2-byte characters and 1-byte characters, arrange and analyze them, and use the frequently used characters Register as a code word. 3. the compression method of 2 byte character data according to claim 1, is characterized in that: Frequency numbers are measured from characters represented by combinations of 2 bytes or more, and only frequently used characters are registered in the dictionary as basic code words. 4. the compression method of 2 byte character data according to claim 2, is characterized in that: The 2-byte characters are Chinese, and the 1-byte characters are English characters. 5. the compression method of 2 byte character data according to claim 2, is characterized in that: The 2-byte characters are Korean and the 1-byte characters are English characters. 6. the compression method of 2 byte character data according to claim 1, is characterized in that: The step of searching and outputting matching codes through a mapping process from the dictionary table includes: The step of reading the first byte of the input data; The step of judging whether the first byte is located in the first assignment range; When the first byte is in the first assignment range, the step of reading the second byte of the input data; When the first byte is not located in the first assignment range, because it is not a complete North Korean character, the step of determining that it is a character in the American Standard Code for Information Interchange; The step of judging whether the second byte is located in the second assignment range; When the second byte is located in the second assignment range, a step of judging whether the input data is included in the dictionary table; When the second byte is not located in the second assignment range, because it is not a complete Korean character, the step of determining that it is a character in the American Standard Code for Information Interchange; When the input data is contained in said dictionary table, the step of determining a code value is met; and When the input data is not included in the dictionary table, since it is not a Korean character with a high frequency of appearance, it is determined that it is a character in the American Standard Code for Information Interchange. 7. the compression method of 2 byte character data according to claim 4, is characterized in that: The first assignment range is from B0 to C8 in hexadecimal. 8. the compression method of 2 byte character data according to claim 4, is characterized in that: The second assignment range is from A1 to FE in hexadecimal.

Images