JPH0335697B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Technical Field of the Invention] The present invention relates to a segmented compression method for character type data, and particularly to a segmented compression method for character type data used when creating an index for a database.

[Conventional technology and problems]

The segmented compression method is a method that efficiently compresses data when creating an index when a variable-length character text key or a composite key is constructed, and also quickly completes a size match comparison with the target key value. It is.

The segmented and compressed key used in the index section of a database management system can be handled in the same way as a single character key, so it is stored in the index section after being subjected to forward compression.

In conventional segmented compression methods for character type data (see Reference 1), blanks are usually positioned as the minimum value regardless of the code system, so control codes below that value must not be included, and , PL/
Comparisons in variable-length data item processing, such as those that appear in I and C language data types, consider trailing blanks and control characters in character strings valid, making it impossible to restore the data length. Ta.

Furthermore, many data processes have been implemented that include nulls, which represent things that are not applicable, such as the maiden name of an unmarried person, or unknown values, such as missing values in statistical data. The existence of data items outside the data item's domain is not allowed and cannot be processed within the segmented compression method, so to achieve this
A separate item was created to indicate whether the item was null or not, and this item was added to the beginning of the composite key as a data item.

Note that in order to indicate a null value, there is a method of promising a special value of a data item as null. However, if we consider Null as an example where the name is known but the date of birth is unknown, DDD=000 can be specified as unknown. However, since it is only necessary to clarify that this amount of information is ``non-existent,'' there should be no need to go to the trouble of storing six digits.

Next, the conventional method will be explained in more detail. The present invention is based on MWBlasgen et al. (see reference 1)
This is a further improvement of the improvement plan based on (see Reference 2).

For example, a problem arises when a composite key consisting of three data items (surname, given name, date of birth) is arranged with letters in alphabetical order and numbers in ascending order.

When allowing kana characters for names, you must usually reserve space for about 20 characters each for the surname and first name.
The date of birth should be around 6 digits (YYYDDD). If you try to store, for example, 100,000 such key values, each value will require 20+20+6=46 characters, a total of 4.7 million characters. However, for the average Japanese person's name, 1.7 + 2 + 6 = 9.7 characters would be sufficient, so it is easy to imagine that 970,000 characters would actually be enough for the amount of information.

Therefore, it is like taking the required length of the surname and first name individually and making them into a single string, (Hayashi, Yozaburou, 990115), (Hayashi Yo,
If we arrange the key values ``Jiro, 990115'' in lexicographical order, the character length of ``林'' is shorter than that of ``林波'', so the former should come before the latter, but ``二'' is When using a code system for characters less than 'three', the order will be reversed if the first and last names are concatenated.

A segmented storage method for character strings has been proposed as a decoded key compression method that does not have such side effects, the key value after compression is a simple comparison of the entire string, and the original key value can be restored.

In other words, the original data item is divided into fixed lengths for each type of data item, and control characters are inserted between them.
The purpose is to allow data to be compared between data and control units to be compared between control units, so that the control unit can decide how to handle the compressed data.

In the method of Reference 1, when assuming a composite key consisting of n data items (Σ ₁ Σ ₂ ...Σ _o ), this Σ _i is a character string σ ₁ σ ₂ ...σ _Mi (σ is the minimum value (consists of only blank character codes), this is divided into k sections of appropriate length 1 that satisfy the following relationship and K _i *l _i >M _i . π is a suitable padding character.

σ ₁ σ ₂ …σ _L ｜σ _L+1 σ _L+2 …σ _2*L ｜………｜σ
_(k-1)*L+1 ...σ _Mi π......π This break "|" is replaced with the control code according to the following rule according to the actually required length M of Σ. However, the length of the control character is the same as σ.

If there are characters other than blank in the entire interval to the right of the break, enter High-Value.

*∀x ('σ _x '＜High−Value) & number of characters,
Define High−Value.

If there are no characters other than blanks in the entire interval on the right, and the number of remaining characters is less than or equal to the interval length, the number of characters other than the last blank is retained.

A section that includes that section and has no characters other than blanks in all sections to the right is omitted.

Replace this with each Σ and treat Σ ₁ Σ ₂ ...Σ _o as one string.

If Σ match, the intervals match perfectly. If they do not match, the comparison is completed in the interval where the magnitude relationship is determined. Characters of different lengths that appear after the matching character are larger because the longer one is or. If it matches up to Σ _i ,
The comparison is repeated using the same rules for the next interval. For keys specified in descending order, the complement of the code is taken.

The method in Reference 2 limits data items to character strings whose minimum code is blank in the method in Reference 1; This was expanded upon the discovery that it is sufficient to concatenate it as a single character string data item consisting only of the final interval. It was also shown that forward compression is possible when further arranging this in dictionary order.

[References]

References 1 Blasgen, MW, Carey, RG, &
Eswaran, KP, 'An Encoding Method for
Mutifield Sorting and Indexing',
Cmumnication of the ACM, Nov.1977,
Vol.20, No.．． 11, pp874-878 Reference 2 Japanese Unexamined Patent Publication No. 58-2938 [Object of the Invention] The present invention is based on the above consideration, and
The purpose of this paper is to provide a partitioned compression method that does not require the assumption that the compression target is the minimum value of a record and can handle null data items.

[Structure of the invention]

Therefore, the segmentation compression method for character data of the present invention divides a composite key having multiple data items of character data as keys into an interval of a specified length for each data item, and inserts a separator at the designated segmentation point. Insert
It is a segmentation compression method for character type data that gives a value to the separator that is determined by the surrounding situation, and when separating data items, if the data item length is not an integral multiple of the specified interval length, it is padded with embedded characters and , If the data item is Null, it is expressed by a predetermined code with a specified interval length, and a separator is placed after it, and if the separator exists in the middle of the data item and the next character other than the first blank is or the separator is in the middle of a data item and the next non-blank character is greater than a blank, or the separator is at the end of a data item and the next data item is not null, It is characterized by determining whether the separator exists in the final section of a data item and the next data item is Null, or whether the corresponding data item is Null, and giving the separator a unique value according to the determination result. It is something.

[Embodiments of the invention]

Hereinafter, the present invention will be explained with reference to the drawings.

Figure 1 is a diagram explaining key compression, and Figure 2 is a diagram of a database management system to which the present invention is applied.
This is an example.

Figure 1A shows a complete key. The complete key is composed of key A, key B, and key P. Key A and key B are data items of the record. Key P is an address. The key length of key A is 50 characters. Null indicates that the value of the data item is undetermined.

FIG. 1B illustrates segmented compression. Key A is compressed in interval 4, key B is compressed in interval 3
, and is assumed not to be compressed. φ is a code smaller than (blank). The blank is
The EBCDIC code has the value "40" in hexadecimal. In FIG. 1B, square frames □ indicate separators for partitioning. The value of the segmentation separator is -1 if characters continue after the interval on the right of the segment; 0F ₁₆ if the first character other than the "blank symbol" is less than or equal to the "blank symbol"; /VARCHAR of I, n of C
It is used for special processing when the length of data is significant, such as delimiting. In other cases, the system is followed in which the space below the blank symbol is omitted from the delimiter.

-2 If a character other than a "blank symbol" exists as the first character, CF ₁₆ This is for faster size determination for the purpose of character string comparison only.

If there are no characters in the entire interval to the right of the division, or the separator of non-character items - 1 If the next item is not Null, Bl ₁₆ -2 If the next item is Null, Cl ₁₆
l is the effective length of the final section.

When this key is Null, FF ₁₆ Used when the first item is Null or when there is no section control immediately before. Note that l indicates the number of valid characters (in this example, the section length is assumed to be 16 or less).

(1) in FIG. 1B is obtained by applying the segmented compression method to the key in (1) in FIG. 1A. When key A is compressed in section 4, the first separator is inserted between "LION" and "φ". The next first non-blank character is φ, and φ has a value less than blank, so the first separator has a value of "OF". The next section is φ, and since there are only two characters, it is filled in with ``φ''. This section is the final section of key A, and the valid characters are the first two, so the second separator is "B2"
It has a value of Since key B has three digits and does not perform partitioned compression, the third separator has a value of "B3". (2), (3), and (4) in Figure 1A are similarly segmented and compressed.

The size order of the complete keys in FIG. 1 is determined as follows. First, use key A to determine the size order, and if key A cannot determine the size order, use key B to determine the size order, and if key B cannot determine the size order, use key P to determine the size order. determine the order. Note that keys with different lengths are padded at the back.
shall be of the same length and compared. In Figure 1B, the first section of the segmented compression key in (4) is
Since it has a value of FFFFFFFF, it is considered the largest among (1) to (4). Since the first sections of the segmented compression keys (1), (2), and (3) are all "LION", the value of the first separator is tried to determine the magnitude relationship. The first separator of the segmented compression key in (1) is "OF", the first separator of the segmented compression key in (2) is "C4", and the first separator of the segmented compression key in (3) Since the separator of is "CF", the order of size of the partitioned compression keys of (1), (2), and (3) is (3), (2), and (1).

FIG. 1C illustrates forward compression. In Figure 1 C, the first digit indicates the number of remaining characters,
The second digit indicates the number of characters that are not omitted. The result of forward compression of the segmented compression key in (1) of FIG. 1B is not shown in the figure because it is determined by comparison with the previous segmented compression key.

FIG. 2 is a diagram showing the configuration of a database management system to which the present invention is applied.

In FIG. 2, 1 is a controller, 2 is a database, 3 is an index section, 4 is an encoder, 5 is an encoder, 6 is a decoder, 7 is a decoder, 8 is an accessor, 9 is a control line, 10 is a bus,
Reference numeral 11 indicates a data line, and reference numeral 12 indicates an input/output interface. The encoder 4 performs segmented compression as explained in FIG. 1B, the encoder 5 performs forward compression as explained in FIG. The decoder 7 performs the reverse processing of the encoder 5.

Data sent to the controller 1 via the input/output interface 12 is segmented and compressed by the encoder 4 under the control of the controller 1, and is further subjected to matching for forward compression via the encoder 5. While searching for the data above the index section 3. The index section 3 contains sets of keys and addresses of records on the database 2 that have the keys in a format that has been segmented and compressed and then forward compressed. This causes the address of the record to be sent to the accessor 8 via the bus 10 and the target record from the database 2 to the input/output interface 12 via the data line 11, the bus 10 and the controller 1. When storing a new record in the data 2, the address is determined by the accessor 8, and the record is stored in the database 2 via the input/output interface 12, the controller 1, the bus 10, and the data line 11. and the key are subjected to segmentation compression and forward compression via the bus 10, encoder 4 and encoder 5, and are stored in the index section 3. Furthermore, when records belonging to a certain key range are sequentially retrieved, the path of the decoder 7, decoder 6, bus 10, and controller 1 is used.

〔Effect of the invention〕

As is clear from the above description, according to the present invention, it is possible to produce effects such as being able to handle null data items and not having to assume that the compression target is the minimum value of the code.

Conventional technology only focuses on string compression, so it is not suitable for PL/I VARCHAR data processing,
It cannot handle data processing types that are followed by a delimiter such as C's \n. In other words, although the area should be restored, spaces can be ignored when comparing strings. Since it is possible to end the comparison between those with and without characters other than spaces at this point, it is possible to achieve both high-speed processing and data restoration. Of course, since this is an example, it depends on the code system, but spaces can be ignored in string comparisons.

[Brief explanation of drawings]

FIG. 1 is a diagram explaining key compression, and FIG. 2 shows an example of a database management system to which the present invention is applied. 1... Controller, 2... Database, 3
...Index section, 4...Encoder, 5...
encoder, 6... decoder, 7... decoder,
8...accessor, 9...control line, 10...bus, 11...data line, 12...input/output interface.

Claims (1)

[Scope of Claims] 1. A composite key having a plurality of data items of character type data as keys is inserted into an interval of a specified length for each data item, and a separator is inserted at a dividing point, and a separator is inserted into the separator to indicate the previous and subsequent situations. This is a segmentation compression method for character type data that gives a value determined by the data item.When segmenting a data item, if the data item length is not an integral multiple of the specified interval length, it is filled with padding characters, and if the data item is Null. In some cases, this is expressed by a predetermined code with a specified interval length, followed by a separator, and if the separator exists in the middle of the data item and the next character other than the first blank is smaller than the blank, or if the separator is in the middle of a data item and the next first nonblank character is greater than a blank, or the separator is at the end of a data item and then the data item is not null, or the separator is the last character in the data item. Partitioning of character type data characterized by determining whether the next data item existing in the interval is Null or the corresponding data item is Null, and giving a unique value to the separator according to the determination result. Compression method.