JP5762151B2 - Numerical data compression system and method - Google Patents

Description

  The present invention mainly relates to processing of numerical data in a floating-point format that is generally employed in scientific and technical calculation and spreadsheet software by a computer, and efficiently compresses and decompresses numerical input data that is continuously input. The present invention relates to a system, a method, and an apparatus therefor.

  As a method for compressing numerical data in a floating-point format, in the prior art, the data is expressed in the form of “A × Z + R (A and R are real numbers, Z is an integer)”, and the integer portion Z is entropy-coded. Some data compression is performed using a known compression technique such as conversion (Patent Documents 1 to 4). In addition, there is a method (for example, LZ format data compression) in which the integer part Z is compressed by a dictionary method (Non-Patent Document 1).

  In addition to these prior arts, there is a system for compressing a data stream having a numerical value of 64-bit floating-point format using a method called FPC for the 64-bit double-precision floating-point method defined in IEEE754. It exists (Non-Patent Document 2).

FIG. 1 shows an outline of the FPC algorithm as the prior art. Note that FIG. 1 is illustrated in FIG. 1 (FPC compression algorithm overview) is quoted as it is. The FPC uses (i) two predictors “fcm” for each 64-bit data value (“double _b ”) in a linear input sequence (“compressed 1D block of doubles”) of numeric data in IEEE 754 double precision floating point format. And “dfcm” to predict each 64-bit data value, and (ii) compare each predicted data value with the input data value “double _b ” (“compare”), respectively (iii) based on the comparison result The 64-bit value closer to “double _b ” (“closer value”) is selected by “selector”, and (iv) the bit string of input data (“double _b ”) and the bit string of the selected “closer value” An exclusive OR “XOR” is calculated. Next, (v) the leading zero byte part is counted in “leading zero byte counter” for the result, and (vi) “encoder” together with “precoder code” from “selector” based on the count value. The data is compressed and output ("compressed block"). Here, in the compression of the leading 0 byte portion in “encoder”, the number of leading 0 bytes is encoded into a 3-bit value, and is concatenated with a 1-bit “predictor code” to be encoded into 4 bits. The final output “compressed block” consists of the 4-bit code (bit _b + cnt _b ) as well as the uncompressed remainder (“residual _b ”).

Patent Literature 1: Patent No. 4049791 Patent Literature 2: Patent No. 4049792 Patent Literature 3: Patent No. 4049793 Patent Literature 4: Table No. 2004/098066

Non-Patent Document 1: “Reversible compression of IEEE754 floating point signal in ISO / IEC MPEG-4 Audio Lossless Coding (ALS)”, Harada et al., Theory of Science, Vol. J89-B, No. 2, pp. 204-313 , 2006.)

Non-Patent Document 2: M. Burtscher and P. Ratanaworabhan, “FPC: A High-Speed Compressor for Double-Presicion Floating-Point Data,” (http://users.ices.utexas.edu/~burtscher/papers/tc09 .pdf)

  An object of the present invention is to improve the compression ratio for a numerical data stream in a floating-point format that is continuously input, based on a method of compressing numerical data by a value prediction method similar to the FPC. It is.

  Table 1 shows a comparison of data compression rates according to the prior art. In the table, “spreading” is data obtained by calculating the interest rate at a single interest rate of 0.5% for 100,000 years, “science and technology calculation” is the compressed data of fdtd in FIG. 2 described later, and “observation data” is the data of obs_temp It shows the size of the compressed data. As is clear from Table 1 above, particularly in the technical fields of spreadsheets and scientific and technological calculations, compared to FPC (Non-Patent Document 2), the dictionary-based compression method represented by ZIP has a sufficient compression rate. In many cases, the numerical data in the floating-point format as described above is expressed in the form of “A × Z + R (A and R are real numbers, Z is an integer)”. Even in a method (Non-patent Document 1) that performs compression by a method, it cannot be said that the compression rate is sufficient.

  Further, the FPC has a problem that it is difficult to predict numerical data in the double precision floating point format in the first place, and it is difficult to obtain a sufficient compression rate. The prediction difficulty will be further explained from the graph referred to in FIG. In the graph of FIG. 2, the vertical axis is a predictive predictive ratio of predictions performed on various data sets using the algorithm disclosed in the above [Non-Patent Document 2], and the horizontal axis is a double-precision floating point according to IEEE754. The bit position when the sign bit of the format is the first bit is shown. Each data set including “obs_temp” is as disclosed in the above [Non-patent Document 2], and “fdtd” is used in numerical calculation in electromagnetic field analysis. The present inventors have created for simulation. Note that the specific contents of the “fdtd” are out of the scope of the present invention.

  As shown in the figure, in many data, the predictive predictive value decreases at bit positions after the 12th bit, and approaches the predictive predictive value “0.5” (that is, the same result as that predicted randomly). I understand. That is, in the conventional FPC method, compression is performed under conditions where it is difficult to predict data in such a double-precision floating point format, and it is difficult to obtain a sufficient compression rate. Further, in the FPC, the compression processing is performed separately for each floating point format data, which is one factor that limits the compression rate.

  Therefore, the present invention is based on a numerical data compression method based on a value prediction method similar to the FPC in order to solve the problems of the conventional FPC as described above, and further, numerical data in a plurality of floating point formats. Are processed in one set in a given number of units, and the bit string of numerical data is separated into an upper bit string part having a particularly high predictive middle rate and other lower bit string parts. An object of the present invention is to realize a more effective floating point format numerical data compression ratio by independently compressing the upper bit string portion and the lower bit string portion.

  The system of the present invention compresses a plurality of continuously input data based on predicted data for each input data, and the floating point format of the input data and predicted data has a predetermined number of bits. It consists of a binary bit string represented in the order of a sign part, an exponent part, and a mantissa part.

  More specifically, in the system of the present invention, (i) a difference bit string based on a difference (for example, exclusive OR) between a bit string of input data and a bit string of prediction data is stored in a memory in a set of a predetermined number of units. (Ii) from each of the buffered difference bit sequences, one or more higher-order bit sequences consisting of a predetermined number of bits including a bit sequence portion corresponding to the sign part and the exponent part. A reconstructing unit that divides the divided blocks into blocks and rearranges the blocks into groups from the higher order to the lower order, and buffers them as a preceding bit string, (iii) the buffered preceding bit string Therefore, the preceding bit string is compressed and compressed based on the number of consecutive blocks whose value from the first block is zero (0). Previously matrix compression portion forming the leading bit string, is intended to include.

  The system of the present invention solves the technical problem of the FPC method described above, and makes it possible to improve the compression rate even in a situation where it is difficult to predict a floating-point number.

  Further, the reconstruction unit in the system of the present invention further inspects the concatenated preceding bit string for blocks whose values are non-zero (not 0) in order from the upper block, and from the lower block. Blocks whose values are zero (0) in order are inspected, and when the non-zero block and the zero block are detected, they are replaced.

  With this configuration, for example, it is possible to ensure a sufficient compression rate even when vibration data in which the absolute values of the floating-point numbers are substantially equal but only the sign bit is different is input continuously.

  Further, the compression unit in the system of the present invention is configured to divide a residual bit string formed by concatenating the lower bit string obtained by removing the upper bit string from the difference bit string for each set in units of blocks each having a predetermined number of bits. And a residual sequence compression unit that compresses the residual bit sequence based on extracting and deleting blocks including zero (0) or more of a predetermined number of bytes among the blocks divided in units of the block.

  In this way, by performing compression processing on the lower-order bit string, that is, the mantissa part in the numerical data in the floating-point format, the compression ratio of the floating-point number can be further improved. This is particularly advantageous when easily predictable data is input such that the same floating point number is input continuously.

FIG. 1 is a block diagram showing an outline of a conventional FPC system. FIG. 2 is a graph showing the difficulty in predicting numerical data in the double precision floating point format of the prior art. FIG. 3 is a diagram showing a display format of double-precision floating point data standardized by IEEE754. FIG. 4 is a block diagram showing an outline of the compression method in the first embodiment of the present invention. FIG. 5 is an image diagram for generating a differential bit string D _n from the input data _An and the prediction data P _n . FIG. 6 is a block diagram showing details of the compressor 5 in the compression method according to the first embodiment of the present invention. FIG. 7 is an image diagram showing the formation of the preceding bit string and the remaining bit string of the reconstruction unit based on the block rearrangement in the compression method of the first embodiment of the present invention. FIG. 8 is an image diagram showing the formation of the compressed preceding bit string of the previous matrix compression unit. FIG. 9 is an image diagram showing block replacement in the address reconstruction unit of the compression method according to the second embodiment of this invention. FIG. 10 is a more detailed image diagram related to the block replacement process in the address restructuring unit in the second embodiment of the present invention. FIG. 11 is a flowchart showing a block replacement process in the address restructuring unit in the second embodiment of the present invention. FIG. 12 is a block diagram showing an outline of the improvement of the compression system in the third embodiment of the present invention. FIG. 13 is an image diagram showing two difference value calculations based on two predicted values in the third embodiment of the present invention. FIG. 14 is an image diagram showing a comparator output based on two difference values in the third embodiment of the present invention. FIG. 15 is an image diagram showing compression of the preceding bit string in the third embodiment of the invention. FIG. 16 is a block diagram showing a compression method of numerical data in the floating point format according to the fourth embodiment of the present invention. FIG. 17 is an image diagram illustrating the formation of the residual bit set sequence R in the residual sequence compression unit according to the fourth embodiment of the present invention. FIG. 18 is an image diagram showing a residual bit set sequence R ′ that is compressed by the residual sequence compression unit according to the fourth embodiment of the present invention.

  A compression / decompression system according to an embodiment of the present invention will be described with reference to the drawings. The following embodiments are merely examples, and the invention described in the claims is not limited thereto.

  FIG. 3 is a schematic diagram for displaying numerical data in the double precision floating point format standardized by IEEE 754 as a binary bit string. Data in the double precision floating point format standardized by IEEE 754 is expressed based on the following calculation formula.

  Here, “S” indicates a sign part, “exp” indicates an exponent part, and “fraction” indicates a mantissa part. In the binary display format as shown in FIG. Bits and 52 bits are formed from a total of 64 bits.

  In the following, each embodiment will be described based on numerical data in the double precision floating point format standardized by IEEE 754, but the present invention is limited to numerical data in 64-bit double precision floating point format in addition to what is particularly noted. Not only the 32-bit single precision floating point format numerical data similarly standardized by IEEE 754, but also other multi-precision floating point format numerical data (for example, 128 bits) The present invention can be applied to all numerical data in a floating-point format represented by a binary bit string composed of a sign part, an exponent part, and a mantissa part in order from the upper part to the lower part.

Embodiment

First Embodiment FIG. 4 is a block diagram showing an outline of a compression system of the present invention. The central processing unit of the computer operates as follows in each component while performing memory operations including a register and a cache memory with respect to a plurality of numerical data in a floating point format that are input.

The input string 1 indicates N pieces of input data A _n (n = 1, 2,..., N) in a floating point format that are continuously input (N is an integer of 1 or more). The input data _An is sequentially input to the differentiator 4 and is also input to the predictor 3 that is delayed by one clock in the delay unit 2 and combined from the delay unit 2. In the predictor 3, the input data A _{0 to} A _(n−1) that have already been input are synchronized with the timing at which the input data _An is input to the differentiator 4 (where A ₀ precedes the input string _). Prediction data P _n in a floating-point format predicted from (virtual data) is predicted and input to the difference unit 4 coupled from the predictor 3. The differentiator 4, the entered input data A _n and predicted difference of each bit string of the predicted data P _n (e.g., exclusive OR) is calculated, and outputs as a difference bit string D _n.

As shown in FIG. 5, the prediction data P _n here is predicted from the input data A _{0 to} A _(n−1) , for example, by applying a known technique disclosed in Non-Patent Document 2. In addition, in the bit string representation of the difference bit string D _n , it is assumed that the higher-order bit having a higher predictive predictive value has a higher appearance probability of “0” (see also FIG. 2).

In the compressor 5 coupled to the differentiator 4, the compression processing of the present invention described in detail in FIG.
(M is an arbitrary natural number).

  FIG. 6 is a block diagram showing details inside the compressor 5 in the compression method of the present invention.

Compressor 5, a plurality of difference bit string D _n a predetermined number set for each differential bit string buffer 51, buffering for buffering in a memory a set of units of (M-number in this case) from the differentiator 4 For D _n , an upper bit string consisting of a predetermined number of bits including a bit string portion corresponding to a sign part and an exponent part in a floating-point format is divided into one or more blocks consisting of a predetermined number of bits, and the divided The blocks are rearranged for each pair in order from the top to the bottom, concatenated, buffered as a preceding bit string, and a bit string formed by concatenating the lower bit string obtained by removing the upper bit string from the difference bit string for each group is a residual bit string A reconstructing unit 52 that performs buffering, and a pre-matrix compression unit 53 that performs compression processing on the preceding bit string and outputs the result Eteiru.

  Further, the reconstructing unit 52 in FIG. 6 will be described. In the form shown in FIG. 7, the bit string of each data is converted into the upper 8 bits (1 byte), 4 bits (1/2 byte), 4 bits (1/2 Byte) and the other 48 bits (12 bytes) in that order, and the upper bit string block (8 bits + 4 bits + 4 bits) and the lower bit string (48 bits) are rearranged in units of groups. An address restructuring unit 521 for performing an address operation, which is connected in order from the block, and buffered in a memory, a destination matrix buffer 522 for buffering a preceding bit sequence, and a residual sequence buffer 523 for buffering a residual bit sequence are provided.

  As shown in the graph of FIG. 2, this configuration has a relatively high predictive predictability for the upper several bytes of the predicted data that are predicted for each of a plurality of input floating-point numbers that are successively input. In particular, the compression rate is improved compared to conventional FPC by compressing the portion of the predetermined M number of difference bit strings after recombining them in units of blocks. It is.

  Here, the configuration of the preceding bit string and the remaining bit string shown in FIG. 7 will be described in more detail. In addition, although it demonstrates as M = 4 here for simplicity, it is not limited to this, M can be made into arbitrary natural numbers.

Of the plurality of data input from the differentiator 4, a bit string (D _{n to} D _{(n + 3)} ) of four (that is, M) data input in succession is buffered in the buffer unit 51. Here, the upper 2 bytes (16 bits) of each bit string include a total of 12 bits including a sign part (upper 1 bit) and an exponent part (11 bits) subsequent thereto. In the reconstructing unit 52, the bit string of each data is a high-order bit string consisting of high-order 16 bits (2 bytes) including a sign part (1 bit) and an exponent part (11 bits), and the other low-order bits of 48 bits (6 bytes) The upper bit string is further divided into a first block of 8 bits (1 byte) and two second and third blocks of 4 bits (1/2 bytes). Here, the division into two second and third blocks is that a total of 12 bits of the first block and the second block is a total of 12 of a sign part (upper 1 bit) and an exponent part (11 bits). This is to match the bits. In FIG. 7, the first block is blocks [1] to [4], the second block is blocks [5] to [8], and the third block is blocks [9] to [12]. The lower block is composed of a 48-bit (6 bytes) lower bit string as it is (blocks I to IV).

Further, as shown in FIG. 7, the reconstruction unit rearranges each block in units of sets for the upper bit string and the lower bit string and concatenates them in order from the upper block, so that the 64-bit preceding bit string L _i and the 192-bit remaining bit string R formed as _i and buffered in the memory (first matrix buffer 522 and residual column buffer 523). That is, the formed preceding bit string L _i includes the first block (blocks [1] to [4]), the second block (blocks [5] to [8]), and the third block (blocks [9] to [12]. ]) In order. Similarly, the remaining bit string R _i formed is connected in the order of the blocks I to IV.

Next, the preceding bit string L _i is input to the preceding matrix compression unit 53, and a compressed preceding bit string (ζ + U _i ) is formed as shown in FIG. That is, regarding the formed 64-bit preceding bit string L _i , one block is one byte (that is, 8 bits), and a continuous block portion Z _i whose value is zero (0) from the first block and the other block portions U _i And the number of consecutive blocks (number of bytes) z of Z _i is counted. Based on the continuous block number z, the preceding bit string L _i is compressed to form a compressed preceding bit string. When M = 4, for example, by referring to the table shown in Table 2 below, 8 × z bits are compressed to 3 bits.

For example, when the entire preceding bit string L _i is zero (0), the 64-bit preceding bit string L _i can be compressed to the maximum to 3 bits. When M is a factorial of 2 (that is, M = 2 ^m ), the preceding bit string L _i of M × 16 (bits) can be compressed to the maximum (m + 1) bits.

In the example of FIG. 8, the number z of consecutive blocks of Z _i is “2”. The 3-bit ζ at this time is expressed as “001” from Table 2. That is, a 64-bit (preceding bit string L _i ) bit string can be compressed to a total of 51 bits of 3 bits (“001”; ζ) +48 bits (non-zero block; U _i ).

In a second embodiment according to the second embodiment the present invention, in order to improve the compression ratio of the preceding bit sequence L _i in the first embodiment, the address reconstruction unit 521, further, the blocks shown in FIGS. 9 to 11 The replacement process is performed.

  When implementing a configuration relating to division and recombination of a plurality of numerical data in the floating-point format according to the first embodiment, compression on vibration data in which only the bits of the sign part are different, although the absolute values are substantially equal. The problem that it may become inadequate can be solved.

As shown in FIG. 9, the preceding bit string L _i is checked for non-zero blocks in order from the upper block, and is checked for zero blocks in order from the lower block, and each block exists and is detected. In this case, the blocks are replaced. As a result, the continuous block portion Z _i having a value of zero (0) can be expanded from the first block, and the other block portions U _i can be shortened.

The decompressed block portion Z ′ _i may be compressed into a total of 12 bits ζ ′ by adding 1 bit of the replacement flag and a bit string (8 bits) indicating the position of the replacement block to the 3 bits ζ of Table 2 described above. it can. Eventually, the preceding bit string L _i can be compressed into a block in which U ′ _i after block replacement is connected to the 12-bit ζ ′ (when M = 4). The bit position indicating the replacement position here is a 2 × M bit length bitmap, and the bit corresponding to the replaced block position is set to “1”.

Figure 10 shows another embodiment of the prior bit sequence L _i in the second embodiment. This embodiment will be described with reference to the flowchart of FIG. Also in FIG. 10, M = 4, one block is composed of 1 byte, and the preceding bit strings L _i and L ′ _i are displayed in hexadecimal for simplicity.

The prior bit sequence _{L i,} to inspect the non-zero blocks from upper block (Block 1) in this order (S100). Here, block 2 is “80” and is detected as a non-zero block (S110). Then, for the preceding bit string L _i , the zero block is checked in order from the lower block (block 8) (S120). Here, block 8 is “00” and is detected as a zero block (S130). In this case, since the relationship between the inspection block positions satisfies “2 <8” (S140), the block 2 “80” and the block 8 “00” are switched (S150). The bit is set to “1” (S160), and the second bit and the eighth bit of the replacement block position bit string are set to “1” (S170). As a result of this processing, ζ and L _i are updated as ζ ′ and L ′ _i shown in FIG.

Subsequently, the inspection is repeated, and the non-zero blocks are inspected sequentially from the block 3 to detect “01” of the block 5 (S100 and S110). Further, the zero block is inspected in order from the block 7, and “00” of the block 6 is detected (S120 and S130). Since the relation between the inspection block positions is “5 <6” (S140), similarly, the operation is performed so that these blocks are replaced (S150), and the replacement flag bit is set to “1” (S160). Further, the fifth bit and the sixth bit of the replacement block position bit string are respectively set to “1” (S170). The result of this process, zeta ', and L' _i is updated as zeta '' and L '' _i shown in FIG. 10.

  Subsequently, the inspection is repeated to detect the block 6 as a non-zero block and the block 5 as a zero block (S100 to S130), but the relationship between the inspection block positions becomes “6> 5” (S140). This means that it has already been completed, and the repeated inspection process is exited, and the replacement process is completed (END).

By such a replacement process, in the first embodiment, the number of blocks z = 1 of the continuous block portion Z _i whose value is zero (0) from the first block is different from the first block in the second embodiment. The number of blocks z ″ of consecutive block portions Z ′ _i having a value of zero (0) can be set to 5 and the compression rate is successfully improved.

  As described above, in this embodiment, the compression rate can be further improved compared to the first embodiment. In addition, each step of the said replacement process is only an example, and is not limited to this. In other words, the processing may be performed in any manner as long as the processing can replace the blocks and update the replacement flag / bit.

Third Embodiment A third embodiment according to the present invention mainly includes a plurality of input data _An and a plurality of predicted predictions using the predictor 3 and the differentiator 4 in the first embodiment. An alternative embodiment of a configuration in which a plurality of difference bit strings D _n are output from the difference (for example, exclusive OR) of each bit string of data P _n is shown.

As shown in FIG. 12, in this embodiment, two types of prediction values are calculated by using two different predictors 31 and 32 by two different methods. In the differentiators 41 and 42, two difference values are calculated and output to the compressor 5 '. Then, the comparison unit 50 provided in the compressor 5 ′ selects a difference value from the two difference values based on a predetermined rule, and a buffer unit 51 ′ (not shown) together with a selection bit indicating which difference value is selected. To output internally. As a result, the compressed output C ′ _i from the compressor 5 ′ also includes a selection bit string (selection signal) of M bits (4 bits in this case).

  Hereinafter, it demonstrates in detail using FIGS. 12-15.

In the first embodiment, as shown in FIG. 5, the input data _An is predicted from the input data A _{0 to} A _(n−1) that have been input in synchronization with the timing at which the input data _An is input to the comparator 4. Prediction data P _n is predicted by the predictor 3, and a difference (for example, exclusive OR) between each bit string of the input data _An and the prediction data P _n is calculated by the difference unit 4 and output as a difference bit string D _n doing. On the other hand, in this embodiment, as shown in FIGS. 12 and 13, the prediction data P _n and Q _n are similarly predicted separately using the two predictors 31 and 32, and the two predictors 31 are used. , 32 are respectively used to calculate the difference (for example, exclusive OR) of each bit string between the input data _An and the prediction data P _n and Q _n using the two differentiators 41 and 42 respectively coupled to the difference data. respectively output to the compressor 5 'as a bit string _{D n} and _{E n.}

The compressor 5 ', first, the comparison unit 50, performs the comparison and selection of the difference bit string _{D n} and _{E n.} In the comparison / selection, processing is performed by dividing into blocks each having a predetermined number of bits. The number of bits here may be 1 byte (that is, 8 bits) or 1 bit, and may be any number. be able to. Here, it is assumed that blocks are formed in units of bytes.

Composed of eight blocks (block 1-8) in FIG. 14 shows a specific example of the 64 two difference bit string _{D n} and _{E n.} Note that the value of each block is displayed in hexadecimal for simplicity. The comparison unit 50 counts the number b of zero blocks whose value is zero (0) for each difference bit string, and selects the difference bit string having the larger number of zero blocks b. In the case of FIG. 14, from the value of each zero block number _b d (= 4) and _b e (= 2) of the difference bit string _{D n} and _{E n,} the difference bit string _{D n} having a larger value as the selected output _{F n} Output. At that time, either 1-bit selection signal S _n which indicates whether to select the difference bit string, for example, if you select the D _n as S _{n =} 1 If you select S _{n =} 0, E _n, together Output.

  Thus, the compression rate in the subsequent compression processing can be improved by configuring the compressor so that the comparison unit selects the difference bit string for the prediction accuracy higher in the predictors 31 and 32. .

Figure 15 is the compressor 5 'of the present embodiment, illustrates the configuration of the compression of the preceding bit sequence L _i of 64 bits buffered in the leading column buffer 522. Here, it is assumed that M = 4, and the preceding bit string _Li itself is configured in the same manner as described in the first embodiment. Also, ζ and U _i formed from the preceding bit string L _i are the same as in the first embodiment (see FIG. 8). The preceding bit string L _i can be compressed by adding 1 × 4 bits (that is, M bits) of selection signals S _{n to} S _{(n + 3)} to this ζ.

  The present invention can further improve the data compression rate by combining the configuration relating to the selection of the difference bit string in this embodiment with the configuration relating to the block replacement for the preceding bit string in the second embodiment.

Note that the compare-select the difference bit string D _n and E _n described above, instead of carrying out the comparison and selection by the comparison part 50 arranged to be coupled to the differentiator 41 and 42, the difference bit string D _n and executes processing and buffering at the buffer portion and the reconstruction unit for both E _n, and, by placing the comparing unit 50 to couple the comparator unit 50 from the leading column buffer, in the comparison unit 50 A similar comparison / selection process may be performed on the previous matrix buffer.

  In the first to third embodiments described above, block division is performed in units of bytes (that is, in units of 8 bits) with respect to a bit string of numerical data in the double-precision floating point format defined in IEEE754. The present invention is not limited to this, and as specified in IEEE 754, numerical data in a floating-point format is expressed using a sign part, an exponent part, and a mantissa part, and a binary bit string is displayed from the upper part. As long as it is configured in this order, it can be realized for arbitrary data, and the unit of the block shown in FIG. As long as it is included, it is not limited to a byte unit, and may be an arbitrary number of bits. That is, a divided block may be constituted by the number of bits of the sign part and the exponent part, and further, one block is constituted by a divisor of the number of bits of the sign part and the exponent part, and the sign part and the exponent part are included in a plurality of blocks. It may be an upper bit string.

Fourth Embodiment In the first to third embodiments described above, the compression process is performed on the preceding bit string. In the fourth embodiment, the compression process is further performed on the remaining bit string.

  The present invention pays attention to the difficulty of predicting the numerical data in the floating-point format as described above, and improves the compression rate of the numerical data in the floating-point format on the assumption of this.

  On the other hand, when such a compression method is applied to data that is generally easily predictable such that data of the same value continues, the prior art as described in the prior art document is applied. It is also assumed that the compression rate is equal to or lower than that in the case of the above.

  Therefore, in this embodiment, in order to improve the compression rate even when such easily predictable numerical data in the floating-point format is input, the compression method described below is applied to the remaining bit string. By doing so, the compression rate is improved.

  FIG. 16 is a block diagram showing a compression system for numerical data in the floating point format according to this embodiment. Compared to FIG. 6, a residual string compression unit 54 is added. The residual sequence compression unit 54 receives the input of the residual bit sequence buffered in the residual sequence buffer 523, performs a compression process, and outputs a compressed output. In addition, about each component other than the residual sequence compression part 54, it can implement | achieve based on the above-mentioned Embodiment 1-3.

Hereinafter, a compression method for the residual bit string R _i in the residual string compression unit 54 will be described with reference to FIGS. 17 to 18. As described above, the remaining bit string R _i is composed of M 48-bit (that is, 6-byte) bit strings (see FIG. 7).

FIG. 17 shows the remaining bit string R _i in the remaining string compression unit 54.
Are buffered and connected together to form a set sequence R of remaining bits. Here, M 6-byte sequences in the remaining bit sequence R _i are r _{(i, 1)} to r _{(i, M)} . The remaining bit set sequence R formed in this way has a length of 6 × N bytes. For the remaining bit set sequence R, as shown in FIG.
Among them, zero (0) continuous 6 bytes (that is, the value is 0) or zero (0) containing 5 bytes are extracted, and a position sequence H indicating the extracted position is formed. For r _{(i, j)} containing 5 bytes of zero (0), the position of the non-zero (not 0) byte in r _{(i, j)} and its bit string are stored separately. In FIG. 18, r _(i0,1) and r _{(i1,3) correspond} to a case in which zero (0) is continuous for 6 bytes or a case in which zero (0) is included in 5 bytes. _A position sequence H is generated as ₁ and H ₂ .

More specifically, a position row H having values H ₁ = M × (i ₀ −1) +1 and H ₂ = M × (i ₁ −1) +3 is formed. Here, each component of the position sequence H has a length of _nh bytes. n _h is a capacity necessary for storing the position, and is smaller than 6 bytes.

On the other hand, the residual bit set sequence R is compressed by deleting these r _(i0,1) and r _(i1,3) from the residual bit set sequence R, thereby forming a compressed residual bit set sequence R ′. .

  As a result, the residual bit set sequence R that was initially 6 × N bytes long can be compressed into a compressed residual bit set sequence R ′ having a length of (6 × N−2 × 6) bytes. The compression residual bit set sequence R ′ and the position sequence H are output as compressed outputs to complete the compression process.

  The present invention implements the compression process for the residual bit string described in the present embodiment in combination with the preceding bit string compression process described in the first to third embodiments, so that the numerical data in the floating-point format that is easy to predict can be obtained. This is effective for continuous input.

Note that the position sequence H in the present embodiment has the absolute position of a specific r _{(i, j)} as a constituent element, and each constituent element is configured with a value smaller than 6 bytes. From an implementation point of view, each value can be expressed by 1 byte as a relative position, not an absolute position, and the relative position, the position of the non-zero byte r _{(i, j)} , the non-zero byte r _{(i , J)} can also be output as a non-zero byte r _{(i, j),} with 3 bytes of the bit pattern as a set. Furthermore, when the same value is continuously included in the relative position row, for example, by applying a compression method by a run length method, it is possible to further compress numerical data.

Heretofore, the compression method for a plurality of numerical values in the floating point format that are successively input according to the first to fourth embodiments of the present invention has been described. As can be seen from the description of these embodiments, the present compression method is a reversible compression method, and can be applied to input data A _n (n = 1, 2,..., N).
, The compressed output C _i is expanded and the input data A _n (n = 1, 2,..., N) is restored by performing the reverse operation to the compression operation in the order of i = 1, 2,. It is possible.

Compression result

   Table 3 is obtained by adding data obtained by implementing and acquiring the data based on one embodiment of the present invention to Table 1 described above. As shown in Table 3, the compression method of the floating point data of the present invention can realize a relatively improved compression rate as compared with the FPC method of the prior art.

Industrial application fields

  Since the compression and decompression system of the present invention can effectively compress numerical input data of a plurality of floating-point formats that are successively input and handle the data, the large-scale data is particularly doubled as defined in IEEE754. This is advantageous for applications that need to be processed in precision floating point format and stored in memory.

  Specifically, (i) when storing large-scale data stored in a cell in spreadsheet software as numerical data in a floating-point format, (ii) when storing calculation progress of scientific and technological calculation software in main memory, (Iii) When compressing and storing a log of real-time data processed by a flight recorder or drive recorder, when communication of large-scale data is required in a narrow-band communication path such as communication from a space observing device to the earth The system and method according to the present invention will be particularly effective.

Claims (7)

A system for compressing a plurality of input data that are continuously input based on prediction data for each of the input data,
The floating point format of the input data and the prediction data is configured to include a binary bit string represented in the order of a sign part having a predetermined number of bits, an exponent part, and a mantissa part,
A buffer unit that buffers a difference bit string of each of the bit string of the input data and the bit string of the prediction data in a set of a predetermined number of units;
Dividing a high-order bit string consisting of a predetermined number of bits including a bit string portion corresponding to the sign part and the exponent part from each of the buffered difference bit strings into one or more blocks consisting of a predetermined number of bits, And a reconstructing unit that buffers the divided blocks by rearranging them for each set in order from the top to the bottom, thereby buffering them as a preceding bit string , and further, for the preceding bit string that has been linked If the block whose value is non-zero in order from the upper block and the block whose value is zero in order from the lower block are inspected, and the non-zero block and the zero block are respectively detected, A reconstruction unit configured to replace them ;
A pre-matrix compression unit that compresses the preceding bit string based on the number of consecutive blocks in which the value from the first block is zero with respect to the buffered preceding bit string, and forms a compressed preceding bit string. The system of claim 1, wherein
The system, wherein the compression preceding bit string formed in the prior matrix compression unit further includes a bit string indicating a position of the replaced block. 3. The system according to claim 1 or 2, wherein the prediction data includes first prediction data and second prediction data, the system further comprising:
The difference bit string buffered in the buffer unit is based on a first difference bit string and a second difference bit string that are the differences between the bit string of the input data and the bit string of the first prediction data and the second prediction data. A comparison unit selected and output to the buffer unit;
The selection is performed based on the number of blocks having the predetermined number of bits included in each of the first differential bit string and the second differential bit string. The system according to any one of claims 1 to 3, wherein the input data and the prediction data are 64-bit double-precision floating-point format data defined in IEEE754.
The blocks divided by the reconstruction unit are in units of bytes, and the upper bit string of the difference bit string is a 1-byte first block, a 1 / 2-byte second block, and a 1 / 2-byte first block. A system including being divided and formed in the order of three blocks. The system according to any one of claims 1 to 3,
The upper bit string comprises a bit string portion corresponding to the code portion and the exponent part, the number of bits in each block divided by the reconstruction unit is determined based on the number of bits of the upper bit string, the system . The system according to any one of claims 1 to 5,
The reconstructing unit further divides a residual bit string formed by concatenating a lower bit string obtained by removing the upper bit string from the difference bit string for each set in units of a fourth block having a predetermined number of bits. In addition,
A system comprising: a residual sequence compression unit that compresses the residual bit sequence based on extracting and deleting a block including zero or more of a predetermined number of bytes among the blocks divided in units of the fourth block . A method of compressing a plurality of input data continuously input based on prediction data for each of the input data,
The method is implemented by a computer;
The floating point format of the input data and the prediction data is configured to include a binary bit string represented in the order of a sign part having a predetermined number of bits, an exponent part, and a mantissa part,
A buffer step of buffering a difference bit string of each of the bit string of the input data and the bit string of the prediction data in a memory in a set of a predetermined number of units;
Dividing each of the buffered difference bit sequences into an upper bit sequence consisting of a predetermined number of bits including a bit sequence corresponding to the sign part and the exponent part into one or more blocks consisting of a predetermined number of bits; and Reconstructing step of buffering as a preceding bit string by recombining the divided blocks for each set in order from the top to the bottom and performing concatenation;
A pre-matrix compression step for compressing the preceding bit string based on the number of consecutive blocks whose value from the first block is zero for the buffered preceding bit string,
The reconstructing step further inspects the concatenated preceding bit string for blocks having a non-zero value in order from the upper block, and for blocks having a value of zero in order from the lower block, A method, characterized in that the non-zero block and the zero block are each configured to be replaced if detected.