CN105630999A - Data compressing method and device of server - Google Patents
Data compressing method and device of server Download PDFInfo
- Publication number
- CN105630999A CN105630999A CN201510999354.6A CN201510999354A CN105630999A CN 105630999 A CN105630999 A CN 105630999A CN 201510999354 A CN201510999354 A CN 201510999354A CN 105630999 A CN105630999 A CN 105630999A
- Authority
- CN
- China
- Prior art keywords
- data
- compression algorithm
- compression
- efficacy parameter
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
- H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
Abstract
The invention discloses a data compressing method and device of a server and belongs to the field of data storage. The method comprises the steps that the server uses a first compression algorithm to compress first data so as to obtain second data; the server uses a second compression algorithm to compress second data sample data so as to obtain third data. The problem of poor compression effect when the single-type compression algorithms are adopted in the prior to compress data in an internal storage is solved, and the effect of improving the compression effect is achieved.
Description
Technical field
The present invention relates to field of data storage, in particular to method and the device of a kind of server compresses data.
Background technology
In order to meet the processing demands of big data, the amount of ram in storing device is also progressively increasing. But the amount of ram in storing device is fixed often, in actual use, the problem that internal memory overflows still may occurs.
Data compression is a kind of effective ways alleviating memory storage pressure, and the method usually compressed by stored data is: adopt a kind of predetermined compression algorithm that the data in internal memory are carried out processed compressed.
When adopting the compression algorithm of single type the data in internal memory to be compressed, compression effect is poor.
Summary of the invention
In order to solve the problem of prior art, the embodiment of the present application provides method and the device of a kind of server compresses data. Described technical scheme is as follows:
First aspect, it provides a kind of method of server compresses data, the method comprises:
First data are carried out compression and obtain the 2nd data by described server use first compression algorithm;
Described 2nd data sampled data is carried out compression and obtains the 3rd data by described server use the 2nd compression algorithm.
First data are carried out compression with the use of the first compression algorithm and obtain the 2nd data, it may also be useful to described 2nd data sampled data is carried out compression and obtains the 3rd data by the 2nd compression algorithm by the method for the server compresses data that the embodiment of the present application provides; Owing to compression algorithm can be utilized the data after compression again to be compressed, make compression better effects if, therefore solve in prior art and when adopting the compression algorithm of single type the data in internal memory to be compressed, compress the poor problem of effect, reach the effect improving compression effect.
In conjunction with first aspect, in the first possible realization of first aspect, described method, also comprises:
The first sampled data that described server is sampled and obtained from described first data;
Described server use in multiple compression algorithm each compression compression algorithm described in the first sampled data and to described first sampled data compress after data decompression to determine the first efficacy parameter; Described first efficacy parameter be used to indicate each compression algorithm described first sampled data is compressed or decompress described first sampled data compress after data time effect;
Described server determines that the best compression algorithm of the first efficacy parameter is as described first compression algorithm.
The method of the server compresses data that the embodiment of the present application provides, the first sampled data is obtained by the first data are carried out sampling, obtain the first efficacy parameter that multiple compression algorithm is corresponding when the first sampled data is carried out compression and decompression, select the compression algorithm that the first efficacy parameter is best, so that when the compression algorithm utilizing the first efficacy parameter best is to the first data compression, it is possible to reach and compress effect preferably.
In conjunction with the first possible realization of first aspect or first aspect, in the 2nd kind of possible realization, described method, also comprises:
The 2nd sampled data that described server is sampled and obtained from described 2nd data;
Described server use in multiple compression algorithm each compression compression algorithm described in the 2nd sampled data and to described 2nd sampled data compress after data decompression to determine the 2nd efficacy parameter; Described 2nd efficacy parameter be used to indicate each compression algorithm described 2nd sampled data is compressed or decompress described 2nd sampled data compress after data time effect;
Described server determines that compression algorithm corresponding to the 2nd best efficacy parameter is as described 2nd compression algorithm.
The method of the server compresses data that the embodiment of the present application provides, the 2nd sampled data is obtained by the 2nd data are carried out sampling, obtain the 2nd efficacy parameter that multiple compression algorithm is corresponding when the 2nd sampled data is carried out compression and decompression, select the compression algorithm that the 2nd efficacy parameter is best, so that when the compression algorithm utilizing the 2nd efficacy parameter best is to the 2nd data compression, it is possible to reach and compress effect preferably.
In conjunction with the first possible realization of first aspect, first aspect or the 2nd kind of possible realization of first aspect, in the realization that the third is possible, the expected value of described first compression algorithm is maximum in the expected value of described multiple compression algorithm; Wherein, the expected value of described first compression algorithm is obtained according to the first efficacy parameter weight calculation of the first efficacy parameter of described first compression algorithm and described first compression algorithm according to preset expected value-based algorithm by described server.
The method of the server compresses data that the embodiment of the present application provides, consider many-sided efficacy parameter and draw expected value, compression algorithm maximum for expected value is defined as the first compression algorithm, ensure that and can obtain multiple compression algorithm compresses the best compression algorithm of effect.
In conjunction with the first possible realizing to any one in the third possible realization of first aspect of first aspect, first aspect, in the 4th kind of possible realization, the expected value of described 2nd compression algorithm is maximum in the expected value of described multiple compression algorithm; Wherein, the expected value of described 2nd compression algorithm is obtained according to the 2nd efficacy parameter weight calculation of the 2nd efficacy parameter of described first compression algorithm and described 2nd compression algorithm according to preset expected value-based algorithm by described server.
The method of the server compresses data that the embodiment of the present application provides, consider many-sided efficacy parameter and draw expected value, compression algorithm maximum for expected value is defined as the 2nd compression algorithm, ensure that and can obtain multiple compression algorithm compresses the best compression algorithm of effect.
Second aspect, it provides the device of a kind of server compresses data. The device of these server compresses data comprises at least one unit, and each unit of the device of these server compresses data is respectively used to realize step corresponding in the method for the server compresses data of above-mentioned first aspect.
The third aspect, it provides a kind of server. This server comprises: treater, the storer being connected with treater and network interface, and this treater is for each step in the method realizing the server compresses data of above-mentioned first aspect.
Fourth aspect, it provides a kind of computer-readable medium, this computer-readable medium stores the instruction for the method realizing the server compresses data that first aspect provides.
Accompanying drawing explanation
In order to the technical scheme being illustrated more clearly in the embodiment of the present invention, below the accompanying drawing used required in embodiment being described is briefly described.
Fig. 1 is the structural representation of the server that the exemplary embodiment of the present invention one provides;
Fig. 2 A is the schema of the method for the server compresses data that the exemplary embodiment of the present invention one provides;
Fig. 2 B is the schema of the method obtaining the first compression algorithm that the exemplary embodiment of the present invention one provides;
Fig. 2 C is the schema of the method obtaining the 2nd compression algorithm that the exemplary embodiment of the present invention one provides;
Fig. 3 is the structural representation of the device of the server compresses data that the exemplary embodiment of the present invention one provides.
Embodiment
For making the object, technical solutions and advantages of the present invention clearly, below in conjunction with accompanying drawing, embodiment of the present invention is described further in detail.
" module " mentioned herein refers to the program that can realize some function or instruction that store in memory; Refer to, at " unit " mentioned in this article, the functional structure logically divided, should " unit " can by pure hardware implementing, or, the combination of software and hardware realizes.
Please refer to Fig. 1, it illustrates the structural representation of the server 110 that the exemplary embodiment of the present invention one provides. This server 110 comprises: treater 11, network interface 12 and storer 13.
Treater 11 comprise one or more than one process core, treater 11 is by runs software program and module, thus performs the application of various function and data processing.
Network interface 12 can be multiple, and wherein part network interface 12 is for communicating with other equipment.
Storer 13 is connected with treater 11, and such as, storer 13 can be connected with treater 11 by bus; Storer 13 can be used for storing software program and module.
Storer 13 can store the application program module 14 needed at least one function, and application program module 14 can comprise the first determination module 141, the 2nd determination module 142 and compression module 143 etc.
Here algorithm first determination module the 141, the 2nd determination module 142 and compression module 143 module can perform the corresponding steps in Fig. 2 A, Fig. 2 B and Fig. 2 C, referring specifically to the description to 2A, Fig. 2 B and Fig. 2 C.
Storer 14 can be realized by the volatibility of any type or non-volatile memory device or their combination, as static RAM is (English: staticrandomaccessmemory, SRAM), electrically erasable read-only storage is (English: electricallyerasableprogrammableread-onlymemory, EEPROM), erasable programmable read-only storage is (English: erasableprogrammablereadonlymemory, EPROM), programmable read only memory is (English: programmablereadonlymemory, PROM), read-only storage is (English: readonlymemoryimage, ROM), magneticstorage, flash device, disk or CD.
It will be appreciated by those skilled in the art that, the structure of the server 110 shown in Fig. 1 does not form the restriction to server 110, it is possible to comprises the parts more more or less than diagram, or combines some parts, or different parts are arranged.
Please refer to Fig. 2 A, it illustrates the schema of the method for the server compresses data that the exemplary embodiment of the present invention one provides, perform following step by the treater 11 of server 110 as shown in Figure 1, the method comprises following step:
Step 202, the first data are carried out compression and obtain the 2nd data by server use first compression algorithm.
Generally, the first data can be the data to be compressed in server, it is also possible to is by the data after other compression compression algorithm in server.
Here the data to be compressed said refer in server without the data compressed.
Before step 202, in order to best compression effect can be known, when the first data are compressed, server needs to obtain the first best compression algorithm of compression effect from multiple compression algorithm, and the process obtaining the first compression algorithm can see the step 201a shown in Fig. 2 B to step 201c.
Step 201a, server is sampled from the first data and is obtained the first sampled data.
The mode that first data sampling obtains the first sampled data is not specifically limited by the present embodiment, can determine according to concrete performance.
Such as, it is possible to every predetermined the first data, first sampled data of sampling, it is also possible to multiple first sampled data of continuous sampling from the first data.
Step 201b, server use in multiple compression algorithm each compression compression algorithm first sampled data and to first sampled data compress after data decompression to determine the first efficacy parameter. Wherein, the first efficacy parameter be used to indicate each compression algorithm the first sampled data is compressed or decompresses first sampled data compression after data time effect.
In general, when data after utilizing a certain compression algorithm the first sampled data to be compressed or the first sampled data compressed decompress, first efficacy parameter corresponding with this compression algorithm can be obtained, here the first efficacy parameter said not only is determined by this compression algorithm, is also subject to the impact of multiple restraining factors. Here the restraining factors said comprise: computing power of the performance data feature of the data structure feature of the first sampled data, the data statistical characteristics of the first sampled data, the first sampled data, the Memory Allocation of server, the internal memory capacity of server, the internal memory behaviour in service of server and server etc.
Owing to each compression effect of the first data being compressed of algorithm affects by multiple restraining factors, it is necessary to just can obtain utilizing, by the calculating of comparatively complexity, the effect that the first data are compressed by this compression algorithm. And in actual mechanical process, when data after using different servers the first data to be compressed or the first sampled data compressed decompress, owing to the computing power of server, internal memory capacity, internal memory behaviour in service etc. exist difference, when the first identical data are carried out compression and decompression by different server by utilizing identical compression algorithm, also can there is difference in compression effect.
Therefore, the first sampled data is obtained in the present embodiment by the first data are carried out sampling, each compression compression algorithm first sampled data in multiple compression algorithm and the data decompression after the first sampled data is compressed to determine the first efficacy parameter, complete each compression algorithm the first data are compressed or the first sampled data are compressed after the result estimate of data decompression.
Here the first efficacy parameter said at least comprises compression ratio, compression speed, solution pressure speed etc. This is not done concrete restriction by the present embodiment, the first efficacy parameter can be set according to practical situation.
Can selection of land, when the first efficacy parameter is compression ratio, compression speed and decompression speed, this step can also realize in the following manner:
Data after utilizing each compression algorithm each first sampled data to be compressed and the first sampled data compressed decompress. Algorithm is compressed for often kind, obtain compression ratio when each first sampled data being compressed, compression speed, and decompression speed when each first sampled data is decompressed, and from, each compression ratio obtained, compression speed and decompression speed, obtaining each and compress the maximum compression ratio corresponding to algorithm, maximum compression speed and maximum decompression speed. It is defined as the maximum compression ratio of acquisition, maximum compression speed and maximum decompression speed compressing the first efficacy parameter corresponding to algorithm.
Specific implementation program is as follows:
// the first sampled data collection, N number of sample
// compression algorithm, M algorithm
The compression speed restriction of // server, S bps
The decompression speed restriction of // server, D bps
The rate of compression restriction of // server, size of data after former size of data/compression > C
The internal memory space constraint of // server, total memory space requirement < T GB
CMark=0; The compression ratio that // record i-th compression algorithm is corresponding
SMark=0; The compression speed that // record i-th compression algorithm is corresponding
DMark=0; The decompression speed that // record i-th compression algorithm is corresponding
CMax=0; The maximum compression ratio information that // record i-th compression algorithm is corresponding
SMax=0; The maximum compression speed information that // record i-th compression algorithm is corresponding
DMax=0; The maximum decompression speed information that // record i-th compression algorithm is corresponding
For (i=1; I��M; I++)
For (j=1; J��N; J++)
// with i-th compression algorithm, the first sampled data is compressed; Recording compressed than c, compression speed s, decompression speed d, the internal memory usage space m of server;
Step 201c, server determines that the best compression algorithm of the first efficacy parameter is as the first compression algorithm. In a kind of possible implementation, the best compression algorithm of the first efficacy parameter can be determined according to a certain efficacy parameter. Such as, the compression algorithm of correspondence maximum for compression ratio numerical value is defined as the best compression algorithm of the first efficacy parameter. For another example, the compression algorithm of correspondence maximum for compression speed numerical value is defined as the best compression algorithm of the first efficacy parameter.
In another kind of possible implementation, the best compression algorithm of the first efficacy parameter can be compression algorithm maximum in the expected value of multiple compression algorithm. Also it is, the expected value of the first compression algorithm is maximum in the expected value of multiple compression algorithm, and what the expected value of the first compression algorithm was obtained according to the first efficacy parameter weight calculation of the first efficacy parameter of the first compression algorithm and the first compression algorithm according to preset expected value-based algorithm by server.
Here the expected value said is used to indicate the comprehensive compression effect of often kind of compression corresponding to algorithm, calculates according to multiple first efficacy parameter, and preset expected value-based algorithm is each first efficacy parameter and the product sum of the first efficacy parameter weight. Also that is, it is desirable to value the=the first efficacy parameter 1* first efficacy parameter 1 weight+the first efficacy parameter 2* first efficacy parameter 2 weight+... + the first efficacy parameter n* first efficacy parameter n weight.
In general, often kind of first efficacy parameter and the first efficacy parameter weight is long-pending belongs to the same order of magnitude, such as all between 0��1, or such as all between 1��10 etc.
Calculate expected value corresponding to often kind of compression algorithm by aforesaid method, compression algorithm maximum for expected value is defined as the first compression algorithm.
Step 204, the 2nd data are carried out compression and obtain the 3rd data by server use the 2nd compression algorithm.
Here the 2nd data are that step 202 utilizes the first compression algorithm to the data obtained after the first data compression.
Same, in order to best compression effect can be known, when the 2nd data being compressed, server needs to obtain the 2nd best compression algorithm of compression effect from multiple compression algorithm, and the process obtaining the 2nd compression algorithm can see the step 203a shown in Fig. 2 C to step 203c.
Step 203a, server is sampled from the 2nd data and is obtained the 2nd sampled data.
The mode that the sampling of the 2nd data is not obtained the 2nd sampled data by the present embodiment specifically limits, and can determine according to concrete performance.
Such as, it is possible to every predetermined the 2nd data, the 2nd sampled data of sampling, it is also possible to multiple 2nd sampled data of continuous sampling from the 2nd data.
Step 203b, server use in multiple compression algorithm each compression compression algorithm the 2nd sampled data and to the 2nd sampled data compress after data decompression to determine the 2nd efficacy parameter. Wherein, the 2nd efficacy parameter be used to indicate each compression algorithm the 2nd sampled data is compressed or decompresses the 2nd sampled data compression after data time effect.
In general, when data after utilizing a certain compression algorithm the 2nd sampled data to be compressed or the 2nd sampled data compressed decompress, two efficacy parameter corresponding with this compression algorithm can be obtained, here the 2nd efficacy parameter said not only is determined by this compression algorithm, is also subject to the impact of multiple restraining factors. Here the restraining factors said comprise: computing power of the performance data feature of the data structure feature of the 2nd sampled data, the data statistical characteristics of the 2nd sampled data, the 2nd sampled data, the Memory Allocation of server, the internal memory capacity of server, the internal memory behaviour in service of server and server etc.
Owing to the effect that the 2nd data are compressed is affected by each compression algorithm by multiple restraining factors, it is necessary to just can obtain utilizing, by the calculating of comparatively complexity, the effect that the 2nd data are compressed by this compression algorithm. And in actual mechanical process, when data after using different servers the 2nd data to be compressed or the 2nd sampled data compressed decompress, owing to the computing power of server, internal memory capacity, internal memory behaviour in service etc. exist difference, when the 2nd identical data are carried out compression and decompression by different server by utilizing identical compression algorithm, also can there is difference in compression effect.
Therefore, the 2nd sampled data is obtained in the present embodiment by the 2nd data are carried out sampling, each compression compression algorithm the 2nd sampled data in multiple compression algorithm and the data decompression after the 2nd sampled data being compressed, to determine the 2nd efficacy parameter, complete the result estimate that each compression algorithm carries out compressing or data after the 2nd sampled data being compressed decompress for the 2nd data.
Here the 2nd efficacy parameter said at least comprises compression ratio, compression speed, solution pressure speed etc. This is not done concrete restriction by the present embodiment, the 2nd efficacy parameter can be set according to practical situation.
Can selection of land, when the 2nd efficacy parameter is compression ratio, compression speed and decompression speed, this step can also realize in the following manner:
Utilize each compression algorithm that each the 2nd sampled data is carried out compression and decompression. Algorithm is compressed for often kind, obtain compression ratio when each the 2nd sampled data being compressed, compression speed, and decompression speed when each the 2nd sampled data is decompressed, and from, each compression ratio obtained, compression speed and decompression speed, obtaining each and compress the maximum compression ratio corresponding to algorithm, maximum compression speed and maximum decompression speed. It is defined as the maximum compression ratio of acquisition, maximum compression speed and maximum decompression speed compressing the 2nd efficacy parameter corresponding to algorithm.
Step 203c, server determines that the best compression algorithm of the 2nd efficacy parameter is as the 2nd compression algorithm.
In a kind of possible implementation, the best compression algorithm of the 2nd efficacy parameter can be determined according to a certain efficacy parameter. Such as, the compression algorithm of correspondence maximum for compression ratio numerical value is defined as the best compression algorithm of the 2nd efficacy parameter. For another example, the compression algorithm of correspondence maximum for compression speed numerical value is defined as the best compression algorithm of the 2nd efficacy parameter.
In another kind of possible implementation, the best compression algorithm of the 2nd efficacy parameter can be compression algorithm maximum in the expected value of multiple compression algorithm. Also it is, the expected value of the 2nd compression algorithm is maximum in the expected value of multiple compression algorithm, and what the expected value of the 2nd compression algorithm was obtained according to the 2nd efficacy parameter weight calculation of the 2nd efficacy parameter of the 2nd compression algorithm and the 2nd compression algorithm according to preset expected value-based algorithm by server.
Here the expected value said is used to indicate the comprehensive compression effect of often kind of compression corresponding to algorithm, calculates according to multiple 2nd efficacy parameter, and preset expected value-based algorithm is the product sum of each the 2nd efficacy parameter and the 2nd efficacy parameter weight. Also that is, it is desirable to value the=the two efficacy parameter 1* the 2nd efficacy parameter 1 weight+the two efficacy parameter 2* the 2nd efficacy parameter 2 weight+... + the two efficacy parameter n* the 2nd efficacy parameter n weight.
In general, often kind of the 2nd efficacy parameter and the 2nd efficacy parameter weight is long-pending belongs to the same order of magnitude, such as all between 0��1, or such as all between 1��10 etc.
Calculate expected value corresponding to often kind of compression algorithm by aforesaid method, compression algorithm maximum for expected value is defined as the 2nd compression algorithm.
It should be noted is that, it is (English: Dictionarycoding), to grow string compression technology (English: Run-lengthEncoding that the compression algorithm said in the present invention at least comprises dictionary compression algorithm; RLE) and incremental encoding compression algorithm (English: DeltaEncoding) etc., the concrete kind of compression algorithm is not limited by the present invention.
In sum, the method of the server compresses data that the embodiment of the present invention provides, using the first compression algorithm that the first data are carried out compression by server and obtain the 2nd data, described 2nd data sampled data is carried out compression and obtains the 3rd data by server use the 2nd compression algorithm; Owing to have employed compression algorithm combination mode, storage data are repeatedly compressed, solve and when prior art adopting the compression algorithm of single type the data in internal memory compressed, compress the poor problem of effect, reach the effect improving compression effect.
In actual applications, along with the storage of data, the feature of the data stream received may change, so cause it has been determined that compression algorithm combination to, when having the data stream compression of new feature, compression effect may be poor. Therefore, after determining the first compression algorithm and the 2nd compression algorithm, obtain the compression effect of server when obtaining the 3rd data in real time; When compressing effect and do not meet re-set target, redefined the first compression algorithm and the 2nd compression algorithm by the mode in Fig. 2 B and Fig. 2 C.
Here the compression effect said comprises central processing unit (English: CentralProcessingUnit, the CPU) occupation rate of server, efficacy parameter, internal memory usage space etc. Re-set target is developer's setting, whether is applicable to currently store compression and the decompression of data for detecting current compression technical combinations algorithm. Due to current compression technical combinations algorithm be according to before storage data formulate, when compressing effect and not meeting re-set target, show that current storage data there occurs change relative to storage data before in data structure, data statistical characteristics and performance data feature etc., it is necessary to redefine the first compression algorithm and the 2nd compression algorithm.
Please refer to Fig. 3, it illustrates the block diagram of the device of the server compresses data that one embodiment of the invention provides. The device of these server compresses data can realize becoming the whole or a part of of server by software, hardware or both combinations. The device of these server compresses data can comprise: the first determining unit 310, the 2nd determining unit 320 and compressed element 330.
First determining unit 310, for realizing the function of at least one step in above-mentioned steps 201a to 201c.
2nd determining unit 320, for realizing the function of at least one step in above-mentioned steps 203a to 203c.
Compressed element 330, for realizing the function of at least one step in above-mentioned steps 202 and 204.
Correlative detail can in conjunction with reference to aforesaid method embodiment.
Above-mentioned embodiment of the present invention sequence number, just to describing, does not represent the quality of embodiment.
One of ordinary skill in the art will appreciate that all or part of step realizing above-described embodiment can be completed by hardware, can also be completed by the hardware that program carrys out instruction relevant, described program can be stored in a kind of computer-readable recording medium, the above-mentioned storage media mentioned can be read-only storage, disk or CD etc.
Claims (10)
1. the method for server compresses data, it is characterised in that, described method comprises:
First data are carried out compression and obtain the 2nd data by described server use first compression algorithm;
Described 2nd data are carried out compression and obtain the 3rd data by described server use the 2nd compression algorithm.
2. method according to claim 1, it is characterised in that, described method, also comprises:
Described server is sampled from described first data and is obtained the first sampled data;
Described server use in multiple compression algorithm each compression compression algorithm described in the first sampled data and to described first sampled data compress after data decompression to determine the first efficacy parameter; Described first efficacy parameter be used to indicate each compression algorithm described first sampled data is compressed or decompress described first sampled data compress after data time effect;
Described server determines that the best compression algorithm of the first efficacy parameter is as described first compression algorithm.
3. method according to claim 1, it is characterised in that, described method, also comprises:
Described server is sampled from described 2nd data and is obtained the 2nd sampled data;
Described server use in multiple compression algorithm each compression compression algorithm described in the 2nd sampled data and to described 2nd sampled data compress after data decompression to determine the 2nd efficacy parameter; Described 2nd efficacy parameter be used to indicate each compression algorithm described 2nd sampled data is compressed or decompress described 2nd sampled data compress after data time effect;
Described server determines that compression algorithm corresponding to the 2nd best efficacy parameter is as described 2nd compression algorithm.
4. method according to claim 2, it is characterised in that, the expected value of described first compression algorithm is maximum in the expected value of described multiple compression algorithm; Wherein, the expected value of described first compression algorithm is obtained according to the first efficacy parameter weight calculation of the first efficacy parameter of described first compression algorithm and described first compression algorithm according to preset expected value-based algorithm by described server.
5. method according to claim 3, it is characterised in that, the expected value of described 2nd compression algorithm is maximum in the expected value of described multiple compression algorithm; Wherein, the expected value of described 2nd compression algorithm is obtained according to the 2nd efficacy parameter weight calculation of the 2nd efficacy parameter of described first compression algorithm and described 2nd compression algorithm according to preset expected value-based algorithm by described server.
6. the device of server compresses data, it is characterised in that, described device comprises:
First data are carried out compression for using the first compression algorithm and obtain the 2nd data by compressed element;
Described compressed element, also obtains the 3rd data for using the 2nd compression algorithm that described 2nd data are carried out compression.
7. device according to claim 6, it is characterised in that, described device, also comprises:
First determining unit, for the first sampled data sampled and obtain from described first data;
Described first determining unit, for use in multiple compression algorithm each compression compression algorithm described in the first sampled data and to described first sampled data compress after data decompression to determine the first efficacy parameter; Described first efficacy parameter be used to indicate each compression algorithm described first sampled data is compressed or decompress described first sampled data compress after data time effect;
Described first determining unit, for determining that the best compression algorithm of the first efficacy parameter is as described first compression algorithm.
8. device according to claim 6, it is characterised in that, described device, also comprises:
Described 2nd determining unit, for the 2nd sampled data sampled and obtain from described 2nd data;
Described 2nd determining unit, also for use in multiple compression algorithm each compression compression algorithm described in the 2nd sampled data and to described 2nd sampled data compress after data decompression to determine the 2nd efficacy parameter; Described 2nd efficacy parameter be used to indicate each compression algorithm described 2nd sampled data is compressed or decompress described 2nd sampled data compress after data time effect;
Described 2nd determining unit, also for determining that compression algorithm corresponding to the 2nd best efficacy parameter is as described 2nd compression algorithm.
9. device according to claim 7, it is characterised in that, the expected value of described first compression algorithm is maximum in the expected value of described multiple compression algorithm; Wherein, the expected value of described first compression algorithm is obtained according to the first efficacy parameter weight calculation of the first efficacy parameter of described first compression algorithm and described first compression algorithm according to preset expected value-based algorithm by described server.
10. device according to claim 8, it is characterised in that, the expected value of described 2nd compression algorithm is maximum in the expected value of described multiple compression algorithm; Wherein, the expected value of described 2nd compression algorithm is obtained according to the 2nd efficacy parameter weight calculation of the 2nd efficacy parameter of described first compression algorithm and described 2nd compression algorithm according to preset expected value-based algorithm by described server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510999354.6A CN105630999A (en) | 2015-12-28 | 2015-12-28 | Data compressing method and device of server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510999354.6A CN105630999A (en) | 2015-12-28 | 2015-12-28 | Data compressing method and device of server |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105630999A true CN105630999A (en) | 2016-06-01 |
Family
ID=56045932
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510999354.6A Pending CN105630999A (en) | 2015-12-28 | 2015-12-28 | Data compressing method and device of server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105630999A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106407285A (en) * | 2016-08-26 | 2017-02-15 | 西安空间无线电技术研究所 | RLE and LZW-based optimized bit file compression and decompression method |
| CN107783990A (en) * | 2016-08-26 | 2018-03-09 | 华为技术有限公司 | A kind of data compression method and terminal |
| CN110489123A (en) * | 2018-05-15 | 2019-11-22 | 腾讯科技（深圳）有限公司 | A kind of preprocess method of compiling, compilation device and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102004935A (en) * | 2010-11-08 | 2011-04-06 | 佟野 | LDPC (Low Density Parity Code)-based method for encoding and decoding two dimensional bar codes |
| CN104268034A (en) * | 2014-10-09 | 2015-01-07 | 中国人民解放军国防科学技术大学 | Data backup method and device and data recovery method and device |
| CN104348490A (en) * | 2014-11-14 | 2015-02-11 | 北京东方国信科技股份有限公司 | Combined data compression algorithm based on effect optimization |
| CN104951473A (en) * | 2014-03-31 | 2015-09-30 | 中国移动通信集团宁夏有限公司 | Method and device for compressing data |
-
2015
- 2015-12-28 CN CN201510999354.6A patent/CN105630999A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102004935A (en) * | 2010-11-08 | 2011-04-06 | 佟野 | LDPC (Low Density Parity Code)-based method for encoding and decoding two dimensional bar codes |
| CN104951473A (en) * | 2014-03-31 | 2015-09-30 | 中国移动通信集团宁夏有限公司 | Method and device for compressing data |
| CN104268034A (en) * | 2014-10-09 | 2015-01-07 | 中国人民解放军国防科学技术大学 | Data backup method and device and data recovery method and device |
| CN104348490A (en) * | 2014-11-14 | 2015-02-11 | 北京东方国信科技股份有限公司 | Combined data compression algorithm based on effect optimization |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106407285A (en) * | 2016-08-26 | 2017-02-15 | 西安空间无线电技术研究所 | RLE and LZW-based optimized bit file compression and decompression method |
| CN107783990A (en) * | 2016-08-26 | 2018-03-09 | 华为技术有限公司 | A kind of data compression method and terminal |
| CN106407285B (en) * | 2016-08-26 | 2019-11-29 | 西安空间无线电技术研究所 | A kind of optimization bit file compression & decompression method based on RLE and LZW |
| CN107783990B (en) * | 2016-08-26 | 2021-11-19 | 华为技术有限公司 | Data compression method and terminal |
| CN110489123A (en) * | 2018-05-15 | 2019-11-22 | 腾讯科技（深圳）有限公司 | A kind of preprocess method of compiling, compilation device and storage medium |
| CN110489123B (en) * | 2018-05-15 | 2022-04-05 | 腾讯科技（深圳）有限公司 | Preprocessing method for compiling, compiling device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102143039B (en) | Data segmentation method and equipment for data compression | |
| CN102694554A (en) | Data compression devices, operating methods thereof, and data processing apparatuses including the same | |
| JP6344486B2 (en) | Method for compressing data by server and device | |
| CN105553937A (en) | System and method for data compression | |
| CN105868194A (en) | Methods and devices for text data compression and decompression | |
| CN107046812A (en) | A kind of data save method and device | |
| EP4350527A1 (en) | Data compression method and apparatus, and computing device and storage medium | |
| CN108965333A (en) | A kind of data compression, decompression method, system and electronic equipment | |
| CN105630999A (en) | Data compressing method and device of server | |
| CN107027326B (en) | The method and device of data backup in storage system | |
| CN105653698A (en) | Data loading method and apparatus for database table Hive Table | |
| CN106227881A (en) | A kind of information processing method and server | |
| CN109325590A (en) | For realizing the device for the neural network processor that computational accuracy can be changed | |
| CN110288666B (en) | Data compression method and device | |
| CN109088636A (en) | A kind of data processing method, system and electronic equipment and storage medium | |
| CN115380267A (en) | Data compression method and device, data compression equipment and readable storage medium | |
| EP3963853B1 (en) | Optimizing storage and retrieval of compressed data | |
| CN107783990B (en) | Data compression method and terminal | |
| CN111417920A (en) | Data processing method and device | |
| CN107436848B (en) | Method and device for realizing conversion between user data and compressed data | |
| CN109617708A (en) | A kind of compression method burying a log, equipment and system | |
| CN109254928A (en) | A kind of method of log processing, system, equipment and computer readable storage medium | |
| CN104618715A (en) | Method and device for obtaining minimal rate-distortion cost | |
| CN112072783B (en) | Method and device for transmitting second-level load data between end-side equipment and edge-side equipment | |
| KR102425039B1 (en) | Apparatus and method for compressing data in distributed deep-learning environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160601 |
|
| RJ01 | Rejection of invention patent application after publication |