CN105718478A - Data storage method and device - Google Patents
Data storage method and device Download PDFInfo
- Publication number
- CN105718478A CN105718478A CN201410729329.1A CN201410729329A CN105718478A CN 105718478 A CN105718478 A CN 105718478A CN 201410729329 A CN201410729329 A CN 201410729329A CN 105718478 A CN105718478 A CN 105718478A
- Authority
- CN
- China
- Prior art keywords
- data
- compression
- stored data
- stored
- type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 68
- 238000013500 data storage Methods 0.000 title claims abstract description 40
- 230000006835 compression Effects 0.000 claims abstract description 352
- 238000007906 compression Methods 0.000 claims abstract description 351
- 238000013507 mapping Methods 0.000 claims description 90
- 238000013144 data compression Methods 0.000 claims description 83
- 238000012795 verification Methods 0.000 claims description 16
- 239000002699 waste material Substances 0.000 abstract description 10
- 238000010586 diagram Methods 0.000 description 12
- 238000012545 processing Methods 0.000 description 8
- 230000000694 effects Effects 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000013499 data model Methods 0.000 description 1
Landscapes
- Compression, Expansion, Code Conversion, And Decoders (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data storage method and device, and belongs to the technical field of communication. The method comprises that a storage node of a Cassandra system receives pre-stored data and identifies the data types of the pre-stored data; and the storage node of the Cassandra system judges that the pre-stored data is compressed and then is stored according to the data types of the pre-stored data. The device comprises a first receiving module, an identification module and a first storage module. According to the method and the device provided by the invention, the problems of too low compression rate and even serious compression-expansion resulting from storage of some data types is avoided; the problem that a great deal of CPU consumption is generated but a great deal of storage space is not saved is avoided; and the waste of the resources of the CPU and the storage space are avoided.
Description
Technical Field
The present application relates to the field of communications technologies, and in particular, to a method and an apparatus for storing data.
Background
Cassandra is a mixed type non-relational database, is based on Amazon (Amazon) proprietary fully distributed Dynamo, combines a GoogleBigTable column-based (ColumnAily) data model, and adopts a decentralized storage architecture. Cassandra is a distributed storage system of a ring structure, which relies on the realization of DHT (distributed hash table) technology, and has no typical central node (each node exists as both an access node and a data node). The Cassandra data storage space can be abstracted into a ring structure, and the data is dispersed on the ring storage space through the Hash. Each node in Cassandra is responsible for managing a contiguous Range (also called Range) on the circular storage space, and data falling on the Range space is stored on the node.
The existing method for storing data in Cassandra is to first call gzip (compression program) compression algorithm to compress pre-stored data when the data is sent to a data node for storage, and then store the compressed data in a disk file.
In the existing data storage method, compression is performed on each pre-stored data, and the compression ratio is large when the data type is a text file type, but is small when the data type is a binary file type. When a large amount of binary file type data needs to be stored, the compression rate is extremely low, even severe compression expansion is generated, so that a large amount of CPU overhead is generated, a large amount of storage space is not saved, and resources of a CPU and the storage space are wasted.
Disclosure of Invention
The technical problem to be solved by the present application is that, in the prior art, a large amount of CPU overhead is generated, but a large amount of storage space is not saved, and resources of the CPU and the storage space are wasted. In order to solve the technical problem, the invention provides a data storage method and a data storage device, which can avoid extremely low compression rate and even serious compression expansion when some data types are stored, avoid the generation of a large amount of CPU (central processing unit) overhead, save a large amount of storage space and avoid the waste of resources of the CPU and the storage space.
In order to solve the above problem, the present application discloses a method of data storage, the method comprising:
a storage node of a Cassandra system receives pre-stored data and identifies the data type of the pre-stored data;
and the storage node of the Cassandra system judges that the pre-stored data is compressed and stored according to the data type of the pre-stored data.
In order to solve the above problem, the present application also discloses a data storage method, including:
a storage node of a Cassandra system receives pre-stored data and acquires storage mode information carried in the pre-stored data;
and the storage node of the Cassandra system selects a compression algorithm according to the storage mode information, and compresses and stores the pre-stored data according to the selected compression algorithm.
In order to solve the above problem, the present application discloses an apparatus for data storage, the apparatus comprising:
the first receiving module is used for receiving pre-stored data;
the identification module is used for identifying the data type of the pre-stored data;
and the first storage module is used for judging that the pre-stored data is compressed and stored according to the data type of the pre-stored data.
In order to solve the above problem, the present application also discloses a data storage apparatus, including:
the second receiving module is used for receiving pre-stored data;
the acquisition module is used for acquiring the storage mode information carried in the pre-stored data;
the verification module is used for selecting a compression algorithm according to the storage mode information;
and the third storage module is used for compressing and storing the pre-stored data according to the selected compression algorithm.
Compared with the prior art, the application can obtain the following technical effects:
the method can avoid extremely low compression rate and even serious compression expansion when some data types are stored, avoid the generation of a large amount of CPU overhead, but not save a large amount of storage space, and avoid the waste of resources of the CPU and the storage space. For example, when the data type is a text file type, the data type has a larger compression ratio, pre-stored data is compressed, and the compressed data is stored in a disk file, so that a large amount of storage space can be saved. When the data type is a binary file type, the compression ratio is very small, the pre-stored data is not compressed, and the pre-stored data is directly stored in a disk file, so that the problems of extremely low compression ratio and even serious compression expansion are avoided, a large amount of storage space is not saved while a large amount of CPU (Central processing Unit) overhead is not generated, and the waste of resources of a CPU and the storage space is avoided.
Of course, it is not necessary for any one product to achieve all of the above-described technical effects simultaneously.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
FIG. 1 is a flowchart of a method for storing data according to an embodiment of the present disclosure;
FIG. 2 is a schematic diagram of a data storage process according to an embodiment of the present application;
FIG. 3 is a flowchart of a method for storing data according to a second embodiment of the present disclosure;
FIG. 4 is a schematic process diagram of data storage according to a second embodiment of the present application;
fig. 5 is a schematic structural diagram of a first data storage device according to a third embodiment of the present application;
fig. 6 is a schematic structural diagram of a second data storage device according to a third embodiment of the present application;
FIG. 7 is a schematic structural diagram of a third data storage device according to a third embodiment of the present application;
FIG. 8 is a schematic structural diagram of a fourth data storage device according to a third embodiment of the present application;
FIG. 9 is a schematic structural diagram of a first data storage device according to a fourth embodiment of the present disclosure;
FIG. 10 is a schematic diagram of a second data storage device according to a fourth embodiment of the present application;
FIG. 11 is a schematic structural diagram of a third data storage device according to a fourth embodiment of the present application;
FIG. 12 is a schematic structural diagram of a fourth data storage device according to a fourth embodiment of the present disclosure;
fig. 13 is a schematic structural diagram of a fifth data storage device according to a fourth embodiment of the present application;
fig. 14 is a schematic structural diagram of a sixth data storage device according to a fourth embodiment of the present application.
Detailed Description
Embodiments of the present application will be described in detail with reference to the drawings and examples, so that how to implement technical means to solve technical problems and achieve technical effects of the present application can be fully understood and implemented.
Example one
Referring to fig. 1, an embodiment of the present application provides a data storage method, where the method includes:
step S101: the storage node of the Cassandra system receives pre-stored data.
Exemplarily, after a user uses the distributed storage platform, the user needs to upload pre-stored data to a dedicated storage area of the user; specifically, for the Cassandra mode, when data is sent to a storage node of Cassandra for storage, the storage node of the Cassandra system, that is, the storage server receives the pre-stored data.
Step S102: the storage node of the Cassandra system identifies the data type of the pre-stored data.
If the pre-stored data does not carry the data type information, the encoding format and/or the encoding and decoding information of the header file of the pre-stored data need to be analyzed, and the data type of the pre-stored data is identified according to the encoding format and/or the encoding and decoding information.
If the pre-stored data carries the data type information, the data type information carried in the pre-stored data is acquired, and the data type of the pre-stored data can be directly identified.
Step S103: and the storage node of the Cassandra system judges that the pre-stored data is compressed and then stored or directly stored according to the data type of the pre-stored data.
Specifically, the step of determining, by a storage node of the Cassandra system according to the data type of the pre-stored data, that the pre-stored data is compressed and then stored includes:
traversing the data compression mapping table by the storage node of the Cassandra system according to the data type of the pre-stored data, finding a compression algorithm corresponding to the data type of the pre-stored data, compressing the pre-stored data and storing the compressed data; the data compression mapping table stores the corresponding relationship between the pre-collected data types and the compression algorithms supported by the data types.
Specifically, the determining, by a storage node of the Cassandra system, that the pre-stored data is directly stored according to the data type of the pre-stored data includes:
traversing the data compression mapping table by the storage node of the Cassandra system according to the data type of the pre-stored data, finding out that a compression algorithm corresponding to the data type of the pre-stored data is a null value, and directly storing the pre-stored data; the data compression mapping table stores the corresponding relationship between the pre-collected data types and the compression algorithms supported by the data types.
In particular, a data compression mapping table needs to be maintained. The data compression mapping table stores the corresponding relation between the data types collected in advance and the compression algorithms supported by the data types. And judging whether the data type of the pre-stored data is compressed and stored according to a data compression mapping table.
This correspondence is set in advance. For example, the method simulates the compression of any data type by each compression algorithm in advance, and sets weights for each compression algorithm according to the compression rate and the weighting of compression overhead of each compression algorithm. For any data type, the applicable compression algorithms are necessarily many, the compression algorithms also calculate weights, and the compression algorithm with the weight larger than a threshold and the highest weight is selected from the weights, so that the corresponding relation between the data type and the compression algorithm is established.
It should be noted that, for any data type, if the weight of the supported compression algorithm is not greater than the threshold, it indicates that the compression algorithm has a low compression benefit for the data type or causes compression expansion, and therefore, for the data type, the corresponding compression algorithm is null (i.e., there is no corresponding compression algorithm). In addition, for any data type, a plurality of compression algorithms which are more than a threshold in weight and are possibly arranged in parallel with the highest weight are listed, and the plurality of compression algorithms which are consistent with the situation are listed; and then, all the corresponding compression algorithms are given according to the data type traversal time, as for which compression algorithm can be selected to prompt a user to select, the compression algorithm can be selected according to the condition of a compression engine of the system, or the compression algorithm can be selected randomly, and the application is not limited to the method.
According to the data type of the pre-stored data, the corresponding compression algorithm is found in the data compression mapping table, so that a decision is made to select a proper compression algorithm and then the compression storage is compressed (compressed write), and if the corresponding compression algorithm is not found in the data compression mapping table (namely, a null value is returned), the data is not compressed and is directly stored (unopposed write).
For example, if the data type of the pre-stored data is a text file type, finding the compression algorithm corresponding to the text file type according to the data compression mapping table at least includes one of the following: gzip Algorithm, lzma (Lempel-Ziv-Markovchain-Algorithm) Algorithm, and lzo (Lempel-Ziv) Algorithm; compressing the pre-stored data according to the compression algorithm, and storing the compressed data into a disk file; and if the data type of the pre-stored data is a binary file type, finding out that the corresponding compression algorithm is a null value according to the data compression mapping table, not compressing the pre-stored data, and directly storing the pre-stored data into the disk file.
Because the data type has a larger compression ratio when the data type is the text file type, if the data type of the pre-stored data is the text file type, the pre-stored data is determined to be compressed, and the compressed data is stored in the disk file, so that a large amount of storage space can be saved. When the data type is a binary file type, the compression ratio is very small, if the data type of the pre-stored data is the binary file type, the pre-stored data is determined not to be compressed, and the pre-stored data is directly stored in a disk file, so that the situation that the compression ratio is extremely low, even serious compression expansion is generated is avoided, a large amount of storage space is not saved while a large amount of CPU (central processing unit) overhead is not generated, and the waste of resources of a CPU and the storage space is avoided.
Specifically, referring to fig. 2, a corresponding API (application programming interface) interface may be set for the method of this embodiment, and the pre-stored data is directly sent to the API interface, so that the pre-stored data is stored according to the method of this embodiment through the API interface. The API interface may also be referred to as an adaptive storage (adaptive write) interface. The specific process is as follows: uploading pre-stored data to a system through an API (application program interface); the API interface sends the pre-stored data to a decision module, identifies the data type of the pre-stored data through the decision module, and determines to store the pre-stored data after compressing the pre-stored data according to the data type of the pre-stored data and by combining a data compression mapping table (which compression algorithm is specifically adopted for compressing is also determined according to the data compression mapping table), or determines to directly store the pre-stored data. And calling a preset compression algorithm to compress and store the pre-stored data, or directly storing the pre-stored data. When reading data, if the pre-read data is stored after being compressed, the compressed data is decompressed according to a compression algorithm used during compression, and the pre-read data is sent to an API (application program interface) through a path opposite to the path stored after being compressed; if the pre-read data is directly stored, the pre-read data is directly sent to the API interface for reading through a path opposite to that of the direct storage.
The data storage method described in this embodiment can distinguish whether the pre-stored data is worth compressing according to the data type of the pre-stored data, and can avoid that the compression rate is extremely low and even severe compression expansion is generated when some data types are stored, so that a large amount of CPU overhead is generated, a large amount of storage space is not saved, and resources of the CPU and the storage space are avoided being wasted. For example, when the data type is a text file type, the data type has a larger compression ratio, pre-stored data is compressed, and the compressed data is stored in a disk file, so that a large amount of storage space can be saved. When the data type is a binary file type, the compression ratio is very small, the pre-stored data is directly stored in a disk file, so that extremely low compression ratio and even serious compression expansion are avoided, a large amount of CPU (Central processing Unit) overhead is not generated, a large amount of storage space is not saved, and the waste of resources of a CPU and the storage space is avoided.
Example two
Referring to fig. 3, an embodiment of the present application provides a data storage method, where the method includes:
step S201: the storage node of the Cassandra system receives pre-stored data.
Exemplarily, after a user uses the distributed storage platform, the user needs to upload pre-stored data to a dedicated storage area of the user; specifically, for the Cassandra mode, when data is sent to a storage node of Cassandra for storage, the storage node of the Cassandra system, that is, the storage server receives the pre-stored data.
Step S202: and the storage node of the Cassandra system acquires the storage mode information carried in the pre-stored data.
Specifically, when submitting pre-stored data, a user provides explicit storage mode information, and carries the storage mode information in the pre-stored data.
Step S203: and selecting a compression algorithm by the storage node of the Cassandra system according to the storage mode information.
Here, the storage method corresponding to the storage method information may be directly used as the selected storage method, and the subsequent steps may be executed, where the storage method information includes: information indicating a specific compression algorithm (compressed storage), or information indicating that the compression algorithm is null (direct storage).
However, it is more preferable that there is a possibility that an error exists in the storage means information designated by the user or there is a more preferable scheme, so that the storage means information designated by the user can be verified.
The storage node of the Cassandra system identifies the data type of the pre-stored data and analyzes a compression algorithm corresponding to the data type of the pre-stored data; a storage node of the Cassandra system identifies a compression identifier in the storage mode information and inquires a compression calculation corresponding to the compression identifier; and the storage node of the Cassandra system compares the analyzed compression algorithm corresponding to the data type of the pre-stored data with the inquired compression algorithm corresponding to the compression identifier, and selects the compression algorithm according to the comparison result. If the comparison result shows that the two are consistent, which indicates that the verification result is consistent with the mode specified by the user, the compression algorithm corresponding to the storage mode information is used as the selected compression algorithm; if the comparison result shows that the two are inconsistent, the verification result is inconsistent with the mode specified by the user, there may be a better way (e.g. the user indicates one compression algorithm, but finds other compression algorithms, which should be judged to be more suitable) or a user-specified way may be wrong (e.g. the user indicates that direct storage does not require compression, i.e. the compression algorithm is null, but can compress and has a suitable compression algorithm), at which point both the verified way and the user-specified way may be prompted to the user, of course, for better selection by the user, the compression rate and/or compression overhead of the pre-stored data respectively compressed by both can also be prompted to the user for selection, and selecting one of the two as the selected compression algorithm according to the received user selected instruction.
There are two ways to identify the data type of the pre-stored data: if the pre-stored data does not carry the data type information, the encoding format and/or the encoding and decoding information of the header file of the pre-stored data needs to be analyzed, and the data type of the pre-stored data is identified according to the encoding format and/or the encoding and decoding information. If the pre-stored data carries the data type information, the data type information carried in the pre-stored data is acquired, and the data type of the pre-stored data can be directly identified.
The corresponding storage manner analyzed by the data type of the pre-stored data can be identified by the following manners:
in particular, a data compression mapping table needs to be maintained. The data compression mapping table stores the corresponding relation between the data types collected in advance and the compression algorithms supported by the data types. And identifying the data type of the pre-stored data, and analyzing a supported compression algorithm corresponding to the data type of the pre-stored data by combining the data compression mapping table. And comparing the analyzed compression algorithm corresponding to the data type of the pre-stored data with the compression algorithm corresponding to the inquired compression identifier, and selecting the compression algorithm according to the comparison result.
This correspondence is set in advance. For example, the method simulates the compression of any data type by each compression algorithm in advance, and sets weights for each compression algorithm according to the compression rate and the weighting of compression overhead of each compression algorithm. For any data type, the applicable compression algorithms are necessarily many, the compression algorithms also calculate weights, and the compression algorithm with the weight larger than a threshold and the highest weight is selected from the weights, so that the corresponding relation between the data type and the compression algorithm is established.
It should be noted that, for any data type, if the weight of the supported compression algorithm is not greater than the threshold, it indicates that the compression algorithm has a low compression benefit for the data type or causes compression expansion, and therefore, for the data type, the corresponding compression algorithm is null (i.e., there is no corresponding compression algorithm). In addition, for any data type, a plurality of compression algorithms which are more than a threshold in weight and are possibly parallel with the highest weight are listed, and the plurality of compression algorithms which are consistent with the situation are listed.
For example, if the data type of the pre-stored data is a text file type, finding the compression algorithm corresponding to the text file type according to the data compression mapping table at least includes one of the following: gzip Algorithm, lzma (Lempel-Ziv-Markovchain-Algorithm) Algorithm, and lzo (Lempel-Ziv) Algorithm; compressing the pre-stored data according to the compression algorithm, and storing the compressed data into a disk file; and if the data type of the pre-stored data is a binary file type, finding out that the corresponding compression algorithm is a null value according to the data compression mapping table, not compressing the pre-stored data, and directly storing the pre-stored data into the disk file.
Because the data type has a larger compression ratio when the data type is the text file type, if the data type of the pre-stored data is the text file type, the pre-stored data is determined to be compressed, and the compressed data is stored in the disk file, so that a large amount of storage space can be saved. When the data type is a binary file type, the compression ratio is very small, if the data type of the pre-stored data is the binary file type, the pre-stored data is determined not to be compressed, and the pre-stored data is directly stored in a disk file, so that the situation that the compression ratio is extremely low, even serious compression expansion is generated is avoided, a large amount of storage space is not saved while a large amount of CPU (central processing unit) overhead is not generated, and the waste of resources of a CPU and the storage space is avoided.
Step S204: and the storage node of the Cassandra system stores the pre-stored data according to the selected compression algorithm, wherein when the selected compression algorithm is null, the pre-stored data is directly stored.
Specifically, referring to fig. 4, a corresponding API interface may be set for the method of this embodiment, and the pre-stored data is directly sent to the API interface, so that the pre-stored data is stored according to the method of this embodiment through the API interface. The specific process is as follows: sending pre-stored data to an API (application program interface), wherein the API receives the pre-stored data, and the pre-stored data carries storage mode information; the API interface selects a compression algorithm according to the storage mode information (as described in embodiment two, the final selection here is not necessarily the compression algorithm specified by the storage mode information, and the system is selected after self-verification), compresses and stores the pre-stored data according to the selected compression algorithm (compressibility), or directly stores the pre-stored data.
When reading data, if the pre-read data is stored after being compressed, decompressing the compressed data through the selected compression algorithm, and then sending the pre-read data to the API through a path opposite to the path of the stored data after being compressed; if the pre-read data is directly stored, the pre-read data is directly sent to the API interface for reading through a path opposite to that of the directly stored data.
The data storage method described in this embodiment may set the storage mode information according to the data type of the pre-stored data, and distinguish whether the pre-stored data is worth compressing, so as to avoid that the compression rate is extremely low, even severe compression and expansion are generated when some data types are stored, avoid that a large amount of storage space is not saved while a large amount of CPU overhead is generated, and avoid wasting resources of the CPU and the storage space. For example, when the data type is a text file type, the data type has a larger compression ratio, pre-stored data is compressed, and the compressed data is stored in a disk file, so that a large amount of storage space can be saved. When the data type is a binary file type, the compression ratio is very small, the pre-stored data is directly stored in a disk file, so that extremely low compression ratio and even serious compression expansion are avoided, a large amount of CPU (Central processing Unit) overhead is not generated, a large amount of storage space is not saved, and the waste of resources of a CPU and the storage space is avoided.
EXAMPLE III
Referring to fig. 5, an embodiment of the present application provides an apparatus for data storage, where the apparatus includes:
a first receiving module 301, configured to receive pre-stored data;
an identifying module 302, configured to identify a data type of pre-stored data;
the first storage module 303 is configured to determine, according to a data type of the pre-stored data, to compress and store the pre-stored data.
Specifically, the identifying module 302 is further configured to analyze a coding format and/or coding and decoding information of a header file of the pre-stored data, and identify a data type of the pre-stored data according to the coding format and/or coding and decoding information; or, the method and the device are used for acquiring data type information carried in the pre-stored data and identifying the data type of the pre-stored data.
Referring to fig. 6, the apparatus for storing data provided in the embodiment of the present application further includes: a data compression map 304.
A data compression mapping table 304, which stores the corresponding relationship between the pre-collected data types and the compression algorithms supported by the data types;
the first storage module 303 is configured to traverse through the data compression mapping table according to the data type of the pre-stored data, find a compression algorithm corresponding to the data type of the pre-stored data, compress the pre-stored data, and store the compressed data.
It should be noted that, in the data compression mapping table 304, the compression algorithm supported by any data type is a compression algorithm that is applicable to any data type and has a weight greater than a threshold and the highest weight; and simulating in advance the compression of any data type through each compression algorithm, and setting weights for each compression algorithm according to the compression ratio and the weighting of compression overhead of each compression algorithm. When the data type of the pre-stored data is a binary file type, the compression algorithm corresponding to the binary file type in the data compression mapping table at least includes one of the following: the gzip algorithm, the lzma algorithm, and the lzo algorithm; and when the data type of the pre-stored data is a binary file type, the compression algorithm corresponding to the binary file type in the data compression mapping table is null.
Referring to fig. 7, the apparatus for storing data provided in the embodiment of the present application further includes: and a second storage module 305, configured to determine to directly store the pre-stored data according to a data type of the pre-stored data.
Further, referring to fig. 8, the method further includes: a data compression mapping table 304;
a data compression mapping table 304, which stores the corresponding relationship between the pre-collected data types and the compression algorithms supported by the data types;
the second storage module 305 is configured to traverse through the data compression mapping table according to the data type of the pre-stored data, find that the compression algorithm corresponding to the data type of the pre-stored data is a null value, and directly store the pre-stored data.
It should be noted that, in the data compression mapping table 304, the compression algorithm supported by any data type is a compression algorithm that is applicable to any data type and has a weight greater than a threshold and the highest weight; and simulating in advance the compression of any data type through each compression algorithm, and setting weights for each compression algorithm according to the compression ratio and the weighting of compression overhead of each compression algorithm. When the data type of the pre-stored data is a binary file type, the compression algorithm corresponding to the binary file type in the data compression mapping table at least includes one of the following: the gzip algorithm, the lzma algorithm, and the lzo algorithm; and when the data type of the pre-stored data is a binary file type, the compression algorithm corresponding to the binary file type in the data compression mapping table is null.
The data storage device described in this embodiment corresponds to the features of the first embodiment, and for the disadvantages, refer to the description of the first embodiment.
The data storage device described in this embodiment can distinguish whether the pre-stored data is worth compressing according to the data type of the pre-stored data, and can avoid that the compression rate is extremely low and even severe compression expansion is generated when some data types are stored, so that a large amount of CPU overhead is generated, a large amount of storage space is not saved, and resources of the CPU and the storage space are avoided being wasted. For example, when the data type is a text file type, the data type has a larger compression ratio, pre-stored data is compressed, and the compressed data is stored in a disk file, so that a large amount of storage space can be saved. When the data type is a binary file type, the compression ratio is very small, the pre-stored data is directly stored in a disk file, the extremely low compression ratio and even serious compression expansion are avoided, a large amount of CPU (Central processing Unit) overhead is not generated, a large amount of storage space is not saved, and the waste of resources of a CPU and the storage space is avoided
Example four
Referring to fig. 9, an embodiment of the present application provides an apparatus for data storage, where the apparatus includes:
a second receiving module 401, configured to receive pre-stored data;
an obtaining module 402, configured to obtain storage mode information carried in pre-stored data;
a verification module 403, configured to select a compression algorithm according to the storage mode information;
and a third storage module 404, configured to compress and store the pre-stored data according to the selected compression algorithm.
Referring to fig. 10, the apparatus for storing data provided in the embodiment of the present application further includes: an identification module 405 and a query module 406.
An identifying module 405, configured to identify a data type of pre-stored data, and analyze a compression algorithm corresponding to the data type of the pre-stored data;
the query module 406 is configured to identify a compression identifier in the storage manner information, and query a compression algorithm corresponding to the compression identifier;
the verification module 403 is configured to compare the analyzed compression algorithm corresponding to the data type of the pre-stored data with the compression algorithm corresponding to the queried compression identifier, and select the compression algorithm according to the comparison result.
Specifically, the identifying module 405 is configured to analyze an encoding format and/or codec information of a header file of the pre-stored data, and identify a data type of the pre-stored data according to the encoding format and/or codec information; or, the obtaining module 402 is configured to obtain data type information carried in pre-stored data; the identifying module 405 is configured to identify a data type of the pre-stored data according to data type information carried in the pre-stored data.
The verification module 403, when it is determined that the comparison result shows that the two are consistent, takes the compression algorithm corresponding to the storage mode information as the selected compression algorithm; and when the judgment and comparison result shows that the two are inconsistent, respectively using the compression rate and/or the compression cost of the pre-stored data compressed by the two to prompt, and selecting one of the two as the selected compression algorithm according to the received instruction.
Referring to fig. 11, the apparatus for storing data provided in the embodiment of the present application further includes: a data compression mapping table 407.
A data compression mapping table 407, which stores the correspondence between the pre-collected data types and the compression algorithms supported by the data types;
the verification module 403 is configured to traverse through the data compression mapping table according to the identified data type of the pre-stored data, and find a compression algorithm corresponding to the data type of the pre-stored data.
It should be noted that, in the data compression mapping table 407, a compression algorithm supported by any data type is a compression algorithm that is applicable to any data type and has a weight greater than a threshold and the highest weight; and simulating in advance the compression of any data type through each compression algorithm, and setting weights for each compression algorithm according to the compression ratio and the weighting of compression overhead of each compression algorithm. When the data type of the pre-stored data is a binary file type, the compression algorithm corresponding to the binary file type in the data compression mapping table at least includes one of the following: the gzip algorithm, the lzma algorithm, and the lzo algorithm; and when the data type of the pre-stored data is a binary file type, the compression algorithm corresponding to the binary file type in the data compression mapping table is null.
Referring to fig. 12, the apparatus for storing data provided in the embodiment of the present application further includes: a compression identification algorithm mapping table 408;
a compression identifier algorithm mapping table 408 storing a preset correspondence between compression identifiers and compression algorithms;
and the query module 406 is configured to query the compression identifier algorithm mapping table according to the identified compression identifier, so as to obtain a compression algorithm corresponding to the compression identifier.
Referring to fig. 13, the apparatus for storing data provided in the embodiment of the present application further includes: :
a fourth storage module 409, configured to directly store the pre-stored data when it is identified that the storage manner information includes information indicating that the compression algorithm is a null value.
Further, referring to fig. 14, the method further includes: a data compression mapping table 407;
a data compression mapping table 407, which stores the correspondence between the pre-collected data types and the compression algorithms supported by the data types;
the verification module 403 is configured to traverse through the data compression mapping table according to the identified data type of the pre-stored data, and find that a compression algorithm corresponding to the data type of the pre-stored data is a null value, where a correspondence relationship between a pre-collected data type and a compression algorithm supported by the pre-collected data type is stored in the data compression mapping table.
It should be noted that, in the data compression mapping table 407, a compression algorithm supported by any data type is a compression algorithm that is applicable to any data type and has a weight greater than a threshold and the highest weight; and simulating in advance the compression of any data type through each compression algorithm, and setting weights for each compression algorithm according to the compression ratio and the weighting of compression overhead of each compression algorithm. When the data type of the pre-stored data is a binary file type, the compression algorithm corresponding to the binary file type in the data compression mapping table at least includes one of the following: the gzip algorithm, the lzma algorithm, and the lzo algorithm; and when the data type of the pre-stored data is a binary file type, the compression algorithm corresponding to the binary file type in the data compression mapping table is null.
The data storage device described in this embodiment corresponds to the features of the second embodiment, and for the disadvantages, refer to the description of the second embodiment.
The data storage device in this embodiment may set the storage mode information according to the data type of the pre-stored data, and distinguish whether the pre-stored data is worth compressing, so as to avoid that a compression rate is extremely low, even severe compression and expansion are generated when some data types are stored, avoid that a large amount of storage space is not saved while a large amount of CPU overhead is generated, and avoid wasting resources of the CPU and the storage space. For example, when the data type is a text file type, the data type has a larger compression ratio, pre-stored data is compressed, and the compressed data is stored in a disk file, so that a large amount of storage space can be saved. When the data type is a binary file type, the compression ratio is very small, the pre-stored data is directly stored in a disk file, so that extremely low compression ratio and even serious compression expansion are avoided, a large amount of CPU (Central processing Unit) overhead is not generated, a large amount of storage space is not saved, and the waste of resources of a CPU and the storage space is avoided.
a1, the method is characterized in that the method comprises:
a storage node of a Cassandra system receives pre-stored data and identifies the data type of the pre-stored data;
and the storage node of the Cassandra system judges that the pre-stored data is compressed and stored according to the data type of the pre-stored data.
a2 the method of claim a1, wherein the determining, by the storage node of the Cassandra system according to the data type of the pre-stored data, that the pre-stored data is compressed and stored includes:
traversing the data compression mapping table by the storage node of the Cassandra system according to the data type of the pre-stored data, finding a compression algorithm corresponding to the data type of the pre-stored data, and compressing and storing the pre-stored data;
the data compression mapping table stores the corresponding relationship between the pre-collected data types and the compression algorithms supported by the data types.
a3 the method as claimed in claim a2, wherein when the data type of the pre-stored data is a text file type, the compression algorithm corresponding to the text file type in the data compression mapping table includes at least one of the following: the gzip algorithm, the lzma algorithm, and the lzo algorithm.
a4, the method of claim a1, further comprising: and the storage node of the Cassandra system judges that the pre-stored data is directly stored according to the data type of the pre-stored data.
a5, the method of claim a4, wherein the determining, by the storage node of the Cassandra system, that the pre-stored data is directly stored according to the data type of the pre-stored data includes:
traversing the data compression mapping table by the storage node of the Cassandra system according to the data type of the pre-stored data, finding out that a compression algorithm corresponding to the data type of the pre-stored data is a null value, and directly storing the pre-stored data;
the data compression mapping table stores the corresponding relationship between the pre-collected data types and the compression algorithms supported by the data types.
a6, the method according to claim a5, wherein when the data type of the pre-stored data is a binary file type, the compression algorithm corresponding to the binary file type in the data compression mapping table is null.
a7, the method of any one of claims a2-a3 and a5-a6,
in the data compression mapping table, a compression algorithm supported by any data type is a compression algorithm which is applicable to any data type and has a weight greater than a threshold and the highest weight;
and simulating in advance the compression of any data type through each compression algorithm, and setting weights for each compression algorithm according to the compression ratio and the weighting of compression overhead of each compression algorithm.
a8, the method of any one of claims a1-a6, wherein identifying the data type of the pre-stored data further comprises:
analyzing the coding format and/or coding and decoding information of the header file of the pre-stored data, and identifying the data type of the pre-stored data according to the coding format and/or coding and decoding information; or,
and acquiring data type information carried in the pre-stored data, and identifying the data type of the pre-stored data.
b9, a method for data storage, the method comprising:
a storage node of a Cassandra system receives pre-stored data and acquires storage mode information carried in the pre-stored data;
and the storage node of the Cassandra system selects a compression algorithm according to the storage mode information, and compresses and stores the pre-stored data according to the selected compression algorithm.
b10, the method of claim b9, wherein the storage node of the Cassandra system selects a compression algorithm according to the storage mode information, further comprising:
the storage node of the Cassandra system identifies the data type of the pre-stored data and analyzes a compression algorithm corresponding to the data type of the pre-stored data;
the storage node of the Cassandra system identifies a compression identifier in the storage mode information and inquires a compression algorithm corresponding to the compression identifier;
and the storage node of the Cassandra system compares the analyzed compression algorithm corresponding to the data type of the pre-stored data with the inquired compression algorithm corresponding to the compression identifier, and selects the compression algorithm according to the comparison result.
b11, the method of claim b10, wherein selecting a compression algorithm based on the comparison further comprises:
if the comparison result shows that the two are consistent, the compression algorithm corresponding to the storage mode information is used as the selected compression algorithm;
and if the comparison result shows that the two are inconsistent, prompting the compression rate and/or the compression cost of the pre-stored data by using the two respectively, and selecting one of the two as the selected compression algorithm according to the received selected instruction.
b12, the method of claim b10, wherein analyzing a compression algorithm corresponding to a data type of the pre-stored data, further comprises:
traversing a data compression mapping table according to the identified data type of the pre-stored data, and finding a compression algorithm corresponding to the data type of the pre-stored data, wherein the data compression mapping table stores a correspondence between a pre-collected data type and a compression algorithm supported by the pre-collected data type.
b13 the method of claim b12,
when the data type of the pre-stored data is a text file type, the compression algorithm corresponding to the text file type in the data compression mapping table at least includes one of the following: the gzip algorithm, the lzma algorithm, and the lzo algorithm.
b14, the method of claim b10, wherein querying the compression algorithm corresponding to the compression identification, further comprises:
inquiring a preset compression identification algorithm mapping table according to the identified compression identification to obtain a compression algorithm corresponding to the compression identification; the preset compression identifier algorithm mapping table stores the corresponding relation between the preset compression identifier and the compression algorithm.
b15, the method of any one of claims b10-b11, wherein the compressing the pre-stored data according to the selected compression algorithm comprises:
and when the storage mode information is identified to comprise information indicating that the compression algorithm is null, directly storing the pre-stored data.
b16, the method of claim b15, wherein analyzing a compression algorithm corresponding to a data type of the pre-stored data, further comprises:
traversing a data compression mapping table according to the identified data type of the pre-stored data, and finding that a compression algorithm corresponding to the data type of the pre-stored data is a null value, wherein the data compression mapping table stores a correspondence between a pre-collected data type and a compression algorithm supported by the pre-collected data type.
b17 the method of claim b16,
and when the data type of the pre-stored data is a binary file type, the compression algorithm corresponding to the binary file type in the data compression mapping table is null.
b18 the method of claim b12 or b16,
in the data compression mapping table, a compression algorithm supported by any data type is a compression algorithm which is applicable to any data type and has a weight greater than a threshold and the highest weight;
and simulating in advance that any data type is compressed through each compression algorithm, and setting weights for each compression algorithm according to the compression rate and the weighting of compression overhead of each compression algorithm.
b19, the method of claim b10, wherein identifying the data type of the pre-stored data further comprises:
analyzing the coding format and/or coding and decoding information of the header file of the pre-stored data, and identifying the data type of the pre-stored data according to the coding format and/or coding and decoding information; or,
and acquiring data type information carried in the pre-stored data, and identifying the data type of the pre-stored data.
c20, an apparatus for data storage, comprising:
the first receiving module is used for receiving pre-stored data;
the identification module is used for identifying the data type of the pre-stored data;
and the first storage module is used for judging that the pre-stored data is compressed and stored according to the data type of the pre-stored data.
c21, the apparatus of claim c20, further comprising: a data compression mapping table;
the data compression mapping table stores the corresponding relation of the pre-collected data types and the compression algorithms supported by the data types;
the first storage module is configured to traverse through a data compression mapping table according to the data type of the pre-stored data, find a compression algorithm corresponding to the data type of the pre-stored data, and store the pre-stored data after compressing the pre-stored data.
c22, the apparatus of claim c21, wherein when the data type of the pre-stored data is a text file type, the compression algorithm corresponding to the text file type in the data compression mapping table includes at least one of the following: the gzip algorithm, the lzma algorithm, and the lzo algorithm.
c23, the apparatus of claim c20, further comprising: and the second storage module is used for judging that the pre-stored data is directly stored according to the data type of the pre-stored data.
c24, the apparatus of claim c23, further comprising: a data compression mapping table;
the data compression mapping table stores the corresponding relation of the pre-collected data types and the compression algorithms supported by the data types;
the second storage module is configured to traverse through a data compression mapping table according to the data type of the pre-stored data, find that a compression algorithm corresponding to the data type of the pre-stored data is a null value, and directly store the pre-stored data.
c25, the apparatus of claim c24, wherein when the data type of the pre-stored data is a binary file type, the compression algorithm corresponding to the binary file type in the data compression mapping table is null.
c26, the device of any one of claims c21-c22, c24-c25,
in the data compression mapping table, a compression algorithm supported by any data type is a compression algorithm which is applicable to any data type and has a weight greater than a threshold and the highest weight; and simulating in advance the compression of any data type through each compression algorithm, and setting weights for each compression algorithm according to the compression ratio and the weighting of compression overhead of each compression algorithm.
c27, the device of any one of claims c20-c25,
the identification module is used for analyzing the coding format and/or coding and decoding information of the header file of the pre-stored data and identifying the data type of the pre-stored data according to the coding format and/or coding and decoding information; or, the method is used to acquire data type information carried in the pre-stored data and identify the data type of the pre-stored data.
d28, an apparatus for data storage, the apparatus comprising:
the second receiving module is used for receiving pre-stored data;
the acquisition module is used for acquiring the storage mode information carried in the pre-stored data;
the verification module is used for selecting a compression algorithm according to the storage mode information;
and the third storage module is used for compressing and storing the pre-stored data according to the selected compression algorithm.
The apparatus of claim d28, further comprising: an identification module and a query module;
the identification module is used for identifying the data type of the pre-stored data and analyzing a compression algorithm corresponding to the data type of the pre-stored data;
the query module is used for identifying the compression identifier in the storage mode information and querying the compression algorithm corresponding to the compression identifier;
and the verification module is used for comparing the analyzed compression algorithm corresponding to the data type of the pre-stored data with the compression algorithm corresponding to the inquired compression identifier, and selecting the compression algorithm according to the comparison result.
d30, the device of claim d29,
the verification module is used for taking the compression algorithm corresponding to the storage mode information as the selected compression algorithm when the comparison result is judged to show that the two are consistent; and when the judgment and comparison result shows that the two are inconsistent, respectively using the two to compress the compression rate and/or the compression cost of the pre-stored data to prompt, and selecting one of the two as the selected compression algorithm according to the received selected instruction.
The apparatus of claim d29, further comprising: a data compression mapping table;
the data compression mapping table stores the corresponding relation of the pre-collected data types and the compression algorithms supported by the data types;
and the verification module is used for traversing the data compression mapping table according to the identified data type of the pre-stored data to find out the compression algorithm corresponding to the data type of the pre-stored data.
d32, the device of claim d31,
when the data type of the pre-stored data is a text file type, the compression algorithm corresponding to the text file type in the data compression mapping table at least includes one of the following: the gzip algorithm, the lzma algorithm, and the lzo algorithm.
d33, the apparatus of claim d29, further comprising: compressing the identification algorithm mapping table;
the compression identification algorithm mapping table stores the corresponding relation between the preset compression identification and the compression algorithm;
and the query module is used for querying the compression identifier algorithm mapping table according to the identified compression identifier to obtain the compression algorithm corresponding to the compression identifier.
d34 the apparatus of any of claims d28-d29, further comprising:
and the fourth storage module is used for directly storing the pre-stored data when the storage mode information is identified to comprise information indicating that the compression algorithm is null.
The apparatus of claim d34, further comprising: a data compression mapping table;
the data compression mapping table stores the corresponding relation of the pre-collected data types and the compression algorithms supported by the data types;
the verification module is configured to traverse through the data compression mapping table according to the identified data type of the pre-stored data, and find that a compression algorithm corresponding to the data type of the pre-stored data is a null value, where a correspondence between a pre-collected data type and a compression algorithm supported by the pre-collected data type is stored in the data compression mapping table.
d36, the device of claim d35,
and when the data type of the pre-stored data is a binary file type, the compression algorithm corresponding to the binary file type in the data compression mapping table is null.
d37, the device of claim d31 or d35,
in the data compression mapping table, a compression algorithm supported by any data type is a compression algorithm which is applicable to any data type and has a weight greater than a threshold and the highest weight;
and simulating in advance that any data type is compressed through each compression algorithm, and setting weights for each compression algorithm according to the compression rate and the weighting of compression overhead of each compression algorithm.
d38, the device of claim d28,
the identification module is further used for analyzing the coding format and/or the coding and decoding information of the header file of the pre-stored data and identifying the data type of the pre-stored data according to the coding format and/or the coding and decoding information; or,
the acquisition module is further configured to acquire data type information carried in the pre-stored data;
the identification module is further configured to identify a data type of the pre-stored data according to data type information carried in the pre-stored data.
The foregoing description shows and describes several preferred embodiments of the present application, but as aforementioned, it is to be understood that the application is not limited to the forms disclosed herein, but is not to be construed as excluding other embodiments and is capable of use in various other combinations, modifications, and environments and is capable of changes within the scope of the inventive concept as expressed herein, commensurate with the above teachings, or the skill or knowledge of the relevant art. And that modifications and variations may be effected by those skilled in the art without departing from the spirit and scope of the application, which is to be protected by the claims appended hereto.
Claims (10)
1. A method of data storage, the method comprising:
a storage node of a Cassandra system receives pre-stored data and identifies the data type of the pre-stored data;
and the storage node of the Cassandra system judges that the pre-stored data is compressed and stored according to the data type of the pre-stored data.
2. The method of claim 1, wherein the determining, by the storage node of the Cassandra system, that the pre-stored data is compressed and stored according to the data type of the pre-stored data comprises:
traversing the data compression mapping table by the storage node of the Cassandra system according to the data type of the pre-stored data, finding a compression algorithm corresponding to the data type of the pre-stored data, and compressing and storing the pre-stored data;
the data compression mapping table stores the corresponding relationship between the pre-collected data types and the compression algorithms supported by the data types.
3. The method as claimed in claim 2, wherein when the data type of the pre-stored data is a text file type, the compression algorithm corresponding to the text file type in the data compression mapping table includes at least one of the following: the gzip algorithm, the lzma algorithm, and the lzo algorithm.
4. A method of data storage, the method comprising:
a storage node of a Cassandra system receives pre-stored data and acquires storage mode information carried in the pre-stored data;
and the storage node of the Cassandra system selects a compression algorithm according to the storage mode information, and compresses and stores the pre-stored data according to the selected compression algorithm.
5. The method of claim 4, wherein the storage node of the Cassandra system selects a compression algorithm based on the storage mode information, further comprising:
the storage node of the Cassandra system identifies the data type of the pre-stored data and analyzes a compression algorithm corresponding to the data type of the pre-stored data;
the storage node of the Cassandra system identifies a compression identifier in the storage mode information and inquires a compression algorithm corresponding to the compression identifier;
and the storage node of the Cassandra system compares the analyzed compression algorithm corresponding to the data type of the pre-stored data with the inquired compression algorithm corresponding to the compression identifier, and selects the compression algorithm according to the comparison result.
6. An apparatus for data storage, the apparatus comprising:
the first receiving module is used for receiving pre-stored data;
the identification module is used for identifying the data type of the pre-stored data;
and the first storage module is used for judging that the pre-stored data is compressed and stored according to the data type of the pre-stored data.
7. The apparatus of claim 6, further comprising: a data compression mapping table;
the data compression mapping table stores the corresponding relation of the pre-collected data types and the compression algorithms supported by the data types;
the first storage module is configured to traverse through a data compression mapping table according to the data type of the pre-stored data, find a compression algorithm corresponding to the data type of the pre-stored data, and store the pre-stored data after compressing the pre-stored data.
8. The apparatus of claim 7, wherein when the data type of the pre-stored data is a text file type, the compression algorithm corresponding to the text file type in the data compression mapping table comprises at least one of the following: the gzip algorithm, the lzma algorithm, and the lzo algorithm.
9. An apparatus for data storage, the apparatus comprising:
the second receiving module is used for receiving pre-stored data;
the acquisition module is used for acquiring the storage mode information carried in the pre-stored data;
the verification module is used for selecting a compression algorithm according to the storage mode information;
and the third storage module is used for compressing and storing the pre-stored data according to the selected compression algorithm.
10. The apparatus of claim 9, further comprising: an identification module and a query module;
the identification module is used for identifying the data type of the pre-stored data and analyzing a compression algorithm corresponding to the data type of the pre-stored data;
the query module is used for identifying the compression identifier in the storage mode information and querying the compression algorithm corresponding to the compression identifier;
and the verification module is used for comparing the analyzed compression algorithm corresponding to the data type of the pre-stored data with the compression algorithm corresponding to the inquired compression identifier, and selecting the compression algorithm according to the comparison result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410729329.1A CN105718478A (en) | 2014-12-03 | 2014-12-03 | Data storage method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410729329.1A CN105718478A (en) | 2014-12-03 | 2014-12-03 | Data storage method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105718478A true CN105718478A (en) | 2016-06-29 |
Family
ID=56143256
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410729329.1A Pending CN105718478A (en) | 2014-12-03 | 2014-12-03 | Data storage method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105718478A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109144949A (en) * | 2017-06-27 | 2019-01-04 | 阿瓦亚公司 | System and method for reducing the memory space in liaison centre |
| CN109814809A (en) * | 2019-01-14 | 2019-05-28 | 杭州宏杉科技股份有限公司 | Data compression method and apparatus |
| WO2020259191A1 (en) * | 2019-06-28 | 2020-12-30 | 深圳前海微众银行股份有限公司 | Data centre node allocation method, apparatus, and system and computer device |
| WO2024066547A1 (en) * | 2022-09-29 | 2024-04-04 | 华为技术有限公司 | Data compression method, apparatus, computing device, and storage system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102684705A (en) * | 2012-05-30 | 2012-09-19 | 奇智软件（北京）有限公司 | Method and device for data compression |
| CN102761540A (en) * | 2012-05-30 | 2012-10-31 | 北京奇虎科技有限公司 | Data compression method, device and system and server |
| CN103631927A (en) * | 2013-12-03 | 2014-03-12 | 南京邮电大学 | Compression and storage method based on ticket data |
-
2014
- 2014-12-03 CN CN201410729329.1A patent/CN105718478A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102684705A (en) * | 2012-05-30 | 2012-09-19 | 奇智软件（北京）有限公司 | Method and device for data compression |
| CN102761540A (en) * | 2012-05-30 | 2012-10-31 | 北京奇虎科技有限公司 | Data compression method, device and system and server |
| CN103631927A (en) * | 2013-12-03 | 2014-03-12 | 南京邮电大学 | Compression and storage method based on ticket data |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109144949A (en) * | 2017-06-27 | 2019-01-04 | 阿瓦亚公司 | System and method for reducing the memory space in liaison centre |
| CN109144949B (en) * | 2017-06-27 | 2021-12-03 | 阿瓦亚公司 | System and method for reducing storage space in a contact center |
| CN109814809A (en) * | 2019-01-14 | 2019-05-28 | 杭州宏杉科技股份有限公司 | Data compression method and apparatus |
| WO2020259191A1 (en) * | 2019-06-28 | 2020-12-30 | 深圳前海微众银行股份有限公司 | Data centre node allocation method, apparatus, and system and computer device |
| WO2024066547A1 (en) * | 2022-09-29 | 2024-04-04 | 华为技术有限公司 | Data compression method, apparatus, computing device, and storage system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10089323B2 (en) | Telemetry system for a cloud synchronization system | |
| US20180196609A1 (en) | Data Deduplication Using Multi-Chunk Predictive Encoding | |
| CN110768912B (en) | API gateway current limiting method and device | |
| CN110737658A (en) | Data fragment storage method, device, terminal and readable storage medium | |
| TWI600305B (en) | Method and apparatus for compaction of data received over a network | |
| CN105718478A (en) | Data storage method and device | |
| US10476928B2 (en) | Network video playback method and apparatus | |
| US20200210411A1 (en) | Data storage in blockchain-type ledger | |
| CN104572966A (en) | Zip file unzipping method and device | |
| CN107085613B (en) | Method and device for filtering files to be put in storage | |
| CN110830551B (en) | Service request processing method, device and system | |
| CN108197168A (en) | A kind of data compression method, equipment and computer readable storage medium | |
| CN105915646A (en) | Method and device for dispersing load of servers | |
| CN109145053B (en) | Data processing method and device, client and server | |
| CN104182519A (en) | File scanning method and device | |
| CN110597461B (en) | Data storage method, device and equipment in block chain type account book | |
| CN104092754A (en) | File storage system and method | |
| US8607218B2 (en) | Updating computer files | |
| CN110333984B (en) | Interface abnormality detection method, device, server and system | |
| CN116760698A (en) | Service processing method, device, computer equipment and storage medium | |
| CN114205424B (en) | Bill file decompression method, device, computer equipment and storage medium | |
| CN107783990B (en) | Data compression method and terminal | |
| CN104765790B (en) | A kind of method and apparatus of data query | |
| CN109032804B (en) | Data processing method and device and server | |
| CN112910988A (en) | Resource acquisition method and resource scheduling device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160629 |