WO2021243531A1 - 数据压缩方法、装置、电子设备及存储介质 - Google Patents

数据压缩方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2021243531A1
WO2021243531A1 PCT/CN2020/093846 CN2020093846W WO2021243531A1 WO 2021243531 A1 WO2021243531 A1 WO 2021243531A1 CN 2020093846 W CN2020093846 W CN 2020093846W WO 2021243531 A1 WO2021243531 A1 WO 2021243531A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
compressed
compressed data
time
compression
Prior art date
Application number
PCT/CN2020/093846
Other languages
English (en)
French (fr)
Inventor
郭子亮
Original Assignee
深圳市欢太科技有限公司
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市欢太科技有限公司, Oppo广东移动通信有限公司 filed Critical 深圳市欢太科技有限公司
Priority to PCT/CN2020/093846 priority Critical patent/WO2021243531A1/zh
Priority to CN202080099532.9A priority patent/CN115398406A/zh
Publication of WO2021243531A1 publication Critical patent/WO2021243531A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation

Definitions

  • This application relates to the technical field of electronic equipment, and more specifically, to a data compression method, device, electronic equipment, and storage medium.
  • this application proposes a data compression method, device, electronic equipment, and storage medium to improve the above-mentioned problems.
  • an embodiment of the present application provides a data compression method, the method includes: acquiring data to be compressed in a memory; acquiring the length of time the data to be compressed is continuously stored in the memory as the first time ; If the first time is greater than or equal to the first time threshold, use the first compression algorithm to compress the data to be compressed to obtain first compressed data; if the first compressed data meets a preset condition, then The first compressed data is compressed again to obtain second compressed data.
  • an embodiment of the present application provides a data compression device.
  • the device includes: a data acquisition module, a time acquisition module, a first compression module, and a second compression module, wherein the data acquisition module is used to acquire Data to be compressed.
  • the time acquisition module is configured to acquire the length of time that the data to be compressed is continuously stored in the memory as the first time.
  • the first compression module is configured to compress the data to be compressed by using a first compression algorithm if the first time is greater than or equal to a first time threshold to obtain first compressed data.
  • the second compression module is configured to, if the first compressed data meets a preset condition, compress the first compressed data again to obtain second compressed data.
  • an embodiment of the present application provides an electronic device, including: one or more processors; a memory; one or more application programs, wherein the one or more application programs are stored in the memory and It is configured to be executed by the one or more processors, and the one or more programs are configured to execute the data compression method provided in the above-mentioned first aspect.
  • an embodiment of the present application provides a computer-readable storage medium.
  • the computer-readable storage medium stores program code, and the program code can be called by a processor to execute the data provided in the first aspect. Compression method.
  • the solution provided by this application implements the first compression by acquiring the data to be compressed in the memory and the length of time the data to be compressed is continuously stored in the memory, that is, when the length of time the data to be compressed is continuously stored in the memory is greater than or equal to the first At a time threshold, the first compression algorithm is used to compress the compressed data to obtain the first compressed data, and then when the first compressed data meets the preset condition, the present application can compress the first compressed data again to obtain the second compressed data. Data, which not only saves memory and disk space to a certain extent, but also improves memory and disk utilization.
  • Fig. 1 shows a flowchart of a data compression method according to an embodiment of the present application.
  • Fig. 2 shows a flowchart of a data compression method according to another embodiment of the present application.
  • Fig. 3 shows a flowchart of a data compression method according to another embodiment of the present application.
  • Fig. 4 shows a flow chart of a data compression method according to still another embodiment of the present application.
  • Fig. 5 shows a block diagram of a data compression device according to an embodiment of the present application.
  • Fig. 6 shows a block diagram of a second compression module in a data compression device according to an embodiment of the present application.
  • Fig. 7 shows another block diagram of the second compression module in the data compression device according to an embodiment of the present application.
  • Fig. 8 shows another block diagram of the second compression module in the data compression device according to an embodiment of the present application.
  • FIG. 9 is a block diagram of an electronic device for executing the data compression method according to the embodiment of the present application according to an embodiment of the present application.
  • FIG. 10 is a storage unit for storing or carrying program code for implementing the data compression method according to the embodiment of the present application according to an embodiment of the present application.
  • Time series database As a kind of time series data storage system infrastructure, has gradually been widely used in monitoring systems.
  • Time series data refers to the data sequence recorded by the same indicator in chronological order
  • the time series database is a specialized database used to store and manage time series data, that is, it is used to store the massive data of time series.
  • memory and disks are one of the important components. Memory is a bridge to communicate with the CPU. Most of the data in electronic devices are stored in disks. Therefore, the performance of memory and disks has a greater impact on electronic devices.
  • time series data Due to the massive characteristics of time series data, it is impossible to store all the data in the memory of the electronic device.
  • the data in the current specified time is usually placed in the memory as hot data to provide fast Query service.
  • the time series database compresses the data and persists it to the disk as cold data.
  • users still have a large number of query requirements for persistent data. That is, although a single compression algorithm can achieve a high compression ratio of time series data, the corresponding decompression performance cannot meet the query requirements of large-scale cold data. , And even become the performance bottleneck of electronic equipment.
  • the inventors proposed the data compression method, device, electronic device, and storage medium provided by the embodiments of the present application, which can effectively compress the data to be compressed in the memory.
  • the specific data compression method will be described in detail in the subsequent embodiments.
  • FIG. 1 shows a schematic flowchart of a data compression method provided by an embodiment of the present application.
  • the data compression method is used for the electronic device, when the data in the memory meets the preset conditions, this application can compress the data obtained by compression again, so as to not only save the memory space and improve the memory utilization rate, but also the first A compressed data is compressed again to save disk space utilization.
  • the data compression method is applied to the data compression device 500 shown in FIG. 5 and the electronic device 100 (FIG. 9) equipped with the data compression device 500.
  • FIG. 9 the electronic device 100
  • the electronic device applied in this embodiment can be a smart phone, a tablet computer, an e-book, a smart watch, etc., which is not limited here. .
  • the data compression method may include step S101 to step S104.
  • Step S101 Obtain the data to be compressed in the memory.
  • the data to be compressed may be the data written to the memory for the first time.
  • the memory also known as random access memory, RAM
  • RAM random access memory
  • the memory represents the memory used when running the program (that is, the running memory), which can only be temporarily Store data, used to exchange cache data with the processor, but the memory itself cannot be used for long-term storage of data.
  • an application When an application is running, it needs to take up memory space to store the data in the running process, and the memory space occupied by different applications in different running states is not necessarily the same.
  • the data written to the memory for the first time may be hot data in time series data
  • the time series data refers to time series data.
  • Time series data is a data sequence recorded in chronological order according to a unified index, and the time series data includes time information, data type, and data information.
  • the time information may specifically be a time stamp, but there is no fixed restriction on its specific implementation form in this application, as long as it can distinguish data representing different times.
  • the data type can be a known data type or a data type that may appear in the future.
  • the existing data type can include Integer, Long, Float, Double, String, Binary Array, Image and other types.
  • the data information can be the data itself, that is, the original data that does not include time information and data type description content.
  • hot data may be frequently accessed data, which is usually stored in a cache storage device.
  • Cold data refers to data with low access frequency and low access performance requirements, such as data that will not be accessed in a short period of time. Even data that is no longer needed in the future.
  • the electronic device may set the time interval every interval, obtain the data to be compressed in the memory, to determine the data that can be compressed subsequently from the data to be compressed, and compress the determined data to be compressed, that is, enter the step S102.
  • the electronic device can also execute the operation of obtaining the data to be compressed currently stored in the memory according to the trigger instruction input by the user, so as to determine the data that can be compressed subsequently from the data to be compressed, and to determine the data to be compressed.
  • the compressed data is compressed, that is, step S102 is entered.
  • the specific method for triggering step S101 may not be limited in the embodiment of the present application.
  • Step S102 Obtain the length of time the data to be compressed is continuously stored in the memory as the first time.
  • the electronic device after the electronic device obtains the data to be compressed in the memory, it can obtain the time that the data to be compressed is stored in the memory, that is, the length of time the data to be compressed is continuously stored in the memory.
  • the length can be the life cycle of the data to be compressed.
  • the electronic device when the data to be compressed enters the memory, the electronic device can start to perform the time recording operation, and then use the recorded time as the first time. For example, the time when data A enters the memory is 7:45 on September 10, 2019, and the current time is 16:45 on September 10, 2019. At this time, the length of time that data A is continuously stored in the memory is 7 hours .
  • Step S103 If the first time is greater than or equal to the first time threshold, use the first compression algorithm to compress the data to be compressed to obtain the first compressed data.
  • the electronic device after the electronic device obtains the length of time the data to be compressed is continuously stored in the memory for the first time, it can determine whether the first time is greater than or equal to the first time threshold, and if the first time is greater than or equal to the first time For a time threshold, the data to be compressed is compressed using the first compression algorithm to obtain the first compressed data.
  • the first time threshold may be preset or obtained based on empirical values, or may be the average value of the life cycles of all data in the memory. The specific first time threshold is not specifically limited here.
  • the first compression algorithm includes, but is not limited to, DFCM (Differential Finite Context Metho, Differential Finite Context Method) algorithm, FCM (Finite Context Method, Finite Context Method) algorithm, or LZ4 algorithm, etc.
  • LZ4 The compression algorithm is a compression algorithm that aims to be fast and low in resource overhead.
  • the electronic device may include multiple first compression algorithms, and then the multiple first compression algorithms may be used to treat them separately
  • the compressed data is compressed to obtain multiple compression results, and the multiple compression results are sorted and compared, the compression result with the smallest storage occupation is taken as the target compression result, and finally the compression algorithm corresponding to the target compression result is taken as the target compression algorithm .
  • the electronic device when the electronic device uses the first compression algorithm to compress the data to be compressed, it can first determine whether the data to be compressed is binary machine language, and if the data to be compressed is not binary machine language, the first compression algorithm is used. Compress the data to be compressed. In addition, if the data to be compressed is binary machine language, it is not compressed. Because most of the disk space occupied by the data obtained by binary machine language compression will not change, sometimes the data obtained by binary machine language compression will occupy more disk space, and the compression of binary machine language will not only increase the data compression Time, and to a certain extent, will waste the system resources of the electronic device, so when the data to be compressed is binary machine language, this application does not need to compress it.
  • the electronic device may persist the first compressed data to obtain persistent data, and the persistent data is information processing.
  • the so-called persistence is to save the first compressed data to the disk, so that the previously saved data can be accessed after the application or the machine restarts.
  • This application can use the property list plist, preference setting preference, archive and unarchive NSKeyedArchiver, database SQLite or database CoreData for data persistence.
  • the attribute list is a light-weight storage method of plain text. There are many storage formats. The most common format is xml format. The security of the attribute list is very low.
  • Preferences are mainly used to save the configuration information of the application, which can save all the data in the electronic device in the same file; archiving and unarchiving are data encryption processing, which has high security; the database SQLite is A lightweight, cross-platform small database with high portability, almost the same database statement as MySql, and the advantages of being usable without a server; the database CoreData: Putting local data in memory makes the access speed faster .
  • This application may only use any one of the above-mentioned persistence methods to persist the first compressed data, or it may combine any two or more of the above-mentioned persistence methods to persist the first compressed data. Compressed data for persistence is not clearly restricted here, and can be selected according to actual needs.
  • Step S104 If the first compressed data meets a preset condition, compress the first compressed data again to obtain second compressed data.
  • the electronic device after the electronic device persists the first compressed data to the disk, it can continuously monitor the first compressed data and determine whether the first compressed data meets a preset condition. If the compressed data meets a preset condition, the first compressed data is compressed to obtain the second compressed data; if the first compressed data does not meet the preset condition, it is not compressed, that is, the first compressed data is guaranteed No change.
  • the preset condition may be: the length of time that the first compressed data is continuously stored in the disk is greater than the second time threshold, or the number of uses of the first compressed data is greater than the preset number, or the first compressed data The use time is longer than the preset time, etc.
  • the data compression method provided in the embodiments of the present application obtains the first compressed data by obtaining the data to be compressed in the memory and the length of time the data is continuously stored in the memory, that is, when the first time is greater than or equal to the first time threshold, the first compressed data is obtained by A compression algorithm compresses the compressed data to obtain the first compressed data, and then when the first compressed data meets the preset condition, the electronic device can compress the first compressed data again to obtain the second compressed data.
  • Gradient compression can not only save memory space, but also improve the utilization of disk space.
  • this application may not compress it, which not only reduces the time waste caused by data compression to a certain extent, but also improves the utilization of electronic equipment system resources.
  • FIG. 2 shows a schematic flowchart of a data compression method provided by another embodiment of the present application. This method is applied to the above-mentioned electronic equipment. The process shown in FIG. 2 will be described in detail below. It can be seen from FIG. 2 that the data compression method may include steps S201 to S205.
  • Step S201 Obtain the data to be compressed in the memory.
  • Step S202 Obtain the length of time that the data to be compressed is continuously stored in the memory as the first time;
  • Step S203 If the first time is greater than or equal to the first time threshold, use the first compression algorithm to compress the data to be compressed to obtain the first compressed data.
  • step S201 and step S203 can refer to the content of the foregoing embodiment, and will not be repeated here.
  • Step S204 Obtain the length of time that the first compressed data is continuously stored in the disk as the second time.
  • the first compressed data can be stored in the disk of the electronic device, and when the first compressed data is stored After reaching the disk, the electronic device can record the time it has stored, and the time can be the start time. For example, if the first compressed data A is stored on the disk of the electronic device at 19:20 on September 18, 2019, the start time is 19:20 on September 18, 2019.
  • the electronic device may obtain the length of time that the first compressed data is continuously stored in the disk as the second time, that is, the second time is the time difference between the current time and the start time.
  • the start time of the first compressed data A is 19:20 on September 18, 2019, and the current time is 20:30 on September 19, 2019. At this time, it can be determined that the first compressed data is in the disk
  • the duration of continuous storage is 25 hours and 10 minutes, that is, the second time is 25 hours and 10 minutes.
  • Step S205 If the second time is greater than the second time threshold, use a second compression algorithm to compress the first compressed data again, and the second compression algorithm is different from the first algorithm.
  • the electronic device after the electronic device obtains the length of time the first compressed data is continuously stored in the disk for the second time, it can determine whether the second time is greater than the second time threshold, and if the second time is greater than the second time threshold , The second compression algorithm is used to compress the second compressed data again; if the second time is less than the second time threshold, the first compressed data is not compressed again.
  • the electronic device may use the second compression algorithm to compress the first compressed data again.
  • the second compression algorithm is different from the first compression algorithm.
  • the second compression algorithm in this application is an exclusive OR (XOR algorithm), which is based on a hierarchical XOR between data points.
  • the difference data compression coding algorithm can greatly improve the data compression rate by storing the difference of the data.
  • the electronic device can obtain the remaining storage space of its disk. If the remaining storage space of the disk is less than the storage threshold, the electronic device can The second compressed data is compressed again, and the remaining storage space of the disk is re-acquired until the remaining storage space of the disk is greater than a storage threshold. In addition, in this process, the electronic device can count the number of times that the second compressed data is compressed, that is, obtain the number of times that the second compressed data is compressed, and determine whether the number of times of compression is less than the set number of times, and if the number of times of compressed data is less than Set the number of times, then compress the second compressed data again.
  • the electronic device can record the number of compression operations, that is, the number of times the second compressed data has been compressed. When the number of times is less than the set number of times, the compression operation is continued, and when the number of compressions is greater than or equal to the set number of times, then Stop the compression operation. For example, after statistics, it is determined that the number of compression operations performed on the second compressed data B is 3 times, and the set number is also 3 times, then the compression is stopped.
  • the electronic device can obtain the compression ratio corresponding to the second compressed data, the compression ratio being the sum of the data size of the second compressed data before being compressed.
  • the ratio between the compressed data sizes that is, the data ratio before and after the data to be compressed is compressed.
  • the electronic device can determine whether the compression ratio corresponding to the second compressed data is less than the preset compression ratio. If the compression ratio corresponding to the second compressed data is less than the preset compression ratio, the second compressed data can be compressed again, and the compression ratio corresponding to the second compressed data can be obtained again, and again whether the compression ratio is less than The preset compression rate until the compression rate is less than the preset compression rate.
  • the compression rate refers to the ratio of the size of the second compressed data after compression to the size before compression.
  • the compression rate can be 0.33
  • the compression rate can be 0.44
  • the compression rate may be 0.5.
  • the above compression rate is only an example, and does not represent a limitation on the specific compression rate in the embodiment of the present application.
  • the data compression method provided in the embodiments of the present application determines whether to compress the first compressed data by obtaining the length of time that the first compressed data is continuously stored in the disk as the second time. Specifically, when the second time is greater than At the second time threshold, the present application can use the second compression algorithm to compress the first compressed data again, and the electronic device can automatically perform multi-stage compression on the compressed data, which can not only keep the persistent data at the highest compression ratio, but also solve the problem of cold.
  • the problem of data access performance to a certain extent, can break the performance bottleneck of users for large-scale queries of cold data and improve user experience.
  • FIG. 3 shows a schematic flowchart of a data compression method provided by another embodiment of the present application. This method is applied to the above-mentioned electronic equipment. The process shown in FIG. 3 will be described in detail below. From FIG. 3, it can be seen that the data compression method may include steps S301 to S305.
  • Step S301 Obtain the data to be compressed in the memory.
  • Step S302 Obtain the length of time that the to-be-compressed data is continuously stored in the memory as the first time;
  • Step S303 If the first time is greater than or equal to the first time threshold, use the first compression algorithm to compress the data to be compressed to obtain the first compressed data.
  • step S301 and step S303 can refer to the content of the foregoing embodiment, and will not be repeated here.
  • Step S304 Obtain the number of uses of the first compressed data, and determine whether the number of uses of the first compressed data is less than a preset number.
  • the electronic device uses the first compression algorithm to compress the data to be compressed to obtain the first compressed data, and then the first compressed data can be stored in the magnetic disk of the electronic device, and the electronic device can detect the first compressed data. Whether the data is used or not, if it is used, the number of times it has been used can be obtained. In the embodiment of the present application, when the electronic device obtains the use count of the first compressed data, it can determine whether the use count of the first compressed data is less than the preset count.
  • step S305 If the use count of the first compressed data is less than the preset count, use The second compression algorithm compresses the first compressed data again, that is, step S305 is entered; if the number of uses of the first compressed data is greater than the preset number of times, the second compressed data is not compressed again.
  • the preset number of times may be preset, or may be obtained by analyzing the use times of the first compressed data.
  • the number of uses may be the least used among the plurality of first compressed data, and this application does not explicitly limit the acquisition of the preset number of times.
  • Step S305 If the number of uses of the first compressed data is less than the preset number of times, the first compressed data is compressed again by using a second compression algorithm, and the second compression algorithm is different from the first algorithm.
  • step S305 can refer to the content of the foregoing embodiment, which will not be repeated here.
  • the electronic device after the electronic device compresses the first compressed data again to obtain the second compressed data, it can determine whether the disk of the electronic device includes multiple second compressed data, and if the disk includes multiple second compressed data Then, the access priority of each of the second compressed data can be obtained, and the access priority can be sorted, and then the second compressed data can be compressed again according to the order of the access priority.
  • the electronic device may obtain the priority of each second compressed data according to the access time and the access priority of the second compressed data. Specifically, first obtain the access time and access times of each of the second compressed data, and then obtain the total access time of each of the second compressed data through the access time and access times of each of the second compressed data , And determine the access priority of each of the second compressed data according to the total access time. Among them, the longer the total access time, the higher the corresponding access priority.
  • the present application can compress the second compressed data again according to the order of access priority from low to high, that is, when the electronic device determines that the disk includes multiple second compressed data, it can first perform data on the data with lower priority. compression.
  • the present application can obtain the last used time of each second compressed data in the same priority as the third time, and then set every The third times are sorted in chronological order to obtain a sorting result, and finally the second compressed data of the same priority is compressed according to the sorting result.
  • the data compression method provided by the embodiments of the present application determines whether to recompress the first compressed data by determining whether the number of uses of the first compressed data meets a preset condition, that is, to determine whether the number of uses of the first compressed data is less than the preset number If it is less than the preset number of times, the first compressed data is compressed again using the second compression algorithm.
  • This application avoids the decompression of unnecessary data by comparing the use times of the first compressed data with the preset times. While saving disk space and improving disk utilization, it also avoids the problem when the first compressed data is used again. It is difficult to decompress, which reduces the system burden of electronic equipment.
  • FIG. 4 shows a schematic flowchart of a data compression method provided by another embodiment of the present application. This method is applied to the above-mentioned electronic equipment, and the flow shown in FIG. 4 will be described in detail below. From FIG. 4, it can be seen that the data compression method may include steps S401 to S405.
  • Step S401 Obtain the data to be compressed in the memory.
  • Step S402 Obtain the length of time the data to be compressed is continuously stored in the memory as the first time;
  • step S401 and step S402 can refer to the content of the foregoing embodiment, which will not be repeated here.
  • Step S403 If the first time is greater than or equal to the first time threshold, use the first compression algorithm to compress the data to be compressed to obtain the first compressed data.
  • the electronic device when compressing the data to be compressed using the first compression algorithm, it can first determine whether the memory contains multiple data to be compressed. If the memory contains multiple data to be compressed, the electronic device can determine the multiple data to be compressed. Whether duplicate data is included in the data to be compressed, if duplicate data exists in the multiple data to be compressed, the multiple data to be compressed are deduplicated, and the data to be compressed after the deduplication operation is performed are compressed.
  • the memory includes data A to be compressed, data B to be compressed, and data C to be compressed. The data to be compressed A and data C to be compressed are determined by comparison to be duplicate data. At this time, the data to be compressed A and data C to be compressed can be executed.
  • Deduplication operation is to remove data A to be compressed or data C to be compressed, and then compress the remaining data to be compressed. If data A to be compressed is removed, data B to be compressed and data C to be compressed are compressed. For compressed data C, data A to be compressed and data B to be compressed are compressed.
  • the electronic device uses the first compression algorithm to compress the data to be compressed, and after obtaining the first compressed data, it can obtain the size of the memory occupied by the data before being compressed, and the data to be compressed The size of the memory that it occupies after being compressed. Then, the electronic device can determine whether the size of the memory occupied by the first compressed data is less than the size of the memory occupied by the data to be compressed, and if the size of the memory occupied by the first compressed data is less than the size of the memory occupied by the data to be compressed, the first compressed data Compress data for storage.
  • the electronic device can also determine whether the size of the disk occupied by the first compressed data is smaller than the size of the disk occupied by the data to be compressed, and if the size of the disk occupied by the first compressed data is less than the size of the disk occupied by the data to be compressed, the first compressed data is stored If the size of the disk occupied by the first compressed data is greater than or equal to the size of the disk occupied by the data to be compressed, the first compressed data is not stored. In this way, useless compression operations can avoid unnecessary burdens on disk space. For example, the disk space occupied by the data A to be compressed before it is compressed is 102M, and the disk space occupied by the compressed data A is 150M. Obviously, the disk space occupied by the compressed data A is not reduced, but increased At this time, there is no need to store the compressed data, that is, keep the data A to be compressed unchanged.
  • Step S404 Obtain the use duration of the first compressed data, and determine whether the use duration of the first compressed data is less than a preset duration.
  • the electronic device uses the first compression algorithm to compress the data to be compressed to obtain the first compressed data, and then the first compressed data can be stored in the magnetic disk of the electronic device, and the electronic device can detect the first compressed data. Whether the data is used, and if it is used, record the length of time it has been used.
  • the use duration of the first compressed data may be a single use duration or a total duration of multiple uses, and which is not specifically limited here can be selected according to actual conditions.
  • the electronic device can determine whether the duration of the first compressed data is less than a preset duration, and if the use duration of the first compressed data is less than the preset duration, the second compression is used
  • the algorithm compresses the first compressed data again, that is, enters step S405; if the use duration of the first compressed data is greater than the preset duration, the second compressed data is not compressed again.
  • the preset duration may be preset, or it may be obtained by analyzing the usage duration of the first compressed data.
  • the use duration may be the shortest use duration among the plurality of first compressed data, and the acquisition of the preset duration is not explicitly limited in this application.
  • Step S405 If the use duration of the first compressed data is less than the preset duration, the first compressed data is compressed again by using a second compression algorithm, and the second compression algorithm is different from the first algorithm.
  • step S405 can refer to the content of the foregoing embodiment, which will not be repeated here.
  • the data compression method provided in the embodiments of the present application determines whether to compress the first compressed data again by determining whether the use duration of the first compressed data meets a preset condition, that is, whether the use duration of the first compressed data is less than the preset duration If it is less than the preset duration, the second compression algorithm is used to compress the first compressed data again.
  • This application compares the use duration of the first compressed data with the preset duration to avoid decompression of unnecessary data, which not only saves disk space and improves disk utilization, but also reduces the burden on the electronic device system.
  • FIG. 5 shows a structural block diagram of a data compression apparatus 500 provided by an embodiment of the present application.
  • the data compression device 500 uses the aforementioned electronic equipment, and the data compression device 500 includes: a data acquisition module 501, a time acquisition module 502, a first compression module 503, and a second compression module 504.
  • the data acquisition module 501 is configured to acquire data to be compressed in the memory.
  • the time obtaining module 502 is configured to obtain the length of time that the data to be compressed is continuously stored in the memory as the first time.
  • the first compression module 503 is configured to compress the data to be compressed by using a first compression algorithm if the first time is greater than or equal to a first time threshold to obtain first compressed data.
  • the first compression module 503 is also configured to, when the memory includes multiple data to be compressed, determine whether there is duplicate data in the multiple data to be compressed, and if there is duplicate data in the multiple data to be compressed , Then perform a deduplication operation on the multiple to-be-compressed data, and compress the to-be-compressed data after the deduplication operation is performed.
  • compressing the data to be compressed by using the first compression algorithm includes: determining whether the size of the first compressed data is less than the size of the data to be compressed; The first compressed data is stored. If the size of the first compressed data is greater than or equal to the size of the data to be compressed, the first compressed data is not stored.
  • the first compression module 503 is also used to determine whether the data to be compressed is binary machine language, and if the data to be compressed is not binary machine language, the first compression algorithm is used to compress the data to be compressed. If the data to be compressed is binary machine language, the data to be compressed is not compressed.
  • the second compression module 504 is configured to, if the first compressed data meets a preset condition, compress the first compressed data again to obtain second compressed data.
  • the second compression module 504 may include a time length obtaining unit 5041 and a first compression unit 5042.
  • the time length obtaining unit 5041 is configured to obtain the length of time that the first compressed data is continuously stored in the disk as the second time.
  • the first compression unit 5042 is configured to use a second compression algorithm to compress the first compressed data again if the second time is greater than a second time threshold, and the second compression algorithm is different from the first algorithm.
  • the second compression module 504 may include a time length obtaining unit 5043 and a second compression unit 5044.
  • the use frequency obtaining unit 5043 is configured to obtain the use frequency of the first compressed data, and determine whether the use frequency of the first compressed data is less than a preset number.
  • the second compression unit 5044 is configured to use a second compression algorithm to compress the first compressed data again if the number of uses of the first compressed data is less than the preset number of times, and the second compression algorithm is the same as the first algorithm different.
  • the second compression module 504 may include a time length obtaining unit 5045 and a third compression unit 5046.
  • the use duration acquisition unit 5045 is configured to acquire the use duration of the first compressed data, and determine whether the use duration of the first compressed data is less than a preset duration.
  • the third compression unit 5046 is configured to use a second compression algorithm to compress the first compressed data again if the use duration of the first compressed data is less than the preset duration, and the second compression algorithm is the same as the first algorithm different.
  • the second compression algorithm is an exclusive-or difference compression algorithm.
  • the data compression device 500 is also used to obtain the remaining storage space of the disk. If the remaining storage space of the disk is less than the storage threshold, the second compressed data is compressed again, and the remaining storage space of the disk is obtained again. Storage space until the remaining storage space of the disk is greater than the storage threshold. Compressing the second compressed data again includes: obtaining the number of times the second compressed data has been compressed, and if the number of times of compression is less than a set number of times, compressing the second compressed data again.
  • compressing the second compressed data again includes: if the number of times of being compressed is less than the set number of times, obtaining the compression rate corresponding to the second compressed data
  • the compression ratio is a ratio of the data after compression of the second compressed data to the data before compression, and when the compression ratio corresponding to the second compressed data is less than a preset compression ratio, the second compressed data is compressed again, And re-acquire the compression ratio corresponding to the second compressed data until the compression ratio is less than the preset compression ratio.
  • the data compression device 500 is also used to determine whether the disk includes multiple second compressed data, and if the disk includes multiple second compressed data, obtain the access priority of each second compressed data And sort the access priority, and compress the second compressed data again according to the order of the access priority.
  • Obtaining the access priority of each of the second compressed data includes: obtaining the access time and the number of accesses of each of the second compressed data, and obtaining the access time and number of accesses of each of the second compressed data. According to the total access time of the second compressed data, the access priority of each second compressed data is determined according to the total access time. The longer the total access time, the higher the corresponding access priority.
  • Compressing the second compressed data again according to the order of the access priority includes: compressing the second compressed data again according to the order of the access priority from low to high.
  • compressing the second compressed data again according to the order of the access priority from low to high includes: if the same priority includes multiple second compressed data, acquiring each of the second compressed data in the same priority Second, the last used time of the compressed data is used as the third time, each of the third times is sorted in chronological order to obtain the sorting result, and the second compressed data of the same priority is compressed according to the sorting result.
  • the coupling between the modules may be electrical, mechanical or other forms of coupling.
  • each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module.
  • the above-mentioned integrated modules can be implemented in the form of hardware or software function modules.
  • the solution provided by this application obtains the historical usage data of the memory page by obtaining the currently compressed memory page in the memory, and according to the historical usage data, obtains the memory page that meets the historical usage conditions in the memory page as the target
  • the memory page compresses and stores the target memory page again, so as to achieve deep compression of the memory page, which can save memory space and improve memory utilization.
  • the electronic device 100 may be an electronic device capable of running application programs, such as a smart phone, a tablet computer, an e-book, or a smart wearable device.
  • the electronic device 100 in this application may include one or more of the following components: a processor 110, a memory 120, and one or more application programs, where one or more application programs may be stored in the memory 120 and configured to be configured by One or more processors 110 execute, and one or more programs are configured to execute the methods described in the foregoing method embodiments.
  • the processor 110 may include one or more processing cores.
  • the processor 110 uses various interfaces and lines to connect various parts of the entire electronic device 100, and executes by running or executing instructions, programs, code sets, or instruction sets stored in the memory 120, and calling data stored in the memory 120.
  • Various functions and processing data of the electronic device 100 may adopt at least one of digital signal processing (Digital Signal Processing, DSP), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and Programmable Logic Array (Programmable Logic Array, PLA).
  • DSP Digital Signal Processing
  • FPGA Field-Programmable Gate Array
  • PLA Programmable Logic Array
  • the processor 110 may be integrated with one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a modem, and the like.
  • the CPU mainly processes the operating system, user interface and application programs, etc.; the GPU is used for rendering and drawing of display content; the modem is used for processing wireless communication. It can be understood that the above-mentioned modem may not be integrated into the processor 110, but may be implemented by a communication chip alone.
  • the memory 120 may include random access memory (RAM) or read-only memory (Read-Only Memory).
  • the memory 120 may be used to store instructions, programs, codes, code sets or instruction sets.
  • the memory 120 may include a program storage area and a data storage area, where the program storage area may store instructions for implementing the operating system and instructions for implementing at least one function (such as touch function, sound playback function, image playback function, etc.) , Instructions for implementing the following various method embodiments, etc.
  • the data storage area can also store data created during use of the terminal 100 (such as phone book, audio and video data, chat record data) and the like.
  • FIG. 10 shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application.
  • the computer-readable medium 800 stores program code, and the program code can be invoked by a processor to execute the method described in the foregoing method embodiment.
  • the computer-readable storage medium 800 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM.
  • the computer-readable storage medium 800 includes a non-transitory computer-readable storage medium.
  • the computer-readable storage medium 800 has a storage space for executing the program code 810 of any method step in the above-mentioned method. These program codes can be read from or written into one or more computer program products.
  • the program code 810 may be compressed in a suitable form, for example.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种数据压缩方法、装置、电子设备及存储介质，涉及电子设备技术领域，所述方法包括：获取内存中的待压缩数据（S101）；获取所述待压缩数据在所述内存中持续存储的时间长度，作为第一时间（S102）；如果所述第一时间大于或者等于第一时间阈值，则利用第一压缩算法对所述待压缩数据进行压缩，得到第一压缩数据（S103）；如果所述第一压缩数据满足预设条件，则对所述第一压缩数据再次进行压缩得到第二压缩数据（S104）。本方法可以节省内存空间，提升内存利用率。

Description

数据压缩方法、装置、电子设备及存储介质 技术领域
本申请涉及电子设备技术领域，更具体地，涉及一种数据压缩方法、装置、电子设备及存储介质。
背景技术
现有的电子设备，例如手机、平板电脑等，已经成为人们日常生活中最常用的消费型电子产品之一。随着用户需求的不断增加，电子设备中存储及运行的数据越来越多，因此其内存空间以及磁盘空间的利用成为了研究问题。
发明内容
鉴于上述问题，本申请提出了一种数据压缩方法、装置、电子设备及存储介质，以改善上述问题。
第一方面，本申请实施例提供了一种数据压缩方法，所述方法包括：获取内存中的待压缩数据；获取所述待压缩数据在所述内存中持续存储的时间长度，作为第一时间；如果所述第一时间大于或者等于第一时间阈值，则利用第一压缩算法对所述待压缩数据进行压缩，得到第一压缩数据；如果所述第一压缩数据满足预设条件，则对所述第一压缩数据再次进行压缩得到第二压缩数据。
第二方面，本申请实施例提供了一种数据压缩装置，所述装置包括：数据获取模块、时间获取模块、第一压缩模块以及第二压缩模块，其中，数据获取模块用于获取内存中的待压缩数据。时间获取模块，用于获取所述待压缩数据在所述内存中持续存储的时间长度，作为第一时间。第一压缩模块，用于如果所述第一时间大于或者等于第一时间阈值，则利用第一压缩算法对所述待压缩数据进行压缩，得到第一压缩数据。第二压缩模块，用于如果所述第一压缩数据满足预设条件，则对所述第一压缩数据再次进行压缩得到第二压缩数据。
第三方面，本申请实施例提供了一种电子设备，包括：一个或多个处理器；存储器；一个或多个应用程序，其中所述一个或多个应用程序被存储在所述存储器中并被配置为由所述一个或多个处理器执行，所述一个或多个程序配置用于执行上述第一方面提供的数据压缩方法。
第四方面，本申请实施例提供了一种计算机可读取存储介质，所述计算机可读取 存储介质中存储有程序代码，所述程序代码可被处理器调用执行上述第一方面提供的数据压缩方法。
本申请提供的方案，通过获取内存中的待压缩数据以及该待压缩数据在内存中持续存储的时间长度来实现第一压缩，即当待压缩数据在内存中持续存储的时间长度大于或者等于第一时间阈值时，利用第一压缩算法对待压缩数据进行压缩，得到第一压缩数据，而后当该第一压缩数据满足预设条件时，本申请可以对第一压缩数据再次进行压缩得到第二压缩数据，其在一定程度上不仅可以节省内存和磁盘的空间，同时可以提升内存及磁盘利用率。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1示出了根据本申请一个实施例的数据压缩方法流程图。
图2示出了根据本申请另一个实施例的数据压缩方法流程图。
图3示出了根据本申请又一个实施例的数据压缩方法流程图。
图4示出了根据本申请再一个实施例的数据压缩方法流程图。
图5示出了根据本申请一个实施例的数据压缩装置的一种框图。
图6示出了根据本申请一个实施例的数据压缩装置中第二压缩模块的框图。
图7示出了根据本申请一个实施例的数据压缩装置中第二压缩模块的另一框图。
图8示出了根据本申请一个实施例的数据压缩装置中第二压缩模块的又一框图。
图9是本申请实施例的用于执行根据本申请实施例的数据压缩方法的电子设备的框图。
图10是本申请实施例的用于保存或者携带实现根据本申请实施例的数据压缩方法的程序代码的存储单元。
具体实施方式
为了使本技术领域的人员更好地理解本申请方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述。
随着科技水平的发展，时序数据库作为一种时序数据存储系统基础设施，逐渐被广泛应用于监控系统中。时序数据是指同一指标按时间顺序记录的数据序列，而时序 数据库是用于存储和管理时序数据的专业化数据库，即用于存放时间序列的海量数据，这些海量数据按照时间顺序变化，并且需要支持时序数据的快速写入、持久化、多维度的聚合查询等基本功能。电子设备中，内存和磁盘是重要的部件之一，内存是与CPU进行沟通的桥梁，而电子设备中大部分数据均存储于磁盘中，因此内存和磁盘的性能对电子设备的影响较大。
由于时序数据具有海量的特性，所以无法将数据全量存储于电子设备的内存中，现有技术中，通常是把当前指定时间内的数据置于内存中，作为热数据，以此来提供快速的查询服务。鉴于内存成本高，所以数据在内存中的周期超过指定时间后，时序数据库会对数据进行压缩，作为冷数据持久化到磁盘中。然而在多数应用实践中，用户对于持久化手的数据仍有大量的查询需求，即虽然单一的压缩算法可以实现时序数据的高压缩比，但相应的解压性能难以满足大规模冷数据的查询需求，甚至成为电子设备的性能瓶颈。
针对上述问题，发明人提出了本申请实施例提供的数据压缩方法、装置、电子设备以及存储介质，可以对内存中的待压缩数据进行有效的压缩。其中，具体的数据压缩方法在后续的实施例中进行详细的说明。
请参阅图1，图1示出了本申请一个实施例提供的数据压缩方法的流程示意图。所述数据压缩方法用于对电子设备中，当内存中的数据满足预设条件时本申请可以对压缩获取的数据再次进行压缩，从而不仅可以节省内存空间，提升内存利用率，同时可以对第一压缩数据进行再次压缩以节省磁盘的空间利用率。在具体的实施例中，所述数据压缩方法应用于如图5所示的数据压缩装置500以及配置有所述数据压缩装置500的电子设备100(图9)。下面将以电子设备为例，说明本实施例的具体流程，当然，可以理解的，本实施例所应用的电子设备可以为智能手机、平板电脑、电子书、智能手表等，在此不做限定。从图1可以看出该数据压缩方法可以包括步骤S101至步骤S104。
步骤S101：获取内存中的待压缩数据。
在本申请实施例中，待压缩数据可以为首次写入内存的数据，内存(又称随机存取存储器，random access memory，RAM)表示运行程序时使用的内存(即运行内存)，只能临时存储数据，用于与处理器交换高速缓存数据，但是内存本身不能用于长期存储数据。应用在运行的状态下需要占用内存空间来存储运行过程中的数据，不同的应用在不同的运行状态占用的内存空间大小不一定相同。
在一种实施方式中，首次写入内存的数据可以为时序数据中的热数据，时序数据 是指时间序列数据。时间序列数据是依据统一指标按时间顺序记录的数据列，所述时序数据包括时间信息、数据类型和数据信息。其中的时间信息具体可以是时间戳，但本申请中对其具体实现形式不做固定限制，只要是能够区分表示不同时间的数据即可。数据类型可以为已知的或未来可能出现的数据类型，如现有的数据类型可以包括Integer、Long、Float、Double、String、Binary Array、Image等类型。其中的数据信息可以为数据本身，即不包含时间信息和数据类型等描述内容的原始数据。
由于时序数据具有海量的特性所以无法将数据全量存储于电子设备的内存中，为了解决该问题本申请将当前指定时间内的数据置于内存中，作为热数据，而后当该热数据超过指定时间后，电子设备会对其进行压缩，最后作为冷数据持久化。本申请实施例中，热数据可以是频繁访问的数据，通常情况下其存储在高速缓存存储设备中，冷数据是指访问频率较低和访问性能要求较低的数据，如短期内不会访问甚至在未来不再需要的数据。
作为一种实施方式，电子设备可以每间隔设定时长，获取内存中的待压缩数据，以从这些待压缩数据中确定后续可以被压缩的数据，对确定的待压缩数据进行压缩，即进入步骤S102。
作为另一种实施方式，电子设备也可以根据用户输入的触发指令，执行获取内存中当前存储的待压缩数据的操作，以从这些待压缩数据中确定后续可以被压缩的数据，对确定的待压缩数据进行压缩，也就是说进入步骤S102。当然，具体触发步骤S101的方式在本申请实施例中可以不作为限定。
步骤S102：获取所述待压缩数据在所述内存中持续存储的时间长度，作为第一时间。
在一种实施方式中，电子设备获取到内存中的待压缩数据后，可以获取到所述待压缩数据在内存中存储的时间，即获取待压缩数据在内存中持续存储的时间长度，该时间长度可以是待压缩数据的生命周期。换句话说，当待压缩数据进入内存后，电子设备就可以开始执行时间记录操作，而后将其记录的时间作为第一时间。例如，数据A进入内存的时间为2019年9月10日7点45分，当前时刻为2019年9月10日16点45分，此时数据A在内存中持续存储的时间长度则为7小时。
步骤S103：如果所述第一时间大于或者等于第一时间阈值，则利用第一压缩算法对所述待压缩数据进行压缩，得到第一压缩数据。
本申请实施例中，电子设备获取到待压缩数据在内存中持续存储的时间长度第一时间后，可以判断所述第一时间是否大于或者等于第一时间阈值，如果第一时间大于 或者等于第一时间阈值，则利用第一压缩算法对所述待压缩数据进行压缩，得到第一压缩数据。其中，第一时间阈值可以是预先设置好的，也可以根据经验值获取的，或者也可以是内存中所有数据生命周期的平均值，第一时间阈值具体为哪个这里不进行明确限制。
在一种实施方式中，电子设备获取到第一时间和第一时间阈值后，可以将所述第一时间和第一时间阈值进行对比，并判断所述第一时间与第一时间阈值是否相等，如果第一时间与第一时间阈值相等，则可以利用第一压缩算法对第一时间对应的待压缩数据进行压缩。例如，内存中存在待压缩数据A和待压缩数据B，待压缩数据A对应的第一时间T A＝5小时，待压缩数据B对应的时间为T B＝1小时，第一时间阈值T为5小时，通过对比可以发现待压缩数据A对应的第一时间T A与第一时间阈值T相同，此时就可以利用第一压缩算法对待压缩数据A进行压缩。
本申请实施例中，第一压缩算法包括但不限于DFCM((Differential Finite Context Metho，差分有限上下文预测方法)算法、FCM(Finite Context Method，有限上下文预测方法)算法或者LZ4算法等。其中，LZ4压缩算法是一种对以快速和资源开销小为目标的压缩算法。在另一些实施例中，电子设备中可以包括多个第一压缩算法，而后可以利用所述多个第一压缩算法分别对待压缩数据进行压缩，得到多个压缩结果，并将所述多个压缩结果进行排序比对，将占用存储最小的压缩结果作为目标压缩结果，最后将该目标压缩结果对应的压缩算法作为目标压缩算法。
在一些实施方式中，电子设备在利用第一压缩算法对待压缩数据进行压缩时，可以先判断即将要压缩的数据是否为二进制机器语言，如果待压缩数据不是二进制机器语言，则利用第一压缩算法对所述待压缩数据进行压缩。另外，如果待压缩数据为二进制机器语言，则不对其进行压缩。因为二进制机器语言经过压缩得到的数据所占磁盘空间多数不会发生改变，有时候二进制机器语言经过压缩获取的数据占用磁盘空间反而会更大，且对二进制机器语言进行压缩不仅会增加数据压缩的时间，而且在一定程度上会浪费电子设备的系统资源，所以当待压缩数据为二进制机器语言时，本申请可以不对其进行压缩。
需要说明的是，利用第一压缩算法对所述待压缩数据进行压缩得到第一压缩数据之后，电子设备可以将所述第一压缩数据进行持久化，得到持久化数据，持久化数据是信息处理中访问频率较低并且不会修改的数据，即所谓持久化就是将第一压缩数据保存到磁盘中，使得在应用程序或机器重启后可以继续访问之前保存的该数据。本申请可以利用属性列表plist、偏好设置preference、归档解档NSKeyedArchiver、数据库 SQLite或者数据库CoreData进行数据的持久化。其中，属性列表是一种明文的轻量级存储方式、其存储格式有多种，最常规格式为xml格式，属性列表的安全性很低，其主要用于指定应用的默认设置于相应的数据存储；偏好设置主要用于保存应用程序的配置信息，其可以将电子设备中的所有数据保存在同一个文件中；归档、解档是进行数据加密处理的，其安全性较高；数据库SQLite是一个轻量级、跨平台的小型数据库、其拥有可移植性高、有着和MySql几乎相同的数据库语句以及无需服务器即可使用的优点；数据库CoreData：将本地数据放到内存中使得访问速度更快。本申请可以只利用上述任意一种持久化方式对第一压缩数据进行持久化，也可以结合上述任意两种或者两种以上的持久化方式对第一压缩数据进行持久化，具体如何将第一压缩数据进行持久化这里不进行明确限制，可以根据实际需求进行选择。
步骤S104：如果所述第一压缩数据满足预设条件，则对所述第一压缩数据再次进行压缩得到第二压缩数据。
本申请实施例中，电子设备将第一压缩数据持久化到磁盘后，可以对所述第一压缩数据进行持续监测，并判断所述第一压缩数据是否满足预设条件，如果所述第一压缩数据满足预设条件，则对所述第一压缩数据进行压缩得到第二压缩数据；如果所述第一压缩数据不满足预设条件，则不对其进行压缩，即保证所述第一压缩数据不发生改变。其中，预设条件可以是：第一压缩数据在磁盘中持续存储的时间长度大于第二时间阈值，也可以是第一压缩数据的使用次数大于预设次数，或者是所述第一压缩数据的使用时长大于预设时长等。
本申请实施例提供的数据压缩方法，通过获取内存中的待压缩数据和该数据在内存中持续存储的时间长度得到第一压缩数据，即当第一时间大于或者等于第一时间阈值，利用第一压缩算法对待压缩数据进行压缩，得到第一压缩数据，而后当第一压缩数据满足预设条件时，电子设备可以对第一压缩数据再次进行压缩得到第二压缩数据，本申请通过对数据的梯度压缩，不仅可以节省内存空间，同时可以提升磁盘空间的利用率。另外，当待压缩数据为二进制机器语言时本申请可以不对其进行压缩，如此在一定程度上不仅可以减少因数据压缩造成的时间浪费，而且可以提高电子设备系统资源的利用率。
请参阅图2，图2示出了本申请另一个实施例提供的数据压缩方法的流程示意图。该方法应用于上述电子设备，下面将针对图2所示的流程进行详细的阐述，所从图2可以看出数据压缩方法可以包括步骤S201至步骤S205。
步骤S201：获取内存中的待压缩数据。
步骤S202：获取所述待压缩数据在所述内存中持续存储的时间长度，作为第一时间；
步骤S203：如果所述第一时间大于或者等于第一时间阈值，则利用第一压缩算法对所述待压缩数据进行压缩，得到第一压缩数据。
在本申请实施例中，步骤S201以及步骤S203可以参阅前述实施例的内容，在此不再赘述。
步骤S204：获取所述第一压缩数据在磁盘中持续存储的时间长度，作为第二时间。
通过上述介绍可以知道，电子设备利用第一压缩算法对待压缩数据进行压缩得到第一压缩数据之后，可以将所述第一压缩数据存储至电子设备的磁盘中，而当所述第一压缩数据存储至磁盘后电子设备可以对其存储的时间进行记录，该时间可以为起始时间。例如，第一压缩数据A于2019年9月18日19点20分存储至电子设备磁盘的，则其起始时间即为2019年9月18日19点20分。本申请实施例中，电子设备可以获取所述第一压缩数据在磁盘中持续存储的时间长度，作为第二时间，即第二时间为当前时间与起始时间之间的时间差。例如，第一压缩数据A的起始时间为2019年9月18日19点20分，当前时间为2019年9月19日20点30分，此时即可确定出第一压缩数据在磁盘中持续存储的时间长度为25小时10分钟，即第二时间为25小时10分钟。
步骤S205：如果所述第二时间大于第二时间阈值，则利用第二压缩算法对所述第一压缩数据再次进行压缩，所述第二压缩算法与第一算法不同。
本申请实施例中，电子设备获取到第一压缩数据在磁盘中持续存储的时间长度第二时间后，可以判断所述第二时间是否大于第二时间阈值，如果第二时间大于第二时间阈值，则利用第二压缩算法对第二压缩数据再次进行压缩；如果第二时间小于第二时间阈值则不对所述第一压缩数据再次进行压缩。在一些实施方式中，当第一时间等于第二时间阈值时，电子设备可以利用第二压缩算法对所述第一压缩数据再次进行压缩。其中，第二压缩算法与第一压缩算法不同，本申请中第二压缩算法为异或差量压缩算法(exclusive OR，XOR算法)，异或差量压缩算法是一种基于数据点间分级XOR的差量数据压缩编码算法，通过存储数据的差量可以极大的提高数据的压缩率。
在一些实施方式中，对第一压缩数据再次进行压缩得到第二压缩数据之后，电子设备可以获取其磁盘的剩余存储空间，如果所述磁盘的剩余存储空间小于存储阈值，电子设备则可以对所述第二压缩数据再次进行压缩，并重新获取所述磁盘的剩余存储空间，直至所述磁盘的剩余存储空间大于存储阈值。另外，在此过程中电子设备可以 对第二压缩数据被压缩的次数进行统计，即获取第二压缩数据的被压缩次数，并判断所述被压缩次数是否小于设定次数，如果被压缩次数小于设定次数，则对第二压缩数据再次进行压缩。例如，对第一压缩数据A再次进行压缩得到第二压缩数据B，获取到磁盘的剩余存储空间为285M，而存储阈值为300M，显然，剩余存储空间为285M小于存储阈值为300M，此时则可以对所述第二压缩数据B再次压缩，再次获取磁盘的剩余存储空间，并判断其是否小于存储阈值。在此过程中，电子设备可以记录压缩操作的次数，即第二压缩数据的被压缩次数，当该次数小于设定次数，则继续执行压缩操作，而当压缩次数大于或者等于设定次数，则停止压缩操作。例如，经过统计确定出针对所述第二压缩数据B执行的压缩操作次数为3次，而设定次数也为3次，此时则停止压缩。
在另一些实施方式中，如果被压缩次数小于设定次数，电子设备则可以获取该第二压缩数据对应的压缩率，所述压缩率为所述第二压缩数据在被压缩前其数据大小和被压缩后数据大小之间的比值，即待压缩数据被压缩前后的数据比值，在此过程中电子设备可以判断所述第二压缩数据对应的压缩率是否小于预设压缩率。如果第二压缩数据对应的压缩率小于预设压缩率，则可以对所述第二压缩数据再次进行压缩，并重新获取所述第二压缩数据对应的压缩率，并再次判断该压缩率是否小于预设压缩率，直至所述压缩率小于预设压缩率。可以理解的，当第二压缩数据被压缩多次后，第二压缩数据被压缩的层级越深(压缩次数越多)，其每次可压缩的难度会加大，压缩大小相应的也会变小，因此，本申请可以利用压缩率，来确定是否继续对第二压缩数据进行压缩。压缩率指第二压缩数据压缩后的大小与压缩前的大小之比。例如，第二压缩数据在进行第一层级的压缩时，压缩率可以为0.33，第二压缩数据在进行第二层级的压缩时，压缩率可以为0.44，在进行第三层级的压缩时，压缩率可以为0.5，当然，以上压缩率仅为举例，并不代表对本申请实施例中具体的压缩率的限定。
本申请实施例提供的数据压缩方法，通过获取第一压缩数据在磁盘中持续存储的时间长度，作为第二时间来确定是否将所述第一压缩数据进行压缩，具体的，当第二时间大于第二时间阈值时，本申请可以利用第二压缩算法对第一压缩数据再次进行压缩，电子设备可以自动对待压缩数据进行多阶段压缩，不仅可以使持久化数据保持最高压缩比，而且可以解决冷数据访问性能的问题，在一定程度上可以打破用户对冷数据大量查询的性能瓶颈，提高用户的使用体验。
请参阅图3，图3示出了本申请又一个实施例提供的数据压缩方法的流程示意图。该方法应用于上述电子设备，下面将针对图3所示的流程进行详细的阐述，所从图3 可以看出数据压缩方法可以包括步骤S301至步骤S305。
步骤S301：获取内存中的待压缩数据。
步骤S302：获取所述待压缩数据在所述内存中持续存储的时间长度，作为第一时间；
步骤S303：如果所述第一时间大于或者等于第一时间阈值，则利用第一压缩算法对所述待压缩数据进行压缩，得到第一压缩数据。
在本申请实施例中，步骤S301以及步骤S303可以参阅前述实施例的内容，在此不再赘述。
步骤S304：获取所述第一压缩数据的使用次数，并判断所述第一压缩数据的使用次数是否小于预设次数。
在一种实施方式中，电子设备利用第一压缩算法对待压缩数据进行压缩得到第一压缩数据，而后可以将所述第一压缩数据存储至电子设备的磁盘中，电子设备可以检测该第一压缩数据是否被使用，如果被使用则可以获取其被使用的次数。本申请实施例中，电子设备获取到第一压缩数据的使用次数时可以判断所述第一压缩数据的使用次数是否小于预设次数，如果第一压缩数据的使用次数小于预设次数，则利用第二压缩算法对所述第一压缩数据再次进行压缩，即进入步骤S305；如果第一压缩数据的使用次数大于预设次数，则不对所述第二压缩数据进行再次压缩。第一压缩数据的使用次数越多，则表示该第一压缩数据被使用的可能性越高，及数据被使用的次数越多，则其压缩率越高，以避免第一压缩数据被再次使用时的解压难度较大，增加电子设备的系统负担，本申请对使用次数大于预设次数的第一压缩数据不进行再次压缩。需要说明的是，预设次数可以是预先设置的，也可以是根据所述第一压缩数据的使用次数分析获取的。例如，所述使用次数可以是多个第一压缩数据中使用次数最少的，本申请对预设次数的获取不进行明确限制。
步骤S305：如果所述第一压缩数据的使用次数小于预设次数，则利用第二压缩算法对所述第一压缩数据再次进行压缩，所述第二压缩算法与第一算法不同。
在本申请实施例中，步骤S305可以参阅前述实施例的内容，在此不再赘述。
在一些实施方式中，电子设备对第一压缩数据再次进行压缩得到第二压缩数据之后，其可以判断电子设备的磁盘中是否包括多个第二压缩数据，如果磁盘中包括多个第二压缩数据，则可以获取每个所述第二压缩数据的访问优先级，并对所述访问优先级进行排序，而后根据所述访问优先级的顺序对第二压缩数据再次进行压缩。
在另一些实施方式中，电子设备可以根据第二压缩数据的访问时间和访问优先级 获取每个第二压缩数据的优先级。具体的，先获取每个所述第二压缩数据的访问时间和访问次数，而后通过每个所述第二压缩数据的访问时间和访问次数，获取每个所述第二压缩数据的访问总时间，并根据所述访问总时间确定每个所述第二压缩数据的访问优先级。其中，访问总时间越长其对应的访问优先级越高。另外，本申请可以根据访问优先级从低到高的顺序对第二压缩数据再次进行压缩，即电子设备在确定磁盘中包括多个第二压缩数据时，可以先对优先级较低的数据进行压缩。
需要说明的是，如果同一优先级存在多个第二压缩数据，本申请则可以获取所述同一优先级中每个第二压缩数据的最近一次被使用的时间，作为第三时间，而后对每个所述第三时间按照时间先后顺序进行排序，获得排序结果，最后根据该排序结果对同一优先级的所述第二压缩数据进行压缩。
本申请实施例提供的数据压缩方法，通过判断第一压缩数据的使用次数是否满足预设条件来确定是否对第一压缩数据进行再次压缩，即判断第一压缩数据的使用次数是否小于预设次数，如果小于预设次数，则利用第二压缩算法对第一压缩数据再次进行压缩。本申请通过将第一压缩数据的使用次数和预设次数比较，避免不必要数据的解压缩，其在节省磁盘空间，提升磁盘利用率的同时，也避免了第一压缩数据被再次使用时的解压难度较大，降低电子设备的系统负担。
请参阅图4，图4示出了本申请再一个实施例提供的数据压缩方法的流程示意图。该方法应用于上述电子设备，下面将针对图4所示的流程进行详细的阐述，所从图4可以看出数据压缩方法可以包括步骤S401至步骤S405。
步骤S401：获取内存中的待压缩数据。
步骤S402：获取所述待压缩数据在所述内存中持续存储的时间长度，作为第一时间；
在本申请实施例中，步骤S401以及步骤S402可以参阅前述实施例的内容，在此不再赘述。
步骤S403：如果所述第一时间大于或者等于第一时间阈值，则利用第一压缩算法对所述待压缩数据进行压缩，得到第一压缩数据。
本申请实施例中，利用第一压缩算法对待压缩数据进行压缩时，可以先判断内存中是否包含多个待压缩数据，如果内存中包括多个待压缩数据，电子设备则可以判断所述多个待压缩数据中，是否包括重复数据，如果多个待压缩数据中存在重复数据，则对所述多个待压缩数据进行去重操作，并对执行去重操作后的待压缩数据进行压缩。例如，内存中包括待压缩数据A、待压缩数据B和待压缩数据C，通过对比判定待压 缩数据A和待压缩数据C为重复数据，此时则可以对待压缩数据A和待压缩数据C执行去重操作，即去掉待压缩数据A或者去掉待压缩数据C，而后对剩余的待压缩数据进行压缩，如果去掉待压缩数据A，则对待压缩数据B和待压缩数据C进行压缩，如果去掉待压缩数据C，则对待压缩数据A和待压缩数据B进行压缩。
在另一些实施方式中，电子设备利用第一压缩算法对待压缩数据进行压缩，得到第一压缩数据之后，其可以获取所述待压缩数据在被压缩前数据占用内存的大小，以及该待压缩数据被压缩后其占用内存的大小。然后，电子设备可以判断所述第一压缩数据占用内存的大小是否小于待压缩数据占用内存的大小，如果第一压缩数据占用内存的大小小于待压缩数据占用内存的大小，则对所述第一压缩数据进行存储。另外，电子设备也可以判断第一压缩数据占用磁盘的大小是否小于待压缩数据占用磁盘的大小，如果第一压缩数据占用磁盘的大小小于待压缩数据占用磁盘的大小，则将第一压缩数据存储于磁盘中，而如果第一压缩数据占用磁盘的大小大于或者等于待压缩数据占用磁盘的大小，则不对所述第一压缩数据进行存储。如此可以避免无用的压缩操作为磁盘空间带来不必要的负担。例如，待压缩数据A在未压缩之前其占用磁盘的空间是102M，被压缩后其占用磁盘的空间为150M，显然，将待压缩数据A压缩后其占用磁盘的空间不仅没有减少，而且增大了，此时则不需要对压缩后的数据进行存储，即保持待压缩数据A不变。
步骤S404：获取所述第一压缩数据的使用时长，并判断所述第一压缩数据的使用时长是否小于预设时长。
在一种实施方式中，电子设备利用第一压缩算法对待压缩数据进行压缩得到第一压缩数据，而后可以将所述第一压缩数据存储至电子设备的磁盘中，电子设备可以检测该第一压缩数据是否被使用，如果被使用则记录其被使用的时长。本申请实施例中，第一压缩数据的使用时长可以是单次使用时长，也可以是多次使用的总时长，具体是哪个这里不进行明确限制可以根据实际情况进行选择。
可选地，电子设备获取到第一压缩数据的使用时长后可以判断所述第一压缩数据的时长是否小于预设时长，如果第一压缩数据的使用时长小于预设时长，则利用第二压缩算法对所述第一压缩数据再次进行压缩，即进入步骤S405；如果第一压缩数据的使用时长大于预设时长，则不对所述第二压缩数据进行再次压缩。第一压缩数据的使用时长越长，则表示该第一压缩数据越重要，以避免第一压缩数据被使用时的解压难度较大，增加电子设备的系统负担，因此本申请对使用时长大于预设时长的第一压缩数据不进行再次压缩。需要说明的是，预设时长可以是预先设置的，也可以是根据所 述第一压缩数据的使用时长分析获取的。例如，所述使用时长可以是多个第一压缩数据中使用时长最短的，本申请对预设时长的获取不进行明确限制。
步骤S405：如果所述第一压缩数据的使用时长小于预设时长，则利用第二压缩算法对所述第一压缩数据再次进行压缩，所述第二压缩算法与第一算法不同。
在本申请实施例中，步骤S405可以参阅前述实施例的内容，在此不再赘述。
本申请实施例提供的数据压缩方法，通过判断第一压缩数据的使用时长是否满足预设条件来确定是否对第一压缩数据进行再次压缩，即判断第一压缩数据的使用时长是否小于预设时长，如果小于预设时长，则利用第二压缩算法对第一压缩数据再次进行压缩。本申请通过将第一压缩数据的使用时长和预设时长比较，避免不必要数据的解压缩，其不但可以在节省磁盘的空间，提升磁盘的利用率，同时也可以降低电子设备系统的负担。
请参阅图5，其示出了本申请实施例提供的一种数据压缩装置500的结构框图。该数据压缩装置500应用上述的电子设备，该数据压缩装置500包括：数据获取模块501、时间获取模块502、第一压缩模块503以及第二压缩模块504。
数据获取模块501，用于获取内存中的待压缩数据。
时间获取模块502，用于获取所述待压缩数据在所述内存中持续存储的时间长度，作为第一时间。
第一压缩模块503，用于如果所述第一时间大于或者等于第一时间阈值，则利用第一压缩算法对所述待压缩数据进行压缩，得到第一压缩数据。
进一步的，第一压缩模块503还用于当所述内存中包括多个待压缩数据时，判断所述多个待压缩数据中是否存在重复数据，如果所述多个待压缩数据中存在重复数据，则对所述多个待压缩数据进行去重操作，并对执行去重操作后的待压缩数据进行压缩。
进一步的，利用第一压缩算法对所述待压缩数据进行压缩，得到第一压缩数据之后，包括：判断所述第一压缩数据的大小是否小于待压缩数据的大小；如果小于，则对所述第一压缩数据进行存储。如果第一压缩数据的大小大于或者等于待压缩数据的大小，则不对所述第一压缩数据进行存储。
进一步的，第一压缩模块503还用于判断所述待压缩数据是否为二进制机器语言，如果待压缩数据不是二进制机器语言，则利用第一压缩算法对所述待压缩数据进行压缩。如果待压缩数据为二进制机器语言，则不对所述待压缩数据进行压缩。
第二压缩模块504，用于如果所述第一压缩数据满足预设条件，则对所述第一压缩数据再次进行压缩得到第二压缩数据。
请参阅图6，第二压缩模块504可以包括时间长度获取单元5041和第一压缩单元5042。
时间长度获取单元5041，用于获取所述第一压缩数据在磁盘中持续存储的时间长度，作为第二时间。
第一压缩单元5042，用于如果所述第二时间大于第二时间阈值，则利用第二压缩算法对所述第一压缩数据再次进行压缩，所述第二压缩算法与第一算法不同。
请参阅图7，第二压缩模块504可以包括时间长度获取单元5043和第二压缩单元5044。
使用次数获取单元5043，用于获取所述第一压缩数据的使用次数，并判断所述第一压缩数据的使用次数是否小于预设次数。
第二压缩单元5044，用于如果所述第一压缩数据的使用次数小于预设次数，则利用第二压缩算法对所述第一压缩数据再次进行压缩，所述第二压缩算法与第一算法不同。
请参阅图8，第二压缩模块504可以包括时间长度获取单元5045和第三压缩单元5046。
使用时长获取单元5045，用于获取所述第一压缩数据的使用时长，并判断所述第一压缩数据的使用时长是否小于预设时长。
第三压缩单元5046，用于如果所述第一压缩数据的使用时长小于预设时长，则利用第二压缩算法对所述第一压缩数据再次进行压缩，所述第二压缩算法与第一算法不同。
进一步的，所述第二压缩算法为异或差量压缩算法。
进一步的，该数据压缩装置500还用于获取磁盘的剩余存储空间，如果所述磁盘的剩余存储空间小于存储阈值，则对所述第二压缩数据再次进行压缩，并重新获取所述磁盘的剩余存储空间，直至所述磁盘的剩余存储空间大于存储阈值。对所述第二压缩数据再次进行压缩，包括：获取所述第二压缩数据的被压缩次，如果所述被压缩次数小于设定次数，则对所述第二压缩数据再次进行压缩。另外，如果所述被压缩次数小于设定次数，则对所述第二压缩数据再次进行压缩，包括：如果所述被压缩次数小于设定次数，则获取所述第二压缩数据对应的压缩率，所述压缩率为所述第二压缩数据压缩后与压缩前的数据比值，当所述第二压缩数据对应的压缩率小于预设压缩率时，对所述第二压缩数据再次进行压缩，并重新获取所述第二压缩数据对应的压缩率，直至所述压缩率小于预设压缩率。
进一步的，该数据压缩装置500还用于判断磁盘中是否包括多个第二压缩数据，如果所述磁盘中包括多个第二压缩数据，则获取每个所述第二压缩数据的访问优先级，并对所述访问优先级进行排序，根据所述访问优先级的顺序对所述第二压缩数据再次进行压缩。获取每个所述第二压缩数据的访问优先级，包括：获取每个所述第二压缩数据的访问时间和访问次数，通过每个所述第二压缩数据的访问时间和访问次数，获取每个所述第二压缩数据的访问总时间，根据所述访问总时间确定每个所述第二压缩数据的访问优先级，所述访问总时间越长对应的访问优先级越高。根据所述访问优先级的顺序对所述第二压缩数据再次进行压缩，包括：根据所述访问优先级从低到高的顺序对所述第二压缩数据再次进行压缩。另外，根据所述访问优先级从低到高的顺序对所述第二压缩数据再次进行压缩，包括：如果同一优先级包括多个第二压缩数据，则获取同一优先级中每个所述第二压缩数据的最近一次被使用的时间，作为第三时间，对每个所述第三时间按照时间先后顺序进行排序，获得排序结果，根据排序结果对同一优先级的第二压缩数据进行压缩。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述装置和模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，模块相互之间的耦合可以是电性，机械或其它形式的耦合。
另外，在本申请各个实施例中的各功能模块可以集成在一个处理模块中，也可以是各个模块单独物理存在，也可以两个或两个以上模块集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。
综上所述，本申请提供的方案，通过获取内存中当前已被压缩的内存页，获取内存页的历史使用数据，根据历史使用数据，获取内存页中满足历史使用条件的内存页，作为目标内存页，对目标内存页再次进行压缩并存储，从而实现对内存页的深度压缩，可以节省内存空间，提升内存利用率。
请参考图9，其示出了本申请实施例提供的一种电子设备的结构框图。该电子设备100可以是智能手机、平板电脑、电子书、智能穿戴设备等能够运行应用程序的电子设备。本申请中的电子设备100可以包括一个或多个如下部件：处理器110、存储器120、以及一个或多个应用程序，其中一个或多个应用程序可以被存储在存储器120中并被配置为由一个或多个处理器110执行，一个或多个程序配置用于执行如前述方法实施例所描述的方法。
处理器110可以包括一个或者多个处理核。处理器110利用各种接口和线路连接 整个电子设备100内的各个部分，通过运行或执行存储在存储器120内的指令、程序、代码集或指令集，以及调用存储在存储器120内的数据，执行电子设备100的各种功能和处理数据。可选地，处理器110可以采用数字信号处理(Digital Signal Processing，DSP)、现场可编程门阵列(Field－Programmable Gate Array，FPGA)、可编程逻辑阵列(Programmable Logic Array，PLA)中的至少一种硬件形式来实现。处理器110可集成中央处理器(Central Processing Unit，CPU)、图像处理器(Graphics Processing Unit，GPU)和调制解调器等中的一种或几种的组合。其中，CPU主要处理操作系统、用户界面和应用程序等；GPU用于负责显示内容的渲染和绘制；调制解调器用于处理无线通信。可以理解的是，上述调制解调器也可以不集成到处理器110中，单独通过一块通信芯片进行实现。
存储器120可以包括随机存储器(Random Access Memory，RAM)，也可以包括只读存储器(Read-Only Memory)。存储器120可用于存储指令、程序、代码、代码集或指令集。存储器120可包括存储程序区和存储数据区，其中，存储程序区可存储用于实现操作系统的指令、用于实现至少一个功能的指令(比如触控功能、声音播放功能、图像播放功能等)、用于实现下述各个方法实施例的指令等。存储数据区还可以存储终端100在使用中所创建的数据(比如电话本、音视频数据、聊天记录数据)等。
请参考图10，其示出了本申请实施例提供的一种计算机可读存储介质的结构框图。该计算机可读介质800中存储有程序代码，所述程序代码可被处理器调用执行上述方法实施例中所描述的方法。
计算机可读存储介质800可以是诸如闪存、EEPROM(电可擦除可编程只读存储器)、EPROM、硬盘或者ROM之类的电子存储器。可选地，计算机可读存储介质800包括非易失性计算机可读介质(non-transitory computer-readable storage medium)。计算机可读存储介质800具有执行上述方法中的任何方法步骤的程序代码810的存储空间。这些程序代码可以从一个或者多个计算机程序产品中读出或者写入到这一个或者多个计算机程序产品中。程序代码810可以例如以适当形式进行压缩。
最后应说明的是：以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不驱使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。

Claims (20)

  1. 一种数据压缩方法，其特征在于，包括：
    获取内存中的待压缩数据；
    获取所述待压缩数据在所述内存中持续存储的时间长度，作为第一时间；
    如果所述第一时间大于或者等于第一时间阈值，则利用第一压缩算法对所述待压缩数据进行压缩，得到第一压缩数据；
    如果所述第一压缩数据满足预设条件，则对所述第一压缩数据再次进行压缩得到第二压缩数据。
  2. 根据权利要求1所述的方法，其特征在于，所述如果所述第一压缩数据满足预设条件，则对所述第一压缩数据再次进行压缩得到第二压缩数据，包括：
    获取所述第一压缩数据在磁盘中持续存储的时间长度，作为第二时间；
    如果所述第二时间大于第二时间阈值，则利用第二压缩算法对所述第一压缩数据再次进行压缩，所述第二压缩算法与第一算法不同。
  3. 根据权利要求1所述的方法，其特征在于，所述如果所述第一压缩数据满足预设条件，则对所述第一压缩数据再次进行压缩得到第二压缩数据，包括：
    获取所述第一压缩数据的使用次数，并判断所述第一压缩数据的使用次数是否小于预设次数；
    如果所述第一压缩数据的使用次数小于预设次数，则利用第二压缩算法对所述第一压缩数据再次进行压缩，所述第二压缩算法与第一算法不同。
  4. 根据权利要求1所述的方法，其特征在于，所述如果所述第一压缩数据满足预设条件，则对所述第一压缩数据再次进行压缩得到第二压缩数据，包括：
    获取所述第一压缩数据的使用时长，并判断所述第一压缩数据的使用时长是否小于预设时长；
    如果所述第一压缩数据的使用时长小于预设时长，则利用第二压缩算法对所述第一压缩数据再次进行压缩，所述第二压缩算法与第一算法不同。
  5. 根据权利要求1-4任一项所述的方法，其特征在于，所述对所述第一压缩数据再次进行压缩得到第二压缩数据之后，包括：
    获取磁盘的剩余存储空间；
    如果所述磁盘的剩余存储空间小于存储阈值，则对所述第二压缩数据再次进行压 缩，并重新获取所述磁盘的剩余存储空间，直至所述磁盘的剩余存储空间大于存储阈值。
  6. 根据权利要求5所述的方法，其特征在于，所述对所述第二压缩数据再次进行压缩，包括：
    获取所述第二压缩数据的被压缩次数；
    如果所述被压缩次数小于设定次数，则对所述第二压缩数据再次进行压缩。
  7. 根据权利要求6所述的方法，其特征在于，所述如果所述被压缩次数小于设定次数，则对所述第二压缩数据再次进行压缩，包括：
    如果所述被压缩次数小于设定次数，则获取所述第二压缩数据对应的压缩率，所述压缩率为所述第二压缩数据压缩后与压缩前的数据比值；
    当所述第二压缩数据对应的压缩率小于预设压缩率时，对所述第二压缩数据再次进行压缩，并重新获取所述第二压缩数据对应的压缩率，直至所述压缩率小于预设压缩率。
  8. 根据权利要求1-4任一项所述的方法，其特征在于，所述对所述第一压缩数据再次进行压缩得到第二压缩数据之后，包括：
    判断磁盘中是否包括多个第二压缩数据；
    如果所述磁盘中包括多个第二压缩数据，则获取每个所述第二压缩数据的访问优先级，并对所述访问优先级进行排序；
    根据所述访问优先级的顺序对所述第二压缩数据再次进行压缩。
  9. 根据权利要求8所述的方法，其特征在于，所述获取每个所述第二压缩数据的访问优先级，包括：
    获取每个所述第二压缩数据的访问时间和访问次数；
    通过每个所述第二压缩数据的访问时间和访问次数，获取每个所述第二压缩数据的访问总时间；
    根据所述访问总时间确定每个所述第二压缩数据的访问优先级，所述访问总时间越长对应的访问优先级越高。
  10. 根据权利要求8所述的方法，其特征在于，所述根据所述访问优先级的顺序对所述第二压缩数据再次进行压缩，包括：
    根据所述访问优先级从低到高的顺序对所述第二压缩数据再次进行压缩。
  11. 根据权利要求10任一项所述的方法，其特征在于，所述根据所述访问优先级从低到高的顺序对所述第二压缩数据再次进行压缩，包括：
    如果同一优先级包括多个第二压缩数据，则获取同一优先级中每个所述第二压缩数据的最近一次被使用的时间，作为第三时间；
    对每个所述第三时间按照时间先后顺序进行排序，获得排序结果；
    根据排序结果对同一优先级的第二压缩数据进行压缩。
  12. 根据权利要求1-4任一项所述的方法，其特征在于，所述利用第一压缩算法对所述待压缩数据进行压缩，包括：
    当所述内存中包括多个待压缩数据时，判断所述多个待压缩数据中是否存在重复数据；
    如果所述多个待压缩数据中存在重复数据，则对所述多个待压缩数据进行去重操作，并对执行去重操作后的待压缩数据进行压缩。
  13. 根据权利要求1-4任一项所述的方法，其特征在于，所述利用第一压缩算法对所述待压缩数据进行压缩，得到第一压缩数据之后，包括：
    判断所述第一压缩数据的大小是否小于待压缩数据的大小；
    如果小于，则对所述第一压缩数据进行存储。
  14. 根据权利要求13所述的方法，其特征在于，所述方法还包括：
    如果第一压缩数据的大小大于或者等于待压缩数据的大小，则不对所述第一压缩数据进行存储。
  15. 根据权利要求1-4任一项所述的方法，其特征在于，所述利用第一压缩算法对所述待压缩数据进行压缩，包括：
    判断所述待压缩数据是否为二进制机器语言；
    如果待压缩数据不是二进制机器语言，则利用第一压缩算法对所述待压缩数据进行压缩。
  16. 根据权利要求15所述的方法，其特征在于，所述方法还包括：
    如果待压缩数据为二进制机器语言，则不对所述待压缩数据进行压缩。
  17. 根据权利要求2-4任一项所述的方法，其特征在于，所述第二压缩算法为异或差量压缩算法。
  18. 一种数据压缩装置，其特征在于，所述装置包括：
    数据获取模块，用于获取内存中的待压缩数据；
    时间获取模块，用于获取所述待压缩数据在所述内存中持续存储的时间长度，作为第一时间；
    第一压缩模块，用于如果所述第一时间大于或者等于第一时间阈值，则利用第一 压缩算法对所述待压缩数据进行压缩，得到第一压缩数据；
    第二压缩模块，用于如果所述第一压缩数据满足预设条件，则对所述第一压缩数据再次进行压缩得到第二压缩数据。
  19. 一种电子设备，其特征在于，包括：
    一个或多个处理器；
    存储器；
    一个或多个应用程序，其中所述一个或多个应用程序被存储在所述存储器中并被配置为由所述一个或多个处理器执行，所述一个或多个程序配置用于执行如权利要求1-17任一项所述的方法。
  20. 一种计算机可读取存储介质，其特征在于，所述计算机可读取存储介质中存储有程序代码，所述程序代码可被处理器调用执行如权利要求1-17任一项所述的方法。
PCT/CN2020/093846 2020-06-02 2020-06-02 数据压缩方法、装置、电子设备及存储介质 WO2021243531A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2020/093846 WO2021243531A1 (zh) 2020-06-02 2020-06-02 数据压缩方法、装置、电子设备及存储介质
CN202080099532.9A CN115398406A (zh) 2020-06-02 2020-06-02 数据压缩方法、装置、电子设备及存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/093846 WO2021243531A1 (zh) 2020-06-02 2020-06-02 数据压缩方法、装置、电子设备及存储介质

Publications (1)

Publication Number Publication Date
WO2021243531A1 true WO2021243531A1 (zh) 2021-12-09

Family

ID=78830081

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/093846 WO2021243531A1 (zh) 2020-06-02 2020-06-02 数据压缩方法、装置、电子设备及存储介质

Country Status (2)

Country Link
CN (1) CN115398406A (zh)
WO (1) WO2021243531A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114741205A (zh) * 2022-06-02 2022-07-12 荣耀终端有限公司 一种匿名页回收方法及电子设备
CN115905168A (zh) * 2022-11-15 2023-04-04 本原数据(北京)信息技术有限公司 自适应压缩方法和压缩装置、计算机设备、存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102779060A (zh) * 2012-07-11 2012-11-14 烽火通信科技股份有限公司 一种应用程序多次压缩解压方法
CN103744627A (zh) * 2014-01-26 2014-04-23 武汉英泰斯特电子技术有限公司 对实时采集的数据压缩存储的方法及系统
CN104125458A (zh) * 2013-04-27 2014-10-29 展讯通信（上海）有限公司 内存数据无损压缩方法及装置
CN107193968A (zh) * 2017-05-25 2017-09-22 努比亚技术有限公司 一种位置信息存储的方法、装置及计算机可读存储介质
CN110765031A (zh) * 2019-09-27 2020-02-07 Oppo（重庆）智能科技有限公司 数据存储方法、装置、移动终端及存储介质
CN110908608A (zh) * 2019-11-22 2020-03-24 苏州浪潮智能科技有限公司 一种存储空间节省方法及系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102779060A (zh) * 2012-07-11 2012-11-14 烽火通信科技股份有限公司 一种应用程序多次压缩解压方法
CN104125458A (zh) * 2013-04-27 2014-10-29 展讯通信（上海）有限公司 内存数据无损压缩方法及装置
CN103744627A (zh) * 2014-01-26 2014-04-23 武汉英泰斯特电子技术有限公司 对实时采集的数据压缩存储的方法及系统
CN107193968A (zh) * 2017-05-25 2017-09-22 努比亚技术有限公司 一种位置信息存储的方法、装置及计算机可读存储介质
CN110765031A (zh) * 2019-09-27 2020-02-07 Oppo（重庆）智能科技有限公司 数据存储方法、装置、移动终端及存储介质
CN110908608A (zh) * 2019-11-22 2020-03-24 苏州浪潮智能科技有限公司 一种存储空间节省方法及系统

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114741205A (zh) * 2022-06-02 2022-07-12 荣耀终端有限公司 一种匿名页回收方法及电子设备
CN114741205B (zh) * 2022-06-02 2022-11-11 荣耀终端有限公司 一种匿名页回收方法及电子设备
CN115905168A (zh) * 2022-11-15 2023-04-04 本原数据(北京)信息技术有限公司 自适应压缩方法和压缩装置、计算机设备、存储介质
CN115905168B (zh) * 2022-11-15 2023-11-07 本原数据(北京)信息技术有限公司 基于数据库的自适应压缩方法和装置、设备、存储介质

Also Published As

Publication number Publication date
CN115398406A (zh) 2022-11-25

Similar Documents

Publication Publication Date Title
CN106649670B (zh) 基于流式计算的数据监控方法及装置
WO2021057665A1 (zh) 数据存储方法、装置、移动终端及存储介质
US10649905B2 (en) Method and apparatus for storing data
EP3376393B1 (en) Data storage method and apparatus
US9088540B1 (en) Processing data formatted for efficient communication over a network
WO2021243531A1 (zh) 数据压缩方法、装置、电子设备及存储介质
EP3958120A1 (en) Method and device for anonymous page management, terminal device, and readable storage medium
US11025271B2 (en) Compression of high dynamic ratio fields for machine learning
WO2017113124A1 (zh) 一种服务器以及服务器压缩数据的方法
CN104125458A (zh) 内存数据无损压缩方法及装置
CN103150260A (zh) 重复数据删除方法和装置
WO2020233046A1 (zh) 缓存文件的管理方法、装置、计算机设备及存储介质
US20190370009A1 (en) Intelligent swap for fatigable storage mediums
CN111083933B (zh) 数据存储及获取方法和装置
TWI634421B (zh) 用以存取資料之電子裝置及其資料存取方法
CN111274245B (zh) 一种用于优化数据存储的方法和装置
CN107423425B (zh) 一种对k/v格式的数据快速存储和查询方法
CN110399371B (zh) 基于Redis数据库的减少内存消耗的方法、存储介质及设备
US20240061599A1 (en) Method and system for processing file read-write service, device, and medium
WO2021082926A1 (zh) 一种数据压缩的方法及装置
JP2022527743A (ja) チャンキング方法および装置
CN113297267A (zh) 数据缓存和任务处理方法、装置、设备以及存储介质
CN114065704A (zh) 数据压缩方法、电子设备和计算机程序产品
CN107783990B (zh) 一种数据压缩方法及终端
WO2015058628A1 (zh) 文件的访问方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20939422

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 24.04.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20939422

Country of ref document: EP

Kind code of ref document: A1