CN113965206A - Compression method, compression device, electronic equipment and storage medium - Google Patents

Compression method, compression device, electronic equipment and storage medium Download PDF

Info

Publication number
CN113965206A
CN113965206A CN202111235097.0A CN202111235097A CN113965206A CN 113965206 A CN113965206 A CN 113965206A CN 202111235097 A CN202111235097 A CN 202111235097A CN 113965206 A CN113965206 A CN 113965206A
Authority
CN
China
Prior art keywords
data volume
compressed
compression
target object
threshold
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111235097.0A
Other languages
Chinese (zh)
Inventor
李云飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN202111235097.0A priority Critical patent/CN113965206A/en
Publication of CN113965206A publication Critical patent/CN113965206A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H03ELECTRONIC CIRCUITRY
    • H03MCODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction

Abstract

The present disclosure provides a compression method, an apparatus, an electronic device and a storage medium, which relate to the technical field of data processing, in particular to the technical field of data compression, and are applied to scenes such as text, picture, audio and video compression. The implementation scheme is as follows: in response to a compression trigger event for an object to be compressed, calculating an original data volume of the object to be compressed; comparing the original data volume of the object to be compressed with a data volume threshold, and compressing the object to be compressed into a target object when the original data volume of the object to be compressed is determined to be larger than the data volume threshold; calculating the compression ratio of the target object; upon determining that the compression ratio of the target object is not greater than the compression ratio threshold, retaining the target object. The method can avoid executing excessive times of compression operation with lower compression yield than expected compression operation, and can effectively save computing resources; and the target object is reserved only when the compression ratio of the target object is not greater than the compression ratio threshold value, so that the waste of storage resources is avoided to a greater extent.

Description

Compression method, compression device, electronic equipment and storage medium
Technical Field
The present disclosure relates to the field of data processing, and more particularly to the field of data compression, which can be applied to text, picture, audio, video compression, and other scenes.
Background
Data compression is a technical method for reducing the data volume to reduce the storage space and improve the transmission, storage and processing efficiency of the data without losing information. However, in some scenarios of compressing data, the data amount of the compressed data may increase, which may cause waste of storage resources and computing resources.
Disclosure of Invention
The disclosure provides a compression method, a compression device, an electronic device and a storage medium.
According to a first aspect of the present disclosure, there is provided a compression method comprising:
in response to a compression trigger event for an object to be compressed, calculating an original data volume of the object to be compressed;
comparing the original data volume of the object to be compressed with a data volume threshold, and compressing the object to be compressed into a target object when the original data volume of the object to be compressed is determined to be larger than the data volume threshold;
calculating the compression ratio of the target object;
upon determining that the compression ratio of the target object is not greater than the compression ratio threshold, retaining the target object.
According to a second aspect of the present disclosure, there is provided a compression apparatus comprising:
the data volume calculation module is used for responding to a compression trigger event aiming at the object to be compressed and calculating the original data volume of the object to be compressed;
the data compression module is used for comparing the original data volume of the object to be compressed with a data volume threshold value, and compressing the object to be compressed into a target object when the original data volume of the object to be compressed is determined to be larger than the data volume threshold value;
the compression ratio calculation module is used for calculating the compression ratio of the target object;
and the compression result evaluation module is used for reserving the target object when the compression ratio of the target object is determined to be not more than the compression ratio threshold value.
According to a third aspect of the present disclosure, there is provided an electronic device comprising:
at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the compression method described above.
According to a fourth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to execute the above-described compression method.
According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the compression method described above.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
The technical scheme provided by the disclosure has the following beneficial effects:
in the technical scheme of the disclosure, the compression operation is executed on the object to be compressed only when the original data volume of the object to be compressed is determined to be larger than the data volume threshold value, so that the increase of the compressed data volume caused by the undersize of the data volume of the object to be compressed can be avoided to a greater extent, the compression yield of the object to be compressed is avoided from being inferior to the expected compression operation for too many times, and the calculation resources can be effectively saved; and after the compressed target object is obtained, the target object is reserved only when the compression ratio of the target object is not greater than the compression ratio threshold value, so that the reserved target object is ensured to accord with the expected compression benefit, the storage resource is reasonably and efficiently utilized, and the waste of the storage resource is avoided to a great extent.
Drawings
The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
fig. 1 is a schematic flow chart illustrating a compression method provided by an embodiment of the present disclosure;
FIG. 2 is a flow chart illustrating another compression method provided by embodiments of the present disclosure;
FIG. 3 illustrates one of the schematic diagrams of a compression apparatus provided by embodiments of the present disclosure;
fig. 4 shows a second schematic diagram of a compression apparatus provided by the embodiment of the present disclosure;
fig. 5 shows a schematic block diagram of an example electronic device that may be used to implement the compression methods provided by embodiments of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Data compression is a technical method for reducing the data volume to reduce the storage space and improve the transmission, storage and processing efficiency of the data without losing information. However, in some scenarios of compressing data, the data amount of the compressed data may increase, which may cause waste of storage resources and computing resources. Specifically, compressing data may generate some additional dictionary data, and for original data with a small data volume, after compressing the original data, the additional dictionary data may cause the data after compression to have a larger data volume than the original data, and the compression benefit is not as good as expected; moreover, data compression is a relatively expensive algorithm, and the compression yield is not as good as the expected compression operation, which is a waste of computing resources.
The compression method, apparatus, electronic device and storage medium provided by the embodiments of the present disclosure aim to solve at least one of the above technical problems in the prior art.
Fig. 1 shows a schematic flow chart of a compression method provided by an embodiment of the present disclosure, and as shown in fig. 1, the method mainly includes the following steps:
s110: in response to a compression trigger event for an object to be compressed, an amount of raw data of the object to be compressed is calculated.
The embodiment of the present disclosure refers to data that is desired to be compressed in a current scene as an object to be compressed, and the object to be compressed may be any type of data, such as text data, picture data, audio data, video data, and the like. The compression trigger event may be determined based on actual conditions, for example, the compression trigger event may be a user trigger operation for compressing the object to be compressed, or a related event executed by the system based on a preset determination condition (e.g., a determination condition related to time).
S120: and comparing the original data volume of the object to be compressed with the data volume threshold, and compressing the object to be compressed into the target object when the original data volume of the object to be compressed is determined to be larger than the data volume threshold.
For convenience of understanding and expression, the data size of the object to be compressed is defined as the original data size, and the data obtained after the object to be compressed is defined as the target object in the embodiment of the disclosure.
The data volume threshold of the present disclosure may be determined based on actual design requirements, and it can be understood that, when the original data volume of the object to be compressed is greater than the data volume threshold, it indicates that the object to be compressed is data with a large data volume, and after the object to be compressed is compressed, the obtained data volume of the target object is larger and may be smaller than the original data volume of the object to be compressed.
Optionally, when it is determined that the original data amount of the object to be compressed is not greater than the data amount threshold, the embodiment of the present disclosure may end the compression process for the object to be compressed. It can be understood that, when the original data amount of the object to be compressed is smaller than the data amount threshold, it indicates that the object to be compressed is data with a smaller data amount, and after the object to be compressed is compressed, the obtained data amount of the target object is smaller than the original data amount of the object to be compressed, and the compression yield is not as expected, so that the object to be compressed does not need to be compressed to save the computing resources.
According to the embodiment of the disclosure, the object to be compressed can be compressed into the target object by using the corresponding compression algorithm according to actual needs, the type of the compression algorithm is not limited by the embodiment of the disclosure, and the compression algorithm can be a zstd algorithm, a zlib algorithm, a gzip algorithm and the like.
S130: the compression ratio of the target object is calculated.
It should be noted here that the compression ratio of the target object is a ratio of the data amount of the target object to the original data amount of the object to be compressed. In this step, the data amount of the target object may be calculated, and then the target object data amount is divided by the original data amount of the object to be compressed to obtain the compression ratio.
When the data volume of the target object is larger than the original data volume of the object to be compressed, the compression ratio is larger than 1; when the data volume of the target object is less than the original data volume of the object to be compressed, the compression ratio is between 0 and 1; when the target object data amount is equal to the original data amount of the object to be compressed, the compression ratio is greater than 1. It is understood that when the compression ratio is between 0 and 1, the compression yield is indicated to be positive.
S140: upon determining that the compression ratio of the target object is not greater than the compression ratio threshold, retaining the target object.
The compression ratio threshold value can be determined according to actual design requirements, and it is understood that, in order to ensure that the compression yield of the reserved target object is positive, the compression ratio threshold value is usually not greater than 1. When the compression ratio of the target object is not larger than the compression ratio threshold value, the target object is indicated to achieve the expected compression yield.
Optionally, when it is determined that the compression ratio of the target object is greater than the compression ratio threshold, the embodiment of the present disclosure may delete the target object. It is to be noted here that the above-mentioned,
it can be understood that when the compression ratio of the target object is greater than the compression ratio threshold, the compression benefit of the target object is deemed to be less than expected, i.e., the target object is deleted to avoid occupying too much storage resources. For example, if the compression ratio threshold is set to 1, when the compression ratio threshold is not greater than 1, it indicates that the data size of the target object is greater than the original data size of the data to be compressed, and the compression yield of the target object is negative, so that the target object needs to be deleted, and the file to be compressed before compression is retained, thereby avoiding the waste of storage resources for the target object with the negative compression yield.
According to the compression method provided by the embodiment of the disclosure, when it is determined that the original data volume of the object to be compressed is larger than the data volume threshold value, the compression operation is performed on the object to be compressed, which can largely avoid the increase of the compressed data volume caused by the undersize data volume of the object to be compressed, avoid the excessive compression yield from being inferior to the expected compression operation, and effectively save the calculation resources; and after the compressed target object is obtained, the target object is reserved only when the compression ratio of the target object is not greater than the compression ratio threshold value, so that the reserved target object is ensured to accord with the expected compression benefit, the storage resource is reasonably and efficiently utilized, and the waste of the storage resource is avoided to a great extent.
The embodiment of the disclosure further updates and adjusts the data volume threshold, for example, the data volume threshold may be increased or decreased, so as to more accurately determine that the compression profit can meet the expected object to be compressed based on the data volume threshold, thereby further reducing the number of times that the compression profit is not shorter than the expected compression operation, and saving the computing resource.
Optionally, the embodiment of the present disclosure may increase the data amount threshold after deleting the target object.
Optionally, after comparing the original data volume of the object to be compressed with the data volume threshold, when it is determined that the original data volume of the object to be compressed is not greater than the data volume threshold, the embodiment of the present disclosure randomly compresses the object to be compressed into the target object with a preset probability; calculating the compression ratio of the target object; when it is determined that the compression ratio of the target object is greater than the compression ratio threshold, the data amount threshold is decreased.
Fig. 2 shows a schematic flow chart of a compression method provided by an embodiment of the present disclosure, and as shown in fig. 2, the method mainly includes the following steps:
s210: in response to a compression trigger event for an object to be compressed, an amount of raw data of the object to be compressed is calculated.
The embodiment of the present disclosure refers to data that is desired to be compressed in a current scene as an object to be compressed, and the object to be compressed may be any type of data, such as text data, picture data, audio data, video data, and the like. The compression trigger event may be determined based on actual conditions, for example, the compression trigger event may be a user trigger operation for compressing the object to be compressed, or a related event executed by the system based on a preset determination condition (e.g., a determination condition related to time).
S220: the original data volume of the object to be compressed is compared with a data volume threshold.
For convenience of understanding and expression, the data size of the object to be compressed is defined as the original data size, and the data obtained after the object to be compressed is defined as the target object in the embodiment of the disclosure. The data volume threshold of the present disclosure may be based on actual design requirements, and this step may compare the original data volume of the object to be compressed with the size of the data volume threshold.
S230: and compressing the object to be compressed into the target object when the original data volume of the object to be compressed is determined to be larger than the data volume threshold value.
It can be understood that, when the original data size of the object to be compressed is greater than the data size threshold, it indicates that the object to be compressed is data with a large data size, and after the object to be compressed is compressed, the obtained data size of the target object is larger and may be smaller than the original data size of the object to be compressed.
S240: and when the original data volume of the object to be compressed is determined not to be larger than the data volume threshold value, ending the compression process aiming at the object to be compressed.
Optionally, when it is determined that the original data amount of the object to be compressed is not greater than the data amount threshold, the embodiment of the present disclosure may end the compression process for the object to be compressed. It can be understood that, when the original data amount of the object to be compressed is smaller than the data amount threshold, it indicates that the object to be compressed is data with a smaller data amount, and after the object to be compressed is compressed, the obtained data amount of the target object is smaller than the original data amount of the object to be compressed, and the compression yield is not as expected, so that the object to be compressed does not need to be compressed to save the computing resources.
S250: the compression ratio of the target object is calculated.
It should be noted here that the compression ratio of the target object is a ratio of the data amount of the target object to the original data amount of the object to be compressed. In this step, the data amount of the target object may be calculated, and then the target object data amount is divided by the original data amount of the object to be compressed to obtain the compression ratio. After step S25, step S260 or step S270 may be performed.
When the data volume of the target object is larger than the original data volume of the object to be compressed, the compression ratio is larger than 1; when the data volume of the target object is less than the original data volume of the object to be compressed, the compression ratio is between 0 and 1; when the target object data amount is equal to the original data amount of the object to be compressed, the compression ratio is greater than 1. It is understood that when the compression ratio is between 0 and 1, the compression yield is indicated to be positive.
S260: upon determining that the compression ratio of the target object is not greater than the compression ratio threshold, retaining the target object.
In the disclosed embodiment, the compression ratio threshold may be based on actual design requirements. It will be appreciated that to ensure that the compression yield of the target object that remains is positive, the compression ratio threshold is typically no greater than 1. And when the compression ratio of the target object is not greater than the compression ratio threshold value, indicating that the compression yield of the target object reaches the expectation, reserving the target object, and ending the compression process for the object to be compressed.
S270: deleting the target object when it is determined that the compression ratio of the target object is greater than the compression ratio threshold.
It can be understood that when the compression ratio of the target object is greater than the compression ratio threshold, the compression benefit of the target object is deemed to be less than expected, i.e., the target object is deleted to avoid occupying too much storage resources. For example, if the compression ratio threshold is set to 1, when the compression ratio threshold is not greater than 1, it indicates that the data size of the target object is greater than the original data size of the data to be compressed, and the compression yield of the target object is negative, so that the target object needs to be deleted, and the file to be compressed before compression is retained, thereby avoiding the waste of storage resources for the target object with the negative compression yield.
It can be understood that when the compression ratio of the target object is greater than the compression ratio threshold, it indicates that the data amount threshold needs to be further increased, so as to more accurately determine that the compression yield can meet the expected object to be compressed based on the data amount threshold, thereby further reducing the number of times that the compression yield is less than the expected compression operation, and saving the computing resources.
S280: the data amount threshold is increased.
It can be understood that after the data volume threshold is increased, more data to be compressed can be made smaller than the data volume threshold, so that the number of times that the compression profit is less than the expected compression operation is reduced, and the computing resources are saved.
Alternatively, in a scenario of increasing the data amount threshold, the embodiment of the present disclosure may set the compression ratio threshold to 1, and since the compression ratio of the target object is greater than the compression ratio threshold, the data amount of the target object is greater than the original data amount of the object to be compressed, and therefore the original data amount of the object to be compressed may be used as a new data amount threshold, thereby increasing the data amount threshold.
Optionally, in a scenario of increasing the data amount threshold, the embodiment of the present disclosure adjusts the data amount threshold according to a preset increase ratio, and takes the adjusted data amount threshold as a new data amount threshold. Here, the increase rate may be determined according to actual design requirements, and for example, the increase rate may be 1%, and the original data amount threshold value is increased by 1% to be used as a new data amount threshold value.
In order to avoid that the data size threshold is updated to an unreasonably large value as a result of increasing the data size threshold several times, it is possible to reduce the data size threshold in a probabilistically decreasing manner. Optionally, after comparing the original data amount of the object to be compressed with the data amount threshold, when it is determined that the original data amount of the object to be compressed is not greater than the data amount threshold, the embodiment of the present disclosure randomly compresses the object to be compressed into the target object with a preset probability, and then calculates the compression ratio of the target object; when it is determined that the compression ratio of the target object is greater than the compression ratio threshold, the data amount threshold is decreased.
The value of the predetermined probability may be determined according to actual design requirements, for example, the predetermined probability may be one hundredth, one thousandth, one ten thousandth, or the like. Selecting one object to be compressed from one thousand objects to be compressed with the original data volume not larger than the data volume threshold value by using the preset probability as one thousandth, and taking the selected object to be compressed as a target object; and calculating the compression ratio of the target object, and reducing the data amount threshold when the compression ratio of the target object is determined to be larger than the compression ratio threshold.
Alternatively, in a scenario of reducing the data amount threshold, the embodiment of the present disclosure may set the compression ratio threshold to 1, and since the compression ratio of the target object is smaller than the compression ratio threshold, the data amount of the target object is smaller than the original data amount of the object to be compressed, and therefore the original data amount of the object to be compressed may be used as a new data amount threshold, thereby achieving reduction of the data amount threshold.
Optionally, in a scenario of reducing the data amount threshold, the embodiment of the present disclosure adjusts the data amount threshold according to a preset reduction ratio, and uses the adjusted data amount threshold as a new data amount threshold. Here, the reduction ratio may be determined according to actual design requirements, for example, the reduction ratio may be 1%, and the original data amount threshold value is reduced by 1% to be used as a new data amount threshold value.
It should be noted that, according to the embodiment of the present disclosure, the object to be compressed may be compressed into the target object by using a corresponding compression algorithm according to actual needs, the embodiment of the present disclosure does not limit the type of the compression algorithm, and the compression algorithm may be a zstd algorithm, a zlib algorithm, a gzip algorithm, or the like.
Based on the same principle as the compression method described above, fig. 3 shows one of the schematic diagrams of a compression apparatus provided by the embodiment of the present disclosure, and fig. 4 shows the second schematic diagram of a compression apparatus provided by the embodiment of the present disclosure. As shown in fig. 3, the compression apparatus 300 includes a data amount calculation module 310, a data compression module 320, a compression ratio calculation module 330, and a compression result evaluation module 340.
The data amount calculation module 310 is configured to calculate an original data amount of the object to be compressed in response to a compression trigger event for the object to be compressed.
The data compression module 320 is configured to compare the original data size of the object to be compressed with the data size threshold, and compress the object to be compressed into the target object when it is determined that the original data size of the object to be compressed is greater than the data size threshold.
The compression ratio calculation module 330 is used to calculate the compression ratio of the target object.
The compression result evaluation module 340 is used to retain the target object when it is determined that the compression ratio of the target object is not greater than the compression ratio threshold.
According to the compression device provided by the embodiment of the disclosure, when it is determined that the original data volume of the object to be compressed is larger than the data volume threshold value, the compression operation is performed on the object to be compressed, which can largely avoid the increase of the compressed data volume caused by the undersize data volume of the object to be compressed, avoid the excessive compression yield from being inferior to the expected compression operation, and effectively save the calculation resources; and after the compressed target object is obtained, the target object is reserved only when the compression ratio of the target object is not greater than the compression ratio threshold value, so that the reserved target object is ensured to accord with the expected compression benefit, the storage resource is reasonably and efficiently utilized, and the waste of the storage resource is avoided to a great extent.
In the embodiment of the present disclosure, the compression result evaluation module 340 is further configured to: deleting the target object when it is determined that the compression ratio of the target object is greater than the compression ratio threshold.
In the embodiment of the present disclosure, as shown in fig. 4, the compression apparatus further includes a threshold adjusting module 350, where the threshold adjusting module 350 is configured to: the data amount threshold is increased.
In the embodiment of the present disclosure, when the threshold adjustment module 350 is used to increase the data amount threshold, it is specifically configured to: taking the original data volume of the object to be compressed as a new data volume threshold value; or, adjusting the data volume threshold according to a preset increasing proportion, and taking the adjusted data volume threshold as a new data volume threshold.
In an embodiment of the present disclosure, the data compression module 320 is further configured to: and when the original data volume of the object to be compressed is determined not to be larger than the data volume threshold value, ending the compression process aiming at the object to be compressed.
In an embodiment of the present disclosure, the threshold adjustment module 350 is further configured to: when the original data volume of the object to be compressed is determined to be not larger than the data volume threshold value, randomly compressing the object to be compressed into a target object according to a preset probability; calculating the compression ratio of the target object; when it is determined that the compression ratio of the target object is greater than the compression ratio threshold, the data amount threshold is decreased.
In the embodiment of the present disclosure, when the threshold adjustment module 350 is used to reduce the data amount threshold, it is specifically configured to: taking the original data volume of the object to be compressed as a new data volume threshold value; or, the data amount threshold is adjusted according to a preset reduction ratio, and the adjusted data amount threshold is used as a new data amount threshold.
It is understood that the above modules of the compression apparatus in the embodiment of the present disclosure have functions of implementing the corresponding steps of the compression method described above. The function can be realized by hardware, and can also be realized by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the functions described above. The modules can be software and/or hardware, and each module can be implemented independently or by integrating a plurality of modules. For the functional description of each module of the compression apparatus, reference may be made to the corresponding description of the compression method, which is not described herein again.
In the technical scheme of the disclosure, the collection, storage, use, processing, transmission, provision, disclosure and other processing of the personal information of the related user are all in accordance with the regulations of related laws and regulations and do not violate the good customs of the public order.
The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.
FIG. 5 illustrates a schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 5, the apparatus 500 comprises a computing unit 501 which may perform various appropriate actions and processes in accordance with a computer program stored in a Read Only Memory (ROM)502 or a computer program loaded from a storage unit 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The calculation unit 501, the ROM 502, and the RAM 503 are connected to each other by a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
A number of components in the device 500 are connected to the I/O interface 505, including: an input unit 506 such as a keyboard, a mouse, or the like; an output unit 507 such as various types of displays, speakers, and the like; a storage unit 508, such as a magnetic disk, optical disk, or the like; and a communication unit 509 such as a network card, modem, wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.
The computing unit 501 may be a variety of general-purpose and/or special-purpose processing components having processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 501 executes the respective methods and processes described above, such as the compression method. For example, in some embodiments, the compression method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 500 via the ROM 502 and/or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the compression method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the compression method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel or sequentially or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims (17)

1. A method of compression, comprising:
in response to a compression trigger event for an object to be compressed, calculating an original data volume of the object to be compressed;
comparing the original data volume of the object to be compressed with a data volume threshold, and compressing the object to be compressed into a target object when the original data volume of the object to be compressed is determined to be larger than the data volume threshold;
calculating a compression ratio of the target object;
retaining the target object upon determining that the compression ratio of the target object is not greater than a compression ratio threshold.
2. The method of claim 1, further comprising, after the calculating a compression ratio of the target object: deleting the target object when it is determined that the compression ratio of the target object is greater than the compression ratio threshold.
3. The method of claim 2, further comprising, after said deleting the target object: increasing the data volume threshold.
4. The method of claim 3, wherein the increasing the data volume threshold comprises:
taking the original data volume of the object to be compressed as a new data volume threshold value;
or, the data volume threshold is adjusted according to a preset increasing proportion, and the adjusted data volume threshold is used as a new data volume threshold.
5. The method of claim 1, further comprising, after the comparing the original data volume of the object to be compressed to a data volume threshold:
and when the original data volume of the object to be compressed is determined not to be larger than the data volume threshold value, ending the compression process aiming at the object to be compressed.
6. The method of claim 1, further comprising, after the comparing the original data volume of the object to be compressed to a data volume threshold:
when the original data volume of the object to be compressed is determined to be not larger than the data volume threshold value, randomly compressing the object to be compressed into a target object with preset probability;
calculating a compression ratio of the target object;
decreasing the data amount threshold upon determining that the compression ratio of the target object is greater than the compression ratio threshold.
7. The method of claim 6, wherein the reducing the data volume threshold comprises:
taking the original data volume of the object to be compressed as a new data volume threshold value;
or, the data volume threshold is adjusted according to a preset reduction ratio, and the adjusted data volume threshold is used as a new data volume threshold.
8. A compression device, comprising:
the data volume calculation module is used for responding to a compression trigger event aiming at an object to be compressed and calculating the original data volume of the object to be compressed;
the data compression module is used for comparing the original data volume of the object to be compressed with a data volume threshold value, and compressing the object to be compressed into a target object when the original data volume of the object to be compressed is determined to be larger than the data volume threshold value;
the compression ratio calculation module is used for calculating the compression ratio of the target object;
and the compression result evaluation module is used for reserving the target object when the compression ratio of the target object is determined to be not larger than the compression ratio threshold value.
9. The apparatus of claim 8, the compression result evaluation module further to: deleting the target object when it is determined that the compression ratio of the target object is greater than the compression ratio threshold.
10. The apparatus of claim 9, further comprising a threshold adjustment module to: increasing the data volume threshold.
11. The apparatus according to claim 10, wherein the threshold adjustment module, when configured to increase the data volume threshold, is specifically configured to:
taking the original data volume of the object to be compressed as a new data volume threshold value;
or, the data volume threshold is adjusted according to a preset increasing proportion, and the adjusted data volume threshold is used as a new data volume threshold.
12. The apparatus of claim 8, the data compression module further to: and when the original data volume of the object to be compressed is determined not to be larger than the data volume threshold value, ending the compression process aiming at the object to be compressed.
13. The apparatus of claim 8, further comprising a threshold adjustment module to:
when the original data volume of the object to be compressed is determined to be not larger than the data volume threshold value, randomly compressing the object to be compressed into a target object with preset probability;
calculating a compression ratio of the target object;
decreasing the data amount threshold upon determining that the compression ratio of the target object is greater than the compression ratio threshold.
14. The apparatus of claim 13, wherein the threshold adjustment module, when configured to decrease the data volume threshold, is specifically configured to:
taking the original data volume of the object to be compressed as a new data volume threshold value;
or, the data volume threshold is adjusted according to a preset reduction ratio, and the adjusted data volume threshold is used as a new data volume threshold.
15. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
16. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-7.
17. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-7.
CN202111235097.0A 2021-10-22 2021-10-22 Compression method, compression device, electronic equipment and storage medium Pending CN113965206A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111235097.0A CN113965206A (en) 2021-10-22 2021-10-22 Compression method, compression device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111235097.0A CN113965206A (en) 2021-10-22 2021-10-22 Compression method, compression device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN113965206A true CN113965206A (en) 2022-01-21

Family

ID=79466407

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111235097.0A Pending CN113965206A (en) 2021-10-22 2021-10-22 Compression method, compression device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113965206A (en)

Similar Documents

Publication Publication Date Title
US20190236453A1 (en) Method and system for data transmission, and electronic device
CN112488060B (en) Target detection method, device, equipment and medium
EP4135325A1 (en) Method and apparatus for encoding video, electronic device, and storage medium
CN113360266B (en) Task processing method and device
CN113965206A (en) Compression method, compression device, electronic equipment and storage medium
CN114757214B (en) Selection method and related device for sample corpora for optimizing translation model
CN113377295B (en) Data storage and reading method, device and equipment for multi-producer single-consumer
CN115904240A (en) Data processing method and device, electronic equipment and storage medium
CN113742581B (en) Method and device for generating list, electronic equipment and readable storage medium
CN113556575A (en) Method, apparatus, device, medium and product for compressing data
CN114138358A (en) Application program starting optimization method, device, equipment and storage medium
CN114386577A (en) Method, apparatus, and storage medium for executing deep learning model
CN114219731A (en) Picture optimization method, device, equipment and storage medium
CN114021642A (en) Data processing method and device, electronic equipment and storage medium
CN114139605A (en) Distributed model training method, system, device and storage medium
CN115037937B (en) Image compression method, apparatus, device and medium
CN116719621B (en) Data write-back method, device, equipment and medium for mass tasks
CN117615137B (en) Video processing method, device, equipment and storage medium
CN116614379B (en) Bandwidth adjustment method and device for migration service and related equipment
CN115242626B (en) Cloud resource allocation prediction method, device, equipment, storage medium and program product
CN115086300B (en) Video file scheduling method and device
CN115657964A (en) Power data reading method and device, electronic equipment and storage medium
US20230267060A1 (en) Performance testing method and apparatus, and storage medium
CN115049051A (en) Model weight compression method and device, electronic equipment and storage medium
CN115842892A (en) Picture compression method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination