CN106445412A - Data volume compression ratio assessment method and system - Google Patents

Data volume compression ratio assessment method and system Download PDF

Info

Publication number
CN106445412A
CN106445412A CN201610824302.XA CN201610824302A CN106445412A CN 106445412 A CN106445412 A CN 106445412A CN 201610824302 A CN201610824302 A CN 201610824302A CN 106445412 A CN106445412 A CN 106445412A
Authority
CN
China
Prior art keywords
sampling
data
target volume
data block
compression
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201610824302.XA
Other languages
Chinese (zh)
Inventor
刘明甲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhengzhou Yunhai Information Technology Co Ltd
Original Assignee
Zhengzhou Yunhai Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhengzhou Yunhai Information Technology Co Ltd filed Critical Zhengzhou Yunhai Information Technology Co Ltd
Priority to CN201610824302.XA priority Critical patent/CN106445412A/en
Publication of CN106445412A publication Critical patent/CN106445412A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608Saving storage space on storage systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • G06F3/064Management of blocks

Abstract

The invention discloses a data volume compression ratio assessment method and system. The method comprises the steps that n data sampling threads are preset, wherein n is an integer no less than 2; a target volume is divided into n different target volume zones, wherein the data sampling threads correspond to the target volume zones in a one-to-one mode; random sampling is conducted on the corresponding target volume zones through the data sampling threads; compression calculation is conducted on sampled data blocks through a preset compression algorithm; corresponding compression calculation results of the sampled data blocks are summarized, and a compression ratio assessment result of the target volume is calculated. Before data volume compression, a statistical method is adopted to conduct random sampling analysis and assessment on the compression ratio of the target volume, research is simple, the cost is saved, the assessment accuracy and reliability are ensured, occupation of system resources is decreased, space saving after target volume compression can be accessed without lengthy data compression process, and data support is provided for decision making for compression of a user.

Description

A kind of appraisal procedure of book compression ratio and system
Technical field
The present invention relates to memory system technologies field, more particularly to a kind of appraisal procedure of book compression ratio and it is System.
Background technology
With the development of science and technology, people have come into the information age.In the information age, the most key technology is Process for data.
Often using storage device, data to be stored during to data processing.But during with big data The arrival in generation, data volume is increasing, and this requires to improve the service efficiency of storage system, and at present, common storage system is equal The function of data compression is provided with, and memory space phase is taken will take the larger data block boil down to of memory space in storage To less data block.But, data compression is often time-consuming longer, and occupancy cpu resource is higher, for different types of data For, the difference of compression efficiency is also very big.If being compressed for the relatively low data block of some compression efficiencies, which is saved Memory space is less, but but takes a substantial amount of time and cpu resource, often loses more than gain.
Thus, the space cost of saving that can bring after how knowing data block compression, to judge that the data block is No be worth carrying out data compression, be the current technical issues that need to address of those skilled in the art.
Content of the invention
It is an object of the invention to provide a kind of appraisal procedure of book compression ratio and system, can carried out to data volume Before compression, the space cost of the saving that can bring after knowing data block compression, to judge whether the data block is worth carrying out Data compression, provides the user decision-making foundation.
For solving above-mentioned technical problem, the invention provides following technical scheme:
A kind of appraisal procedure of book compression ratio, including:
N sampling of data thread is pre-set, n is the integer not less than 2;
Target volume is divided into n different target volume region, each sampling of data thread and each target volume area Domain corresponds;
Stochastic sampling is carried out to corresponding target volume region by each sampling of data thread;
By presetting compression algorithm, the data block of each sampling is compressed calculating;
Result of calculation being compressed accordingly to the data block of each sampling to collect, calculates the compression ratio assessment of the target volume As a result.
Preferably, described stochastic sampling is carried out to corresponding target volume region by each sampling of data thread, including:
Predefine the size information of the data block of each sample;
By each described sampling thread to corresponding target volume region according to determined by each data block size information Data block is read in circulation.
Preferably, described by preset compression algorithm to the data block of each sampling be compressed calculate, be:
By VIFO compression algorithm, the data block of each sampling is compressed calculating.
Preferably, the data block to each sampling is compressed result of calculation accordingly and is collected, including:
Whether judgment sampling total amount reaches default sampling total amount lower threshold;
If it is not, then preserving the input data size of the VIFO compression algorithm, output data size and pressure when sampling every time Shrinkage;
If so, then the data of all preservations are collected.
Preferably, also include:
Judge whether each sampling of data thread exceedes default reading to the I O read frequency of failure in corresponding target volume region Take frequency of failure threshold value;
If so, then stop each sampling of data thread and action is sampled to corresponding target volume region.
Preferably, also include:
The empty number of data blocks of sample when judging that each sampling of data thread carries out stochastic sampling to the target volume Whether default sky number of data blocks upper limit threshold is reached;
If so, then stop each sampling of data thread and action is sampled to corresponding target volume region.
A kind of assessment system of book compression ratio, including:Sampling of data threading models, region division module and compression ratio Evaluation module;
Wherein, the sampling of data threading models include n sampling of data thread units, and n is the integer not less than 2, institute Region division module is stated for target volume being divided into n different target volume according to the number of the sampling of data thread units Region, and each sampling of data thread units and each target volume region one-to-one corresponding;
The sampling of data thread units are used for carrying out stochastic sampling to corresponding target volume region, and according to default compression Algorithm is compressed calculating to the data block of each sampling;
The compression ratio evaluation module is used for collecting the data block of each sampling and compresses result of calculation accordingly, and calculates described The compression ratio assessment result of target volume.
Preferably, the sampling of data thread units include:
Sample data block size sets subelement, for determining the size information of the data block of each sample;
Sampling function subelement, reads in data block for circulating according to the size information of each data block, is taken out at random Sample, and according to default compression algorithm, the data block of each sampling is compressed calculating.
Preferably, also include:
Whether the first judge module, reach default sampling total amount lower threshold for judgment sampling total amount, and judging When sampling total amount reaches default sampling total amount lower threshold, control the compression ratio evaluation module to execute and collect function action.
Preferably, also include:
Second judge module, for judging the I O read frequency of failure of the sampling of data threading models to the target volume Whether exceed default reading frequency of failure threshold value, and when judged result is for being, control the sampling of data threading models to stop Sampling action;
3rd judge module, for when judging that the sampling of data threading models carry out stochastic sampling to the target volume Whether the empty number of data blocks of sample reaches default sky number of data blocks upper limit threshold, and when result of determination is for being, controls institute State sampling of data threading models and stop sampling action.
Compared with prior art, technique scheme has advantages below:
The appraisal procedure of the book compression ratio provided by the embodiment of the present invention, including:Pre-set n sampling of data line Journey, n is the integer not less than 2;Target volume is divided into n different target volume region, each sampling of data thread and each target Volume region one-to-one corresponding;Stochastic sampling is carried out to corresponding target volume region by each sampling of data thread;By default compression Algorithm is compressed calculating to the data block of each sampling;Compress result of calculation accordingly to the data block of each sampling to collect, Calculate the compression ratio assessment result of target volume.Before compressing to data volume, by above-mentioned technical scheme to book Compression ratio be estimated, stochastic sampling analysis is carried out to the compression ratio of target volume using mathematical method and statistical method, adjust Look into simply cost-effective, it is ensured that the accurate reliability of assessment, the occupancy to system resource is reduced, without the need for very long data Compression process can be estimated to the space-saving after target volume compression, is that the decision-making whether user is compressed provides number According to support.
Description of the drawings
In order to be illustrated more clearly that the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing Accompanying drawing to be used needed for technology description is had to be briefly described, it should be apparent that, drawings in the following description are the present invention Some embodiments, for those of ordinary skill in the art, on the premise of not paying creative work, can also basis These accompanying drawings obtain other accompanying drawings.
The appraisal procedure flow chart of the book compression ratio that Fig. 1 is provided by a kind of specific embodiment of the present invention;
The assessment system structural representation of the book compression ratio that Fig. 2 is provided by a kind of specific embodiment of the present invention.
Specific embodiment
The core of the present invention is to provide a kind of appraisal procedure of book compression ratio and system, and data volume can carried out Before compression, the space cost of the saving that can bring after knowing data block compression, to judge whether the data block is worth carrying out Data compression, provides the user decision-making foundation.
In order that the above objects, features and advantages of the present invention can become apparent from understandable, below in conjunction with the accompanying drawings to this Bright specific embodiment is described in detail.
Elaborate detail in order to fully understand the present invention in the following description.But the present invention can with multiple not It is same as alternate manner described here to implement, those skilled in the art can do class without prejudice to intension of the present invention in the case of Like popularization.Therefore the present invention is not limited by following public specific embodiment.
Refer to the appraisal procedure stream of the book compression ratio that Fig. 1, Fig. 1 are provided by a kind of specific embodiment of the present invention Cheng Tu.
A kind of specific embodiment of the present invention provides a kind of appraisal procedure of book compression ratio, including:
S11:N sampling of data thread is pre-set, n is the integer not less than 2;
S12:Target volume is divided into n different target volume region, each sampling of data thread and each target volume region one One corresponds to;
S13:Stochastic sampling is carried out to corresponding target volume region by each sampling of data thread;
S14:By presetting compression algorithm, the data block of each sampling is compressed calculating;
S15:Result of calculation being compressed accordingly to the data block of each sampling to collect, calculates the compression ratio assessment of target volume As a result.
In the present embodiment, in order to improve the assessment algorithm efficiency of appraisal procedure and the accuracy of stochastic sampling, adopt Multi-threaded architecture, that is, preset n sampling of data thread, be divided into different regions to target volume by Thread Count, each line Cheng Duli is responsible for a region carries out stochastic sampling.Wherein, each target volume region is equivalent in statistics chester sampling Individual group, therefore, it is convenient, cost-effective that the technical scheme in present embodiment just has investigation.By default compression Algorithm is compressed calculating to the data block that samples, such that it is able to calculate the compression ratio of sample.So to multiple sample calculation As a result the assessment result of target volume compression ratio is drawn by being collected.
Before compressing to data volume, by above-mentioned technical scheme, the compression ratio of book is estimated, adopts Stochastic sampling analysis is mathematically carried out with statistical method to the compression ratio of target volume, is investigated simply cost-effective, it is ensured that The accurate reliability of assessment, reduces the occupancy to system resource, can be to target without the need for very long data compression process Space-saving after crimped contracting is estimated, and is that the decision-making whether user is compressed is supported there is provided data, if evaluate working as Larger space cost can be saved after front target volume compression, then user can be compressed to the target volume, otherwise, it is not necessary to right The target volume is compressed, and can avoid the occupancy to cpu resource, time-consuming.
On the basis of above-mentioned embodiment, in one embodiment of the present invention, by each sampling of data thread to correspondence Target volume region carry out stochastic sampling, including:
Predefine the size information of the data block of each sample;
By each sampling thread to corresponding target volume region according to determined by each data block size information circulation Read in data block.
In the present embodiment, the data block per 64KB being preferably set as a sample, had so both ensured data block not Can be too big or too small, to improve the efficiency of sampling.
It should be noted that the data block of other sizes can also be adopted as a sample, present embodiment is to this simultaneously Do not limit, specifically depend on the circumstances.
Further, by presetting compression algorithm, the data block of each sampling is compressed calculating, is:Compressed by VIFO Algorithm is compressed calculating to the data block of each sampling.
In the present embodiment, using VIFO compression algorithm, i.e. Variable Input Fixed Output compression is calculated Method, the algorithm input buffer length is indefinite and output buffer length is fixed, and can compress more data as much as possible, until defeated Enter data buffer zone full for empty or data output buffer area.
Further, compress result of calculation accordingly to the data block of each sampling to collect, including:
Whether judgment sampling total amount reaches default sampling total amount lower threshold;
If it is not, then preserving the input data size of VIFO compression algorithm, output data size and compression ratio when sampling every time;
If so, then the data of all preservations are collected.
In the present embodiment, sampling total amount lower threshold is provided with, and the preferably sampling total amount lower threshold is 3400 Individual.The setting of sampling total amount lower threshold, needs to meet the demand of sampling statisticses sample size, can take into account algorithm again efficiently fast The basic demand of speed, this be due to the result more at most assessed of quantity that samples more accurate, and in order to ensure the error rate that assesses Less, or even the error rate is within 5%, then minimum sampling sum is if desired error 3400, certainly according to the actual requirements Rate is less, then set corresponding sampling total amount lower threshold.The sampling total amount lower threshold is not reaching in sampling total amount When, do not carry out collecting calculating, to ensure the accuracy of compression ratio assessment result.
In one embodiment of the invention, the method also includes:
Judge each sampling of data thread is lost to whether the I O read frequency of failure in corresponding target volume region exceedes default reading Lose frequency threshold value;
If so, then stop each sampling of data thread and action is sampled to corresponding target volume region.
When target volume is just by current business application, sampling of data thread is often caused to lose the I O read of target volume Lose, and if now proceed by force sampling assessment, the carrying out of current business can be affected, therefore, when each sampling of data thread pair When the I O read frequency of failure in corresponding target volume region exceedes default reading frequency of failure threshold value, terminate automatically subsampling operation.Excellent Selection of land, when reading IO frequency of failure ratio reaches 1% (acquiescence 1%, be 5% to the maximum), terminates operation automatically, and the ratio refers to It is the frequency of failure ratio that read IO number corresponding with sampling total amount lower threshold.
Further, the method also includes:
Whether the empty number of data blocks of sample when judging that each sampling of data thread carries out stochastic sampling to target volume reaches Default sky number of data blocks upper limit threshold;
If so, then stop each sampling of data thread and action is sampled to corresponding target volume region.
In sampling process, if the data block as sample is most of for empty data block, the such as sampling proportion of empty data block During more than 95%, terminate automatically operation.This be due to target volume be sampled calculate assessment result when, need ensure assessment As a result accuracy range, that is, ensure sampling error within the specific limits, if empty data block can excessively affect the standard of sampling resultses True property, if additionally, empty data block excessively illustrates that disk is more empty, free space is enough, even without being compressed and compress Assessment.Therefore, in the present embodiment, two indexs of accuracy rating of the result are combined according to assessment result, more can be increased Effect accurately evaluates the compression ratio of target volume.
Refer to the assessment system knot of the book compression ratio that Fig. 2, Fig. 2 are provided by a kind of specific embodiment of the present invention Structure schematic diagram.
Correspondingly, one embodiment of the present invention additionally provides a kind of assessment system of book compression ratio, including:Data Sampling threading models 1, region division module 2 and compression ratio evaluation module 3;Wherein, sampling of data threading models 1 include n number According to sampling thread units, n is the integer not less than 2, and region division module 2 is used for will according to the number of sampling of data thread units Target volume is divided into n different target volume region, and each sampling of data thread units and each target volume region correspond;Number It is used for carrying out stochastic sampling to corresponding target volume region according to sampling thread units, and according to default compression algorithm to each sampling Data block is compressed calculating;Compression ratio evaluation module 3 is used for collecting the data block of each sampling and compresses result of calculation accordingly, and Calculate the compression ratio assessment result of target volume.
In the present embodiment, compression ratio evaluation module and region division module can be integrated in sampling of data threading models In.Before compressing to data volume, by above-mentioned technical scheme, the compression ratio of book is estimated, using mathematics Method and statistical method carry out stochastic sampling analysis to the compression ratio of target volume, investigate simply cost-effective, it is ensured that assessment Accurate reliability, reduce the occupancy to system resource, without the need for very long data compression process can to target volume compress Space-saving afterwards is estimated, and is that the decision-making whether user is compressed is supported there is provided data, if evaluating current goal Larger space cost can be saved after crimped contracting, then user can be compressed to the target volume, otherwise, it is not necessary to the target Volume is compressed, and can avoid the occupancy to cpu resource, time-consuming.
In one embodiment of the invention, sampling of data thread units include:It is single that sample data block size sets son Unit, for determining the size information of the data block of each sample;Sampling function subelement, for the size according to each data block Message loop reads in data block, carries out stochastic sampling, and is compressed meter to the data block of each sampling according to default compression algorithm Calculate.
Further, the system also includes:Whether the first judge module, reach default sampling for judgment sampling total amount Total amount lower threshold, and when judgement sample total amount reaches default sampling total amount lower threshold, control compression ratio evaluation module Execution collects function action.
In the present embodiment, size and the sampling total amount lower threshold of the data block of each sample is provided with, is preferably set The fixed data block per 64KB had so both ensured that data block was not too large or too small, to improve the effect of sampling as a sample Rate.Preferably the sampling total amount lower threshold is 3400.The setting of sampling total amount lower threshold, needs can meet sampling system The demand of meter sample size, can take into account algorithm efficiently quickly basic demand again, and this is more at most assessed due to the quantity that samples As a result more accurate, and less in order to ensure the error rate that assesses, or even the error rate is within 5%, then minimum sampling is total For 3400, certainly according to the actual requirements, if desired error rate is less, then set corresponding sampling total amount lower threshold.? When sampling total amount is not reaching to the sampling total amount lower threshold, do not carry out collecting calculating, to ensure the essence of compression ratio assessment result True property.
In one embodiment of the invention, the system also includes:
Second judge module, for judging whether sampling of data threading models exceed to the I O read frequency of failure of target volume Default reading frequency of failure threshold value, and when judged result is for being, control data sampling threading models stop sampling action;
3rd judge module, for the sky of sample when judging that sampling of data threading models carry out stochastic sampling to target volume Whether number of data blocks reaches default sky number of data blocks upper limit threshold, and when result of determination is for being, control data sampling line Journey module stops sampling action.
When target volume is just by current business application, sampling of data thread is often caused to lose the I O read of target volume Lose, and if now proceed by force sampling assessment, the carrying out of current business can be affected, therefore, employ the second judge module, When each sampling of data thread exceedes default reading frequency of failure threshold value to the I O read frequency of failure in corresponding target volume region, from Dynamic termination subsampling operation.Preferably, when reading IO frequency of failure ratio reaches 1% (acquiescence 1%, be 5% to the maximum), automatically eventually Only operate, the ratio refers to the frequency of failure ratio that read IO number corresponding with sampling total amount lower threshold.In sampling process In, if the data block major part as sample is empty data block, when such as the sampling proportion of empty data block is more than 95%, automatic termination Operation.This be due to target volume be sampled calculate assessment result when, need ensure assessment result accuracy range, i.e., Ensure sampling error within the specific limits, if empty data block crosses the accuracy that can affect sampling resultses, if additionally, empty data block mistake Many explanation disks are more empty, and free space is enough, even without being compressed and compress assessment.Therefore, in present embodiment In, two indexs of accuracy rating of the result are combined according to assessment result, more efficiently can accurately evaluate target volume Compression ratio.
In sum, data can involved in by the appraisal procedure of book compression ratio provided by the present invention and system Before row compression, the space cost of the saving that can bring after knowing data block compression, using mathematical method and statistical method Stochastic sampling analysis is carried out to the compression ratio of target volume, is investigated simply cost-effective, it is ensured that the accurate reliability of assessment, reduce Occupancy to system resource, can be carried out to the space-saving after target volume compression without the need for very long data compression process pre- Estimate, be that the decision-making whether user is compressed is supported there is provided data, if can save relatively after evaluating the contracting of current goal crimped Big space cost, then be compressed to the target volume by user, otherwise, it is not necessary to which the target volume is compressed, can keep away Exempt from the occupancy to cpu resource, time-consuming.Additionally, two indexs of accuracy rating of the result are combined using assessment result, can Compression ratio with more efficiently accurate assessment target volume.
Above a kind of appraisal procedure of book compression ratio provided by the present invention and system are described in detail.Herein In apply specific case the principle of the present invention and embodiment be set forth, the explanation of above example is only intended to side The assistant solution present invention and its core concept.It should be pointed out that for those skilled in the art, without departing from this On the premise of inventive principle, some improvement can also being carried out to the present invention and being modified, these improve and modification also falls into the present invention In scope of the claims.

Claims (10)

1. a kind of appraisal procedure of book compression ratio, it is characterised in that include:
N sampling of data thread is pre-set, n is the integer not less than 2;
Target volume is divided into n different target volume region, each sampling of data thread and each target volume region one One corresponds to;
Stochastic sampling is carried out to corresponding target volume region by each sampling of data thread;
By presetting compression algorithm, the data block of each sampling is compressed calculating;
Result of calculation being compressed accordingly to the data block of each sampling to collect, calculates the compression ratio assessment knot of the target volume Really.
2. method according to claim 1, it is characterised in that described by each sampling of data thread to corresponding mesh Mark volume region carries out stochastic sampling, including:
Predefine the size information of the data block of each sample;
By each described sampling thread to corresponding target volume region according to determined by each data block size information circulation Read in data block.
3. method according to claim 2, it is characterised in that described by presetting data block of the compression algorithm to each sampling It is compressed calculating, is:
By VIFO compression algorithm, the data block of each sampling is compressed calculating.
4. method according to claim 3, it is characterised in that the data block to each sampling compresses calculating knot accordingly Fruit is collected, including:
Whether judgment sampling total amount reaches default sampling total amount lower threshold;
If it is not, then preserving the input data size of the VIFO compression algorithm, output data size and compression ratio when sampling every time;
If so, then the data of all preservations are collected.
5. the method according to any one of Claims 1-4, it is characterised in that also include:
Judge each sampling of data thread is lost to whether the I O read frequency of failure in corresponding target volume region exceedes default reading Lose frequency threshold value;
If so, then stop each sampling of data thread and action is sampled to corresponding target volume region.
6. method according to claim 5, it is characterised in that also include:
Whether the empty number of data blocks of sample when judging that each sampling of data thread carries out stochastic sampling to the target volume Reach default sky number of data blocks upper limit threshold;
If so, then stop each sampling of data thread and action is sampled to corresponding target volume region.
7. a kind of assessment system of book compression ratio, it is characterised in that include:Sampling of data threading models, region division mould Block and compression ratio evaluation module;
Wherein, the sampling of data threading models include n sampling of data thread units, and n is the integer not less than 2, the area Domain division module is used for, according to the number of the sampling of data thread units, target volume is divided into n different target volume area Domain, and each sampling of data thread units and each target volume region one-to-one corresponding;
The sampling of data thread units are used for carrying out stochastic sampling to corresponding target volume region, and according to default compression algorithm The data block of each sampling is compressed calculating;
The compression ratio evaluation module is used for collecting the data block of each sampling and compresses result of calculation accordingly, and calculates the target The compression ratio assessment result of volume.
8. system according to claim 7, it is characterised in that the sampling of data thread units include:
Sample data block size sets subelement, for determining the size information of the data block of each sample;
Sampling function subelement, reads in data block for circulating according to the size information of each data block, carries out stochastic sampling, and According to default compression algorithm, the data block of each sampling is compressed calculating.
9. system according to claim 8, it is characterised in that also include:
Whether the first judge module, reach default sampling total amount lower threshold for judgment sampling total amount, and in judgement sample When total amount reaches default sampling total amount lower threshold, control the compression ratio evaluation module to execute and collect function action.
10. system according to claim 9, it is characterised in that also include:
To the I O read frequency of failure of the target volume whether the second judge module, for judging the sampling of data threading models Exceed default reading frequency of failure threshold value, and when judged result is for being, control the sampling of data threading models to stop sampling Action;
3rd judge module, for judging the sample when sampling of data threading models carry out stochastic sampling to the target volume Empty number of data blocks whether reach default sky number of data blocks upper limit threshold, and when result of determination is for being, control the number Stop sampling action according to sampling threading models.
CN201610824302.XA 2016-09-14 2016-09-14 Data volume compression ratio assessment method and system Pending CN106445412A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610824302.XA CN106445412A (en) 2016-09-14 2016-09-14 Data volume compression ratio assessment method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610824302.XA CN106445412A (en) 2016-09-14 2016-09-14 Data volume compression ratio assessment method and system

Publications (1)

Publication Number Publication Date
CN106445412A true CN106445412A (en) 2017-02-22

Family

ID=58169022

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610824302.XA Pending CN106445412A (en) 2016-09-14 2016-09-14 Data volume compression ratio assessment method and system

Country Status (1)

Country Link
CN (1) CN106445412A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108984152A (en) * 2018-08-21 2018-12-11 北京睦合达信息技术股份有限公司 A kind of data processing method, system and computer readable storage medium
WO2022021865A1 (en) * 2020-07-29 2022-02-03 苏州浪潮智能科技有限公司 Data structure tree verification method, apparatus, and device, and storage medium
CN117786169A (en) * 2024-02-26 2024-03-29 北京数变科技有限公司 Data self-adaptive storage method and device, electronic equipment and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1127558A (en) * 1994-04-22 1996-07-24 索尼公司 Device and method for transmitting data, and device and method for recording data
US20130111164A1 (en) * 2011-10-31 2013-05-02 Jichuan Chang Hardware compression using common portions of data
CN104462334A (en) * 2014-12-03 2015-03-25 天津南大通用数据技术股份有限公司 Data compression method and device for packing database
CN105051724A (en) * 2013-08-19 2015-11-11 华为技术有限公司 Data object processing method and device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1127558A (en) * 1994-04-22 1996-07-24 索尼公司 Device and method for transmitting data, and device and method for recording data
US20130111164A1 (en) * 2011-10-31 2013-05-02 Jichuan Chang Hardware compression using common portions of data
CN105051724A (en) * 2013-08-19 2015-11-11 华为技术有限公司 Data object processing method and device
CN104462334A (en) * 2014-12-03 2015-03-25 天津南大通用数据技术股份有限公司 Data compression method and device for packing database

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108984152A (en) * 2018-08-21 2018-12-11 北京睦合达信息技术股份有限公司 A kind of data processing method, system and computer readable storage medium
CN108984152B (en) * 2018-08-21 2021-01-29 北京睦合达信息技术股份有限公司 Data processing method, system and computer readable storage medium
WO2022021865A1 (en) * 2020-07-29 2022-02-03 苏州浪潮智能科技有限公司 Data structure tree verification method, apparatus, and device, and storage medium
CN117786169A (en) * 2024-02-26 2024-03-29 北京数变科技有限公司 Data self-adaptive storage method and device, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
CN106445412A (en) Data volume compression ratio assessment method and system
CN108229811B (en) Method for evaluating water injection effect of fractured-vuggy carbonate reservoir
CN111723018A (en) Performance pressure testing method, device, equipment and storage medium
CN104602227A (en) Network-adaptive mobile application data loading method
CN111143143B (en) Performance test method and device
CN113608916B (en) Fault diagnosis method and device, electronic equipment and storage medium
CN111858284A (en) Resource monitoring method and device for artificial intelligence server
CN104217144A (en) Method and system for preventing virtual good brushing in online game
CN111181800B (en) Test data processing method and device, electronic equipment and storage medium
CN109447506A (en) A kind of power quality datum-plane evaluation method and system
CN106598738A (en) Computer cluster system and parallel computing method thereof
CN109375146B (en) Supplementary collection method and system for electricity consumption data and terminal equipment
CN107370783B (en) Scheduling method and device for cloud computing cluster resources
CN112596985B (en) IT asset detection method, device, equipment and medium
CN112749013B (en) Thread load detection method and device, electronic equipment and storage medium
CN106021046A (en) Hard disk performance comparison method, device and server
CN111683296B (en) Video segmentation method and device, electronic equipment and storage medium
CN116450483A (en) Method, device, server and medium for determining load of software distribution
CN111597096B (en) Benchmark test method, system and terminal equipment
CN106201860A (en) The method of testing of application system and device
CN113885803A (en) Data storage method and device, electronic equipment and storage medium
CN104237660A (en) Automatic testing device and method
CN112367384A (en) Kafka cluster-based dynamic speed limiting method and device and computer equipment
CN113238911A (en) Alarm processing method and device
CN107302501B (en) Method and device for adjusting network port aggregation

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20170222

RJ01 Rejection of invention patent application after publication