CN106919617A - A kind of compression and storage method and device - Google Patents
A kind of compression and storage method and device Download PDFInfo
- Publication number
- CN106919617A CN106919617A CN201511000457.3A CN201511000457A CN106919617A CN 106919617 A CN106919617 A CN 106919617A CN 201511000457 A CN201511000457 A CN 201511000457A CN 106919617 A CN106919617 A CN 106919617A
- Authority
- CN
- China
- Prior art keywords
- parameter
- compression
- parameters
- storage engines
- targeted
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000007906 compression Methods 0.000 title claims abstract description 384
- 230000006835 compression Effects 0.000 title claims abstract description 384
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 128
- 238000012545 processing Methods 0.000 claims description 21
- 241000208340 Araliaceae Species 0.000 claims description 12
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 claims description 12
- 235000003140 Panax quinquefolius Nutrition 0.000 claims description 12
- 235000008434 ginseng Nutrition 0.000 claims description 12
- 230000003247 decreasing effect Effects 0.000 claims description 8
- 230000008901 benefit Effects 0.000 abstract description 20
- 238000005516 engineering process Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 230000009471 action Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000003780 insertion Methods 0.000 description 2
- 230000037431 insertion Effects 0.000 description 2
- 238000005303 weighing Methods 0.000 description 2
- 241000270708 Testudinidae Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 210000000988 bone and bone Anatomy 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001035 drying Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007334 memory performance Effects 0.000 description 1
- 230000008685 targeting Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2246—Trees, e.g. B+trees
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Abstract
A kind of compression and storage method and device are the embodiment of the invention provides, method therein is specifically included：The compression parameters of data to be stored are obtained, and/or, obtain the device parameter of storage device corresponding with data to be stored；According to the compression parameters and/or device parameter, selection targeted compression scheme corresponding with the data to be stored；Storage is compressed to the data to be stored using the targeted compression scheme.The embodiment of the present invention can substantially play the advantage of the targeted compression scheme such that it is able to greatly improve the performances such as the handling capacity of database.Also, the embodiment of the present invention can select most to agree with the targeted compression algorithm of current CPU core number, such that it is able to while database performance is ensured, by increasing capacitance it is possible to increase the stability of compression storage, and can as much as possible save the resource of storage device in data-base cluster.
Description
Technical field
The present invention relates to field of computer technology, more particularly to a kind of compression and storage method and device.
Background technology
With the development of information technology, the particularly development of Internet technologies, the information content in each field
All it is in explosive increase trend, higher than 1012The mass data of byte emerges in an endless stream.In order to effectively manage
Reason mass data, is currently suggested compressed data storehouse technology, and compressed data storehouse technology can improve magnanimity
The storage efficiency of data, and the performances such as the handling capacity of database can be improved.
The zlib compression algorithms that existing scheme is generally provided using InnoDB inside carry out the pressure of mass data
Contracting storage；Wherein, InnoDB is most widely used storage engines in MySQL database, and zlib can be with
The function library of data compression is provided, it is possible thereby to about 25% -50% or so memory space is saved,
And IO (input and output, Input Output) consumption, and the handling capacity for lifting database can be reduced.
However, during using existing scheme, such problem is run into sometimes：In data volume
In the case that radix is larger, continue frequently insert (insertion) or update (renewal) operation when,
InnoDB use the characteristic of B-tree (B- trees) index itself be easily caused database performance it is notable under
Drop, for example, it may be possible to cause the throughput degradation of database.
The content of the invention
In view of the above problems, it is proposed that the present invention overcomes above mentioned problem or at least portion to provide one kind
A kind of compression and storage method and device for solving the above problems with dividing.
According to one aspect of the present invention, there is provided a kind of compression and storage method, including：
The compression parameters of data to be stored are obtained, and/or, obtain storage corresponding with data to be stored and set
Standby device parameter；
According to the compression parameters and/or device parameter, selection target corresponding with the data to be stored
Compression scheme；
Storage is compressed to the data to be stored using the targeted compression scheme.
Alternatively, the compression parameters include application scenarios parameter and/or database schema parameter, then institute
State according to the compression parameters and/or device parameter, selection target pressure corresponding with the data to be stored
The step of contracting scheme, further include：
Selected from least one storage engines of database and the application scenarios parameter and/or database
The corresponding target storage engines of configuration parameters；
Corresponding targeted compression is selected to calculate from least one compression algorithm of the target storage engines
Method, the target storage engines and its corresponding targeted compression algorithm constitute the targeted compression scheme.
Alternatively, select to join with the application scenarios at least one storage engines from database
The step of number corresponding target storage engines, further include：
The application scenarios parameter is the first application scenarios parameter, then select corresponding first storage engines
It is target storage engines；Or,
The application scenarios parameter is the second application scenarios parameter, then select corresponding second storage engines
It is target storage engines；
Wherein, the first application scenarios parameter includes：Transaction Processing scenario parameters and reading
At least one in intensive scenario parameters；
Wherein, the second application scenarios parameter includes：On-line analytical processing scenario parameters and batch
Amount loads and reads at least one in intensive scenario parameters.
Alternatively, selected at least one storage engines from database and the database schema
The step of parameter corresponding target storage engines, further include：
The database schema parameter is the first database schema parameter, then select corresponding 3rd storage
Engine is target storage engines；Or,
The database schema parameter is the second database schema parameter, then select corresponding 4th storage
Engine is target storage engines；
Wherein, the first database schema parameter includes：Client/server parameter；Or, described
Two database schema parameters include：Non- principal and subordinate's configuration parameters.
Alternatively, the compression parameters include compression index parameter, then described according to the compression parameters
And/or device parameter, the step of select the data to be stored corresponding targeted compression scheme, further
Including：
From at least one compression algorithm of target storage engines selection agree with the compression index parameter and/
Or the targeted compression algorithm of the device parameter, the target storage engines and its corresponding targeted compression
Algorithm constitutes the targeted compression scheme.
Alternatively, the device parameter includes at least one in following parameter：CPU parameters, disk
Parameter and memory parameters.
Alternatively, the compression index parameter includes at least one in following parameter：Compression time is joined
Number and compression ratio parameter.
Alternatively, the targeted compression calculation for agreeing with the compression index parameter and/or the device parameter
Method includes：
Agree with the first object compression algorithm of first order compression index parameter；Or
Agree with the second targeted compression algorithm of second level compression index parameter；Or
Agree with the 3rd targeted compression algorithm of third level compression index parameter；Wherein, the first order pressure
Contracting index parameter, second level compression index parameter and the corresponding compression index of third level compression index parameter
Successively decrease；Or
Agree with the first object compression algorithm of first order device parameter；Or
Agree with the second targeted compression algorithm of second level device parameter；Or
Agree with the 3rd targeted compression algorithm of third level device parameter；Wherein, the first order equipment ginseng
Number, second level device parameter and the corresponding equipment performance of third level device parameter are successively decreased.
According to another aspect of the present invention, there is provided one kind compression storage device, including：
Acquisition module, the compression parameters for obtaining data to be stored, and/or, obtain and number to be stored
According to the device parameter of corresponding storage device；
Selecting module, for according to the compression parameters and/or device parameter, selecting to be stored with described
The corresponding targeted compression scheme of data；And
Compression memory module, for being pressed the data to be stored using the targeted compression scheme
Contracting storage.
Alternatively, the compression parameters include application scenarios parameter and/or database schema parameter, then described
Selecting module, further includes：
First choice submodule, for selecting to be answered with described from least one storage engines of database
With scenario parameters and/or the corresponding target storage engines of database schema parameter；And
Second selection submodule, for being selected from least one compression algorithm of the target storage engines
Corresponding targeted compression algorithm is selected, the target storage engines and its corresponding targeted compression algorithm are constituted
The targeted compression scheme.
Alternatively, the first choice submodule, further includes：
First choice unit, for when the application scenarios parameter is the first application scenarios parameter, selecting
Corresponding first storage engines are selected for target storage engines；Or,
Second select unit, for when the application scenarios parameter is the second application scenarios parameter, selecting
Corresponding second storage engines are selected for target storage engines；
Wherein, the first application scenarios parameter includes：Transaction Processing scenario parameters and reading
At least one in intensive scenario parameters；
Wherein, the second application scenarios parameter includes：On-line analytical processing scenario parameters and batch
Amount loads and reads at least one in intensive scenario parameters.
Alternatively, the first choice submodule, further includes：
3rd select unit, for being the first database schema parameter in the database schema parameter
When, it is target storage engines to select corresponding 3rd storage engines；Or,
4th select unit, for being the second database schema parameter in the database schema parameter
When, it is target storage engines to select corresponding 4th storage engines；
Wherein, the first database schema parameter includes：Client/server parameter；Or, described
Two database schema parameters include：Non- principal and subordinate's configuration parameters.
Alternatively, the compression parameters include compression index parameter, then the selecting module, further bag
Include：
3rd selection submodule, for selecting contract from least one compression algorithm of target storage engines
The targeted compression algorithm of the compression index parameter and/or the device parameter is closed, the target storage is drawn
Hold up and its corresponding targeted compression algorithm constitutes the targeted compression scheme.
Alternatively, the device parameter includes at least one in following parameter：CPU parameters, disk ginseng
Number and memory parameters.
Alternatively, the compression index parameter includes at least one in following parameter：Compression time parameter
With compression ratio parameter.
Alternatively, the targeted compression algorithm for agreeing with the compression index parameter and/or the device parameter
Including：
Agree with the first object compression algorithm of first order compression index parameter；Or
Agree with the second targeted compression algorithm of second level compression index parameter；Or
Agree with the 3rd targeted compression algorithm of third level compression index parameter；Wherein, the first order pressure
Contracting index parameter, second level compression index parameter and the corresponding compression index of third level compression index parameter
Successively decrease；Or
Agree with the first object compression algorithm of first order device parameter；Or
Agree with the second targeted compression algorithm of second level device parameter；Or
Agree with the 3rd targeted compression algorithm of third level device parameter；Wherein, the first order equipment ginseng
Number, second level device parameter and the corresponding equipment performance of third level device parameter are successively decreased.
A kind of compression and storage method and device according to embodiments of the present invention, can be according to data to be compressed
Corresponding compression parameters and/or device parameter, selection targeted compression side corresponding with the data to be stored
Case；Because above-mentioned compression parameters can be used to represent the parameter related to the characteristic of data to be compressed, or
Person, above-mentioned compression parameters can also be used for representing the parameter related to the compression requirements of data to be compressed, or
Person, above-mentioned compression parameters can also be used for representing the parameter related to the storage environment of data to be compressed, therefore
The embodiment of the present invention can most be agreed with characteristic, the compression of data to be compressed according to compression parameters selection
Demand, the targeted compression scheme of storage environment, therefore, treated to described using the targeted compression scheme
Data storage is compressed storage, can substantially play the advantage of the targeted compression scheme, from
And the performances such as the handling capacity of database can be greatly improved.
Also, can be used to characterize the property of the corresponding storage device of data to be stored due to the said equipment parameter
Can, therefore the embodiment of the present invention can select most to agree with the targeted compression algorithm of current CPU core number, so that
Can be while database performance be ensured, by increasing capacitance it is possible to increase the stability of compression storage, and to the greatest extent can may be used
The resource of storage device in data-base cluster is saved on energy ground.
Described above is only the general introduction of technical solution of the present invention, of the invention in order to better understand
Technological means, and being practiced according to the content of specification, and in order to allow it is of the invention above-mentioned and
Other objects, features and advantages can become apparent, below especially exemplified by specific embodiment party of the invention
Formula.
Brief description of the drawings
By reading the detailed description of hereafter optional embodiment, various other advantage and benefit for
Those of ordinary skill in the art will be clear understanding.Accompanying drawing is only used for showing the mesh of optional embodiment
, and it is not considered as limitation of the present invention.And in whole accompanying drawing, with identical with reference to symbol
Number represent identical part.In the accompanying drawings：
The step of Fig. 1 shows a kind of compression and storage method according to an embodiment of the invention flow is illustrated
Figure；
The step of Fig. 2 shows a kind of compression and storage method according to an embodiment of the invention flow is illustrated
Figure；
The step of Fig. 3 shows a kind of compression and storage method according to an embodiment of the invention flow is illustrated
Figure；And
Fig. 4 shows a kind of structural representation for compressing storage device according to an embodiment of the invention.
Specific embodiment
The exemplary embodiment of the disclosure is more fully described below with reference to accompanying drawings.Although showing in accompanying drawing
The exemplary embodiment of the disclosure is shown, it being understood, however, that may be realized in various forms the disclosure
Without that should be limited by embodiments set forth here.Conversely, there is provided these embodiments are able to more
Thoroughly understand the disclosure, and can be by the complete technology for conveying to this area of the scope of the present disclosure
Personnel.
Reference picture 1, flows the step of show a kind of compression and storage method according to an embodiment of the invention
Journey schematic diagram, specifically may include steps of：
Step 101, the compression parameters for obtaining data to be stored, and/or, obtain and data pair to be stored
The device parameter of the storage device answered；
Step 102, according to the compression parameters and/or device parameter, selection and the data to be stored
Corresponding targeted compression scheme；
Step 103, storage is compressed to the data to be stored using the targeted compression scheme.
The embodiment of the present invention is applicable to the compression storage of mass data in database, wherein, the number
According to storehouse can including ORACLE (inscriptions on bones or tortoise shells), DB2, MySQL etc. type database, can
To understand, the embodiment of the present invention is not any limitation as the particular type of database.For the ease of saying
Bright, the embodiment of the present invention is described by taking MySQL as an example, and other types of database is cross-referenced
.
In the embodiment of the present invention, above-mentioned compression parameters can be used to represent related to the characteristic of data to be compressed
Parameter, such as be used for weigh data to be compressed data volume size parameter；Or, above-mentioned compression ginseng
Number can also be used for representing the parameter related to the compression requirements of data to be compressed, for example, being treated for measurement
The application scenarios parameter of the application scenarios of compressed data, and for example, the compression for weighing data to be compressed
Compression index parameter of index etc.；Or, above-mentioned compression parameters can also be used for representing and data to be compressed
The related parameter of storage environment, the database frame of the database environment for being such as used to weighing data to be compressed
Structure parameter etc..Because the embodiment of the present invention can most be agreed with above-mentioned compression according to compression parameters selection
The targeted compression scheme of parameter, therefore, the data to be stored are entered using the targeted compression scheme
Row compression storage, can substantially play the advantage of the targeted compression scheme such that it is able to significantly
Improve the performances such as the handling capacity of database.
In the embodiment of the present invention, above-mentioned targeted compression scheme can specifically include：Target storage engines and
The corresponding targeted compression algorithm of the targeting engine.Wherein, above-mentioned target storage engines can be user
The storage engines specified, or the storage engines for selecting to obtain by the embodiment of the present invention.Its
In, above-mentioned storage engines can be used to being responsible for the storage of data in database, the data for storage and set up rope
Draw, a series of function system of affairs such as the inquiry of data etc. updates.
In a kind of alternative embodiment of the invention, the storage engines of MySQL database can specifically be wrapped
Include：ISAM, MyISAM, HEAP, InnoDB and TokuDB etc., then the embodiment of the present invention can
Most to be agreed with the target storage engines of above-mentioned compression parameters according to compression parameters selection.
In a kind of application example 1 of the invention, relative to existing scheme in the larger feelings of data volume radix
Under condition, continue frequently insert or update operations, caused database performance and be remarkably decreased
Problem, the embodiment of the present invention can obtain corresponding big number according to the larger situation of data volume radix
According to amount parameter or on-line analytical processing scenario parameters, or, can be larger according to data volume radix
Situation and the continuation characteristic that frequently insert or update is operated, obtain corresponding batch and load and read
Intensive scenario parameters, and selection agrees with above-mentioned big data quantity parameter or on-line analytical processing scenario parameters
Or batch is loaded and reads the TokuDB of intensive scenario parameters as target storage engines；Due to
The index structure of TokuDB is Fractal Tree, and Fractal Tree have insert relative to B-tree
Performance is stronger, be not likely to produce the advantage of " fragment ", thus TokuDB have compression ratio it is higher, batch fill
The more preferable advantage of performance is carried, therefore, the TokuDB that embodiment of the present invention selection is obtained can be answered above-mentioned
Storage is effectively compressed with the corresponding data to be compressed of example 1, such that it is able to improve number as much as possible
The performances such as the handling capacity according to storehouse.
In the embodiment of the present invention, the said equipment parameter can be used to characterizing the corresponding storage of data to be stored and set
Standby performance.In a kind of alternative embodiment of the invention, the device parameter can specifically be included such as
At least one in lower parameter：CPU parameters, disk parameter and memory parameters.
In a kind of alternative embodiment of the invention, the CPU consumption produced by different compression algorithms is not
With, therefore corresponding target CPU parameters can be obtained for each compression algorithm, and according to wait to deposit
The current CPU parameters of the corresponding storage device of storage data select corresponding compression algorithm.For example, storage
Current CPU core (core) number of equipment is 6 cores, and first object compression algorithm requirement CPU core number
More than 8 cores, the second targeted compression algorithm requirement CPU core number between the core of 4 core -8, the 3rd target
Compression algorithm requirement CPU core number can select most to agree with current CPU core number below 4 cores
Targeted compression algorithm, sets such that it is able to while database performance is ensured, save storage as much as possible
Standby resource.
It should be noted that the compression Stored Procedure of the embodiment of the present invention can be held by compression storage device
OK.Then in a kind of alternative embodiment of the invention, those skilled in the art can be according to number to be compressed
According to the factor parameter determination data to be compressed such as characteristic, compression requirements, storage environment compression parameters,
And the compression parameters for determining are input into above-mentioned compression storage device by the first preset interface.Or, this
Art personnel can first determine the corresponding storage device of data to be stored, then obtain storage and set
Standby device parameter, and the equipment for obtaining is input into above-mentioned compression storage device by the second preset interface
Parameter.
In another alternative embodiment of the invention, above-mentioned compression storage device can also be obtained first
The factor parameter such as characteristic, compression requirements, storage environment of data to be compressed, then according to above-mentioned factor
The compression parameters of parameter determination data to be compressed, wherein, above-mentioned factor parameter preset can connect by the 3rd
Mouth is collected, and can also be obtained by accessing database by above-mentioned compression storage device, and the present invention is implemented
Example is not any limitation as the specific acquisition modes of above-mentioned factor parameter.Or, above-mentioned compression storage dress
Putting can also obtain the equipment ginseng of storage device by accessing the corresponding storage device of data to be stored
Number.It is appreciated that the embodiment of the present invention specifically obtaining for above-mentioned compression parameters and the said equipment parameter
Process is taken not to be any limitation as.
To sum up, the embodiment of the present invention is according to the corresponding compression parameters of data to be compressed and/or device parameter,
Selection targeted compression scheme corresponding with the data to be stored；Because above-mentioned compression parameters can be used for table
Show the parameter related to the characteristic of data to be compressed, or, above-mentioned compression parameters can also be used for represent with
The related parameter of the compression requirements of data to be compressed, or, above-mentioned compression parameters can also be used for representing with
The related parameter of the storage environment of data to be compressed, therefore the embodiment of the present invention can be selected according to compression parameters
Characteristic, compression requirements, the targeted compression scheme of storage environment for most being agreed with data to be compressed are selected,
Therefore, storage is compressed to the data to be stored using the targeted compression scheme, can be maximum
Change the advantage that ground plays the targeted compression scheme such that it is able to greatly improve handling capacity of database etc.
Performance.
Also, can be used to characterize the property of the corresponding storage device of data to be stored due to the said equipment parameter
Can, therefore the embodiment of the present invention can select most to agree with the targeted compression algorithm of current CPU core number, so that
Can be while database performance be ensured, by increasing capacitance it is possible to increase the stability of compression storage, and to the greatest extent can may be used
The resource of storage device in data-base cluster is saved on energy ground.
Reference picture 2, flows the step of show a kind of compression and storage method according to an embodiment of the invention
Journey schematic diagram, specifically may include steps of：
Step 201, the compression parameters for obtaining data to be stored, and/or, obtain and data pair to be stored
The device parameter of the storage device answered；
Step 202, according to the compression parameters and/or device parameter, selection and the data to be stored
Corresponding targeted compression scheme；
Step 203, storage is compressed to the data to be stored using the targeted compression scheme；
Relative to embodiment illustrated in fig. 1, the compression parameters in the present embodiment can specifically include application scenarios
Parameter and/or database schema parameter, then above-mentioned steps 202 may further include：
Step 221, select from least one storage engines of database with the application scenarios parameter and
/ or the corresponding target storage engines of database schema parameter；
Step 222, from least one compression algorithm of the target storage engines select corresponding target
Compression algorithm, the target storage engines and its corresponding targeted compression algorithm can constitute the target pressure
Contracting scheme.
From at least one storage engines of database, selection most agrees with the application scenarios to the present embodiment
The target storage engines of parameter and/or database schema parameter, and carried out by above-mentioned target storage engines
Compression storage, therefore, it is possible in the case where the advantage of above-mentioned target storage engines is played as far as possible,
Treat compressed data and effectively compressed storage, such that it is able to improve handling up for database as much as possible
The performances such as amount.
In a kind of alternative embodiment of the invention, can be deposited by each of internet crawl database
The resource information of engine is stored up, and above-mentioned resource information is analyzed to obtain each storage engines correspondence
Advantage keyword and inferior position keyword, so, during implementation steps 221, can be described
Application scenarios parameter and/or database schema parameter are carried out with above-mentioned advantage keyword or inferior position keyword
Matching, if successful with inferior position Keywords matching, can exclude corresponding storage engines, and if advantage
Keywords matching success, then can be by corresponding storage engines alternately storage engines.Wherein, exist
When the number of candidate storage engine is more than 1, can be according to the side such as number of advantage keyword that the match is successful
Formula selects one as target storage engines from candidate storage engine, or, can be all alternative
Storage engines are exported so that user therefrom selects a target storage engines.In the number of candidate storage engine
When mesh is equal to 1, can directly using the candidate storage engine as target storage engines.
It is appreciated that the acquisition process of above-mentioned advantage keyword and inferior position keyword is intended only as optional reality
Example is applied, in fact, those skilled in the art can also be drawn according to practical application request using target storage
Other selection courses held up, such as by the advantage keyword of the 5th preset interface user input and bad
Gesture keyword, or, can by analyze storage engines history usage record or user for
The comment data of storage engines, it is crucial to obtain the corresponding advantage keyword of each storage engines and inferior position
Word etc., embodiment of the present invention advantage keyword corresponding for each storage engines and inferior position keyword
Acquisition process is not any limitation as.
The embodiment of the present invention can provide and select to be answered with described from least one storage engines of database
With scenario parameters and/or the following selection scheme of the corresponding target storage engines of database schema parameter：
Selection scheme 1
In selection scheme 1, selected at least one storage engines from database and the applied field
The step of scape parameter corresponding target storage engines, may further include：
Step A1, the application scenarios parameter are the first application scenarios parameter, then select corresponding first
Storage engines are target storage engines；Or,
Step A2, the application scenarios parameter are the second application scenarios parameter, then select corresponding second
Storage engines are target storage engines；
Wherein, the first application scenarios parameter can specifically include：Transaction Processing scene is joined
At least one in number and the intensive scenario parameters of reading；
Wherein, the second application scenarios parameter can specifically include：On-line analytical processing scene is joined
Number and batch are loaded and read at least one in intensive scenario parameters.
In the embodiment of the present invention, OLTP (Transaction Processing, On-Line Transaction
Processing) be traditional relevant database main application, be mainly used in basic, daily
Issued transaction, such as bank transaction.Because OLTP scenes are usually directed to wall scroll or a small number of database notes
Insert, update, deletion, inquiry of record etc. are operated；And InnoDB can make full use of CPU many
The advantage of core/high frequency, while reducing IO consumption, improves handling capacity, is particularly suitable for reading intensive answering
With, therefore, in a kind of alternative embodiment of the invention, above-mentioned first storage engines can specifically be wrapped
Include：InnoDB.It is appreciated that agreeing with Transaction Processing scenario parameters and/or reading intensive scenario parameters
Any storage engines can be as above-mentioned first storage engines, the embodiment of the present invention is for specific
First storage engines are not any limitation as.
OLAP (on-line analytical processing, Online Analytical Processing) is data warehouse
Main application, it is supported complicated analysis operation, stresses decision support, and provide visual and understandable
Query Result.It is usually directed to the loading of batch data storehouse record due to OLTP scenes, and TokuDB has
There is the advantage that batch loading performance is more preferable, insertion performance is more excellent, therefore, it is optional in one kind of the invention
In embodiment, above-mentioned second storage engines can specifically include：TokuDB.It is appreciated that agreeing with connection
Machine analyzes and processes scenario parameters and/or batch loads and read any storage engines of intensive scenario parameters
As above-mentioned second storage engines, the embodiment of the present invention is not subject to for specific second storage engines
Limitation.
Selection scheme 2
In selection scheme 2, selected at least one storage engines from database and the database
The step of configuration parameters corresponding target storage engines, may further include：
Step B1, the database schema parameter are the first database schema parameter, then select corresponding
3rd storage engines are target storage engines；Or,
Step B2, the database schema parameter are the second database schema parameter, then select corresponding
4th storage engines are target storage engines；
Wherein, the first database schema parameter can specifically include：Client/server parameter；Or
Person, the second database schema parameter can specifically include：Non- principal and subordinate's configuration parameters.
In the embodiment of the present invention, above-mentioned client/server parameter can be used for characterization database framework and support principal and subordinate
The framework of duplication.Using client/server, broken down in master server, it is possible to use come from server
Service is provided, therefore the stability of database can be provided.Also, separately locate on principal and subordinate's server
The request of user is managed, data-handling efficiency can be lifted.The data duplication on master server is arrived in addition
From server, data are protected from unexpected loss.
The embodiment of the present invention it has been investigated that, TokuDB at present client/server support on existing defects,
That is, TokuDB has problem with hypotactic compatibility, and InnoDB can be supported preferably
Client/server.Therefore in a kind of alternative embodiment of the invention, above-mentioned first storage engines specifically can be with
Including：InnoDB, above-mentioned second storage engines can specifically include：TokuDB.It is appreciated that contract
Any storage engines for closing client/server parameter can agree with non-master as above-mentioned first storage engines
Can be as above-mentioned second storage engines from any storage engines of configuration parameters；The embodiment of the present invention
It is not any limitation as specific first storage engines and the second storage engines.
Reference picture 3, flows the step of show a kind of compression and storage method according to an embodiment of the invention
Journey schematic diagram, specifically may include steps of：
Step 301, the compression parameters for obtaining data to be stored, and/or, obtain and data pair to be stored
The device parameter of the storage device answered；
Step 302, according to the compression parameters and/or device parameter, selection and the data to be stored
Corresponding targeted compression scheme；
Step 303, storage is compressed to the data to be stored using the targeted compression scheme；
Relative to embodiment illustrated in fig. 1, the compression parameters in the present embodiment can specifically include compression index
Parameter, then above-mentioned steps 302 may further include：
Step 321, from least one compression algorithm of target storage engines selection agree with it is described compression refer to
The targeted compression algorithm of mark parameter and/or the device parameter, target storage engines and its corresponding
Targeted compression algorithm can constitute the targeted compression scheme.
Compression index parameter can be used to characterize the compression index of data to be stored, for example, compression index is joined
Number can specifically include at least one in following parameter：Compression time parameter and compression ratio parameter.On
State compression time parameter to can be used to characterize compression storage the spent time, compression ratio parameter can be used for table
Levy space accounting of the data relative to data to be stored after compressing.The present embodiment is from target storage engines
Selection agrees with the target of the compression index parameter and/or the device parameter at least one compression algorithm
Compression algorithm, can ensure compression storage stability and save data-base cluster in storage device
Resource while, optimal compression index is realized as far as possible.
In a kind of alternative embodiment of the invention, it is assumed that above-mentioned target storage engines have three kinds or
More than three kinds of targeted compression algorithm, then it is described to agree with the compression index parameter and/or equipment ginseng
Several targeted compression algorithms can specifically include：
Agree with the first object compression algorithm of first order compression index parameter；Or
Agree with the second targeted compression algorithm of second level compression index parameter；Or
Agree with the 3rd targeted compression algorithm of third level compression index parameter；Wherein, the first order pressure
Contracting index parameter, second level compression index parameter and the corresponding compression index of third level compression index parameter
Successively decrease；Or
Agree with the first object compression algorithm of first order device parameter；Or
Agree with the second targeted compression algorithm of second level device parameter；Or
Agree with the 3rd targeted compression algorithm of third level device parameter；Wherein, the first order equipment ginseng
Number, second level device parameter and the corresponding equipment performance of third level device parameter are successively decreased.
In a kind of application example of the invention, it is assumed that above-mentioned target storage engines are TokuDB,
TokuDB has lzma algorithms, zlib algorithms and quicklz algorithms, wherein, lzma algorithms, zlib
Algorithm and the corresponding compression ratio parameter of quicklz algorithms are respectively：2.626857th, 2.285948 and
1.700141；Then can be according to above-mentioned compression ratio parameter setting lzma algorithms, zlib algorithms and quicklz
The compression of the corresponding first order compression index parameter of algorithm, second level compression index parameter and the third level
Index parameter, for example, for lzma algorithms set first order compression index parameter first interval for [2.3,
2.7], the second interval for setting second level compression index parameter for zlib algorithms is [1.8,2.29], for
The 3rd interval that quicklz algorithms set third level compression index parameter is [2,2.28], then can currently
Compression index parameter is matched with above-mentioned first interval, second interval and 3rd interval, will be matched into
Algorithm corresponding to work(is used as current goal compression algorithm.For example, current compression index parameter is 2.4,
Then current goal compression algorithm can be lzma algorithms.
In another application example of the invention, it is assumed that lzma algorithms, zlib algorithms and quicklz are calculated
The corresponding CPU core number of method is respectively：More than 8 cores, between the core of 4 core -8 and below 4 cores；Then can be by
Current compression index parameter is matched with above-mentioned 3 kinds of CPU core numbers, by the match is successful corresponding calculation
Method is used as current goal compression algorithm.For example, the current CPU core number of storage device is 6 cores, then may be used
Most agree with the zlib algorithms of current CPU core number with selection.
It is appreciated that above-mentioned compression ratio parameter and CPU core number can be applied in combination, specifically, can be with
Matching result to two kinds of compression index parameters is weighted averagely, and selects highest weighted average knot
Fruit corresponding to compression algorithm as targeted compression algorithm etc., it will be understood that the embodiment of the present invention for
The detailed process for being applied in combination compression ratio parameter and CPU core number is not any limitation as.
It should be noted that above-mentioned CPU core number is intended only as the alternative embodiment of device parameter, it is actual
On, the said equipment parameter can also include：For characterizing the disk parameter of disk performance and for characterizing
Memory parameters of internal memory performance etc., the embodiment of the present invention is not any limitation as specific device parameter.
In addition, it is necessary to explanation, the target storage engines in the present embodiment can be using shown in Fig. 2
The target storage engines of embodiment selection, or the storage engines that user specifies, the present invention is implemented
Example is not any limitation as the specific acquisition process of above-mentioned target storage engines.
For embodiment of the method, in order to be briefly described, therefore it is all expressed as a series of action group
Close, but those skilled in the art should know, and the embodiment of the present invention is not suitable by described action
The limitation of sequence, because according to the embodiment of the present invention, some steps can using other orders or simultaneously
Carry out.Secondly, those skilled in the art should also know, embodiment described in this description belongs to
Necessary to alternative embodiment, the involved action not necessarily embodiment of the present invention.
Reference picture 4, shows that a kind of structure for compressing storage device according to an embodiment of the invention is shown
It is intended to, can specifically includes such as lower module：
Acquisition module 401, the compression parameters for obtaining data to be stored, and/or, obtain and wait to deposit
The device parameter of the corresponding storage device of storage data；
Selecting module 402, for according to the compression parameters and/or device parameter, selection to be treated with described
The corresponding targeted compression scheme of data storage；And
Compression memory module 403, for being carried out to the data to be stored using the targeted compression scheme
Compression storage.
In a kind of alternative embodiment of the invention, the compression parameters can specifically include application scenarios
Parameter and/or database schema parameter, then the selecting module 402, may further include：
First choice submodule, for selecting to be answered with described from least one storage engines of database
With scenario parameters and/or the corresponding target storage engines of database schema parameter；And
Second selection submodule, for being selected from least one compression algorithm of the target storage engines
Corresponding targeted compression algorithm is selected, the target storage engines and its corresponding targeted compression algorithm can groups
Into the targeted compression scheme.
In another alternative embodiment of the invention, the first choice submodule can be further
Including：
First choice unit, for when the application scenarios parameter is the first application scenarios parameter, selecting
Corresponding first storage engines are selected for target storage engines；Or,
Second select unit, for when the application scenarios parameter is the second application scenarios parameter, selecting
Corresponding second storage engines are selected for target storage engines；
Wherein, the first application scenarios parameter can specifically include：Transaction Processing scene is joined
At least one in number and the intensive scenario parameters of reading；
Wherein, the second application scenarios parameter can specifically include：On-line analytical processing scene is joined
Number and batch are loaded and read at least one in intensive scenario parameters.
In another alternative embodiment of the invention, the first choice submodule can be further
Including：
3rd select unit, for being the first database schema parameter in the database schema parameter
When, it is target storage engines to select corresponding 3rd storage engines；Or,
4th select unit, for being the second database schema parameter in the database schema parameter
When, it is target storage engines to select corresponding 4th storage engines；
Wherein, the first database schema parameter can specifically include：Client/server parameter；Or
Person, the second database schema parameter can specifically include：Non- principal and subordinate's configuration parameters.
In another alternative embodiment of the invention, the compression parameters can specifically refer to including compression
Mark parameter, then the selecting module 402, may further include：
3rd selection submodule, for selecting contract from least one compression algorithm of target storage engines
The targeted compression algorithm of the compression index parameter and/or the device parameter is closed, the target storage is drawn
Hold up and its corresponding targeted compression algorithm can constitute the targeted compression scheme.
In a kind of alternative embodiment of the invention, the device parameter can specifically include following parameter
In at least one：CPU parameters, disk parameter and memory parameters.
In another alternative embodiment of the invention, the compression index parameter can specifically be included such as
At least one in lower parameter：Compression time parameter and compression ratio parameter.
It is described to agree with the compression index parameter and/or institute in another alternative embodiment of the invention
The targeted compression algorithm for stating device parameter can specifically include：
Agree with the first object compression algorithm of first order compression index parameter；Or
Agree with the second targeted compression algorithm of second level compression index parameter；Or
Agree with the 3rd targeted compression algorithm of third level compression index parameter；Wherein, the first order pressure
Contracting index parameter, second level compression index parameter and the corresponding compression index of third level compression index parameter
Successively decrease；Or
Agree with the first object compression algorithm of first order device parameter；Or
Agree with the second targeted compression algorithm of second level device parameter；Or
Agree with the 3rd targeted compression algorithm of third level device parameter；Wherein, the first order equipment ginseng
Number, second level device parameter and the corresponding equipment performance of third level device parameter are successively decreased.
For device embodiment, because it is substantially similar to embodiment of the method, so the ratio of description
Relatively simple, the relevent part can refer to the partial explaination of embodiments of method.
Provided herein algorithm and display not with any certain computer, virtual system or miscellaneous equipment
It is intrinsic related.Various general-purpose systems can also be used together with based on teaching in this.According to above
Description, the structure constructed required by this kind of system is obvious.Additionally, the present invention is not also directed to
Any certain programmed language.It is understood that, it is possible to use various programming languages realize described here
The content of invention, and the description done to language-specific above is to disclose optimal reality of the invention
Apply mode.
In specification mentioned herein, numerous specific details are set forth.It is to be appreciated, however, that
Embodiments of the invention can be put into practice in the case of without these details.In some instances,
Known method, structure and technology is not been shown in detail, so as not to obscure the understanding of this description.
Similarly, it will be appreciated that in order to simplify the disclosure and helping understand in each inventive aspect one
Individual or multiple, in above to the description of exemplary embodiment of the invention, each feature of the invention
Sometimes it is grouped together into single embodiment, figure or descriptions thereof.However, should be by
The method of the disclosure is construed to reflect following intention：I.e. the present invention for required protection requirement ratio is at each
The more features of feature being expressly recited in claim.More precisely, as following right will
As asking book to reflect, inventive aspect is all spies less than single embodiment disclosed above
Levy.Therefore, it then follows thus claims of specific embodiment are expressly incorporated in the specific embodiment party
Formula, wherein each claim are in itself as separate embodiments of the invention.
Those skilled in the art are appreciated that can be carried out to the module in the equipment in embodiment
Adaptively change and they are arranged in one or more equipment different from the embodiment.
Module or unit or component in embodiment can be combined into a module or unit or component, and
In addition multiple submodule or subelement or sub-component can be divided into.Except such feature and/or
Outside at least some in process or unit exclude each other, can be using any combinations to this explanation
All features disclosed in book (including adjoint claim, summary and accompanying drawing) and so disclosed
All processes or unit of any method or equipment are combined.Unless expressly stated otherwise, this theory
Each feature disclosed in bright book (including adjoint claim, summary and accompanying drawing) can be by offer phase
The alternative features of same, equivalent or similar purpose replace.
Although additionally, it will be appreciated by those of skill in the art that some embodiments described herein include
Some included features are rather than further feature, but the feature of different embodiments in other embodiments
Combination mean to be within the scope of the present invention and formed different embodiments.For example, under
In the claims in face, the one of any of embodiment required for protection can be in any combination
Mode is used.
All parts embodiment of the invention can realize with hardware, or with one or more
The software module run on reason device is realized, or is realized with combinations thereof.Those skilled in the art
It should be appreciated that can be realized using microprocessor or digital signal processor (DSP) in practice
Some or all parts in compression and storage method and device according to embodiments of the present invention some or
Person's repertoire.The present invention be also implemented as a part for performing method as described herein or
Person whole equipment or program of device (for example, computer program and computer program product).So
Realize that program of the invention can be stored on a computer-readable medium, or can have one or
The form of person's multiple signal.Such signal can be downloaded from Internet platform and obtained, or carried
There is provided on body signal, or provided in any other form.
It should be noted that above-described embodiment the present invention will be described is limited rather than to the present invention
Make, and those skilled in the art can design without departing from the scope of the appended claims
Alternative embodiment.In the claims, any reference symbol being located between bracket should not be configured to
Limitations on claims.Word " including " do not exclude the presence of element not listed in the claims or
Step.Word "a" or "an" before element does not exclude the presence of unit as multiple
Part.The present invention can be by means of the hardware for including some different elements and by means of properly programmed
Computer is realized.If some in these devices in the unit claim for listing equipment for drying
Individual can be embodied by same hardware branch.Word first, second, and third
Using not indicating that any order.These words can be construed to title.
The invention discloses A1, a kind of compression and storage method, including：
The compression parameters of data to be stored are obtained, and/or, obtain storage corresponding with data to be stored and set
Standby device parameter；
According to the compression parameters and/or device parameter, selection target corresponding with the data to be stored
Compression scheme；
Storage is compressed to the data to be stored using the targeted compression scheme.
A2, the method as described in A1, the compression parameters include application scenarios parameter and/or database
Configuration parameters, then it is described according to the compression parameters and/or device parameter, select and the number to be stored
The step of according to corresponding targeted compression scheme, further include：
Selected from least one storage engines of database and the application scenarios parameter and/or database
The corresponding target storage engines of configuration parameters；
Corresponding targeted compression is selected to calculate from least one compression algorithm of the target storage engines
Method, the target storage engines and its corresponding targeted compression algorithm constitute the targeted compression scheme.
Selection and institute in A3, the method as described in A2, at least one storage engines from database
The step of stating application scenarios parameter corresponding target storage engines, further includes：
The application scenarios parameter is the first application scenarios parameter, then select corresponding first storage engines
It is target storage engines；Or,
The application scenarios parameter is the second application scenarios parameter, then select corresponding second storage engines
It is target storage engines；
Wherein, the first application scenarios parameter includes：Transaction Processing scenario parameters and reading
At least one in intensive scenario parameters；
Wherein, the second application scenarios parameter includes：On-line analytical processing scenario parameters and batch
Amount loads and reads at least one in intensive scenario parameters.
Selection and institute in A4, the method as described in A2, at least one storage engines from database
The step of stating database schema parameter corresponding target storage engines, further includes：
The database schema parameter is the first database schema parameter, then select corresponding 3rd storage
Engine is target storage engines；Or,
The database schema parameter is the second database schema parameter, then select corresponding 4th storage
Engine is target storage engines；
Wherein, the first database schema parameter includes：Client/server parameter；Or, described
Two database schema parameters include：Non- principal and subordinate's configuration parameters.
A5, the method as described in A1, the compression parameters include compression index parameter, then the foundation
The compression parameters and/or device parameter, select the corresponding targeted compression scheme of the data to be stored
Step, further includes：
From at least one compression algorithm of target storage engines selection agree with the compression index parameter and/
Or the targeted compression algorithm of the device parameter, the target storage engines and its corresponding targeted compression
Algorithm constitutes the targeted compression scheme.
A6, the method as described in A5, the device parameter include at least one in following parameter：
CPU parameters, disk parameter and memory parameters.
A7, the method as described in A5, the compression index parameter include at least in following parameter
Kind：Compression time parameter and compression ratio parameter.
A8, the method as described in any in A5 to A7, it is described agree with the compression index parameter and/
Or the targeted compression algorithm of the device parameter includes：
Agree with the first object compression algorithm of first order compression index parameter；Or
Agree with the second targeted compression algorithm of second level compression index parameter；Or
Agree with the 3rd targeted compression algorithm of third level compression index parameter；Wherein, the first order pressure
Contracting index parameter, second level compression index parameter and the corresponding compression index of third level compression index parameter
Successively decrease；Or
Agree with the first object compression algorithm of first order device parameter；Or
Agree with the second targeted compression algorithm of second level device parameter；Or
Agree with the 3rd targeted compression algorithm of third level device parameter；Wherein, the first order equipment ginseng
Number, second level device parameter and the corresponding equipment performance of third level device parameter are successively decreased.
The invention discloses B9, a kind of compression storage device, including：
Acquisition module, the compression parameters for obtaining data to be stored, and/or, obtain and number to be stored
According to the device parameter of corresponding storage device；
Selecting module, for according to the compression parameters and/or device parameter, selecting to be stored with described
The corresponding targeted compression scheme of data；And
Compression memory module, for being pressed the data to be stored using the targeted compression scheme
Contracting storage.
B10, the device as described in B9, the compression parameters include application scenarios parameter and/or database
Configuration parameters, then the selecting module, further includes：
First choice submodule, for selecting to be answered with described from least one storage engines of database
With scenario parameters and/or the corresponding target storage engines of database schema parameter；And
Second selection submodule, for being selected from least one compression algorithm of the target storage engines
Corresponding targeted compression algorithm is selected, the target storage engines and its corresponding targeted compression algorithm are constituted
The targeted compression scheme.
B11, the device as described in B10, the first choice submodule, further include：
First choice unit, for when the application scenarios parameter is the first application scenarios parameter, selecting
Corresponding first storage engines are selected for target storage engines；Or,
Second select unit, for when the application scenarios parameter is the second application scenarios parameter, selecting
Corresponding second storage engines are selected for target storage engines；
Wherein, the first application scenarios parameter includes：Transaction Processing scenario parameters and reading
At least one in intensive scenario parameters；
Wherein, the second application scenarios parameter includes：On-line analytical processing scenario parameters and batch
Amount loads and reads at least one in intensive scenario parameters.
B12, the device as described in B10, the first choice submodule, further include：
3rd select unit, for being the first database schema parameter in the database schema parameter
When, it is target storage engines to select corresponding 3rd storage engines；Or,
4th select unit, for being the second database schema parameter in the database schema parameter
When, it is target storage engines to select corresponding 4th storage engines；
Wherein, the first database schema parameter includes：Client/server parameter；Or, described
Two database schema parameters include：Non- principal and subordinate's configuration parameters.
B13, the device as described in B9, the compression parameters include compression index parameter, the then choosing
Module is selected, is further included：
3rd selection submodule, for selecting contract from least one compression algorithm of target storage engines
The targeted compression algorithm of the compression index parameter and/or the device parameter is closed, the target storage is drawn
Hold up and its corresponding targeted compression algorithm constitutes the targeted compression scheme.
B14, the device as described in B13, the device parameter include at least one in following parameter：
CPU parameters, disk parameter and memory parameters.
B15, the device as described in B13, the compression index parameter include at least in following parameter
Kind：Compression time parameter and compression ratio parameter.
B16, the device as described in any in B13 to B15, it is described agree with the compression index parameter and
/ or the targeted compression algorithm of the device parameter include：
Agree with the first object compression algorithm of first order compression index parameter；Or
Agree with the second targeted compression algorithm of second level compression index parameter；Or
Agree with the 3rd targeted compression algorithm of third level compression index parameter；Wherein, the first order pressure
Contracting index parameter, second level compression index parameter and the corresponding compression index of third level compression index parameter
Successively decrease；Or
Agree with the first object compression algorithm of first order device parameter；Or
Agree with the second targeted compression algorithm of second level device parameter；Or
Agree with the 3rd targeted compression algorithm of third level device parameter；Wherein, the first order equipment ginseng
Number, second level device parameter and the corresponding equipment performance of third level device parameter are successively decreased.
Claims (10)
1. a kind of compression and storage method, including：
The compression parameters of data to be stored are obtained, and/or, obtain storage corresponding with data to be stored and set
Standby device parameter；
According to the compression parameters and/or device parameter, selection target corresponding with the data to be stored
Compression scheme；
Storage is compressed to the data to be stored using the targeted compression scheme.
2. the method for claim 1, it is characterised in that the compression parameters include applied field
Scape parameter and/or database schema parameter, then it is described according to the compression parameters and/or device parameter, choosing
The step of selecting targeted compression scheme corresponding with the data to be stored, further includes：
Selected from least one storage engines of database and the application scenarios parameter and/or database
The corresponding target storage engines of configuration parameters；
Corresponding targeted compression is selected to calculate from least one compression algorithm of the target storage engines
Method, the target storage engines and its corresponding targeted compression algorithm constitute the targeted compression scheme.
3. method as claimed in claim 2, it is characterised in that at least one from database
The step of target storage engines corresponding with the application scenarios parameter are selected in storage engines, further
Including：
The application scenarios parameter is the first application scenarios parameter, then select corresponding first storage engines
It is target storage engines；Or,
The application scenarios parameter is the second application scenarios parameter, then select corresponding second storage engines
It is target storage engines；
Wherein, the first application scenarios parameter includes：Transaction Processing scenario parameters and reading
At least one in intensive scenario parameters；
Wherein, the second application scenarios parameter includes：On-line analytical processing scenario parameters and batch
Amount loads and reads at least one in intensive scenario parameters.
4. method as claimed in claim 2, it is characterised in that at least one from database
The step of target storage engines corresponding with the database schema parameter are selected in storage engines, enters one
Step includes：
The database schema parameter is the first database schema parameter, then select corresponding 3rd storage
Engine is target storage engines；Or,
The database schema parameter is the second database schema parameter, then select corresponding 4th storage
Engine is target storage engines；
Wherein, the first database schema parameter includes：Client/server parameter；Or, described
Two database schema parameters include：Non- principal and subordinate's configuration parameters.
5. the method for claim 1, it is characterised in that the compression parameters refer to including compression
Mark parameter, then it is described according to the compression parameters and/or device parameter, select the data pair to be stored
The step of targeted compression scheme answered, further include：
From at least one compression algorithm of target storage engines selection agree with the compression index parameter and/
Or the targeted compression algorithm of the device parameter, the target storage engines and its corresponding targeted compression
Algorithm constitutes the targeted compression scheme.
6. method as claimed in claim 5, it is characterised in that the device parameter includes following ginseng
At least one in number：CPU parameters, disk parameter and memory parameters.
7. method as claimed in claim 5, it is characterised in that the compression index parameter is included such as
At least one in lower parameter：Compression time parameter and compression ratio parameter.
8. the method as described in any in claim 5 to 7, it is characterised in that described to agree with described
The targeted compression algorithm of compression index parameter and/or the device parameter includes：
Agree with the first object compression algorithm of first order compression index parameter；Or
Agree with the second targeted compression algorithm of second level compression index parameter；Or
Agree with the 3rd targeted compression algorithm of third level compression index parameter；Wherein, the first order pressure
Contracting index parameter, second level compression index parameter and the corresponding compression index of third level compression index parameter
Successively decrease；Or
Agree with the first object compression algorithm of first order device parameter；Or
Agree with the second targeted compression algorithm of second level device parameter；Or
Agree with the 3rd targeted compression algorithm of third level device parameter；Wherein, the first order equipment ginseng
Number, second level device parameter and the corresponding equipment performance of third level device parameter are successively decreased.
9. one kind compresses storage device, including：
Acquisition module, the compression parameters for obtaining data to be stored, and/or, obtain and number to be stored
According to the device parameter of corresponding storage device；
Selecting module, for according to the compression parameters and/or device parameter, selecting to be stored with described
The corresponding targeted compression scheme of data；And
Compression memory module, for being pressed the data to be stored using the targeted compression scheme
Contracting storage.
10. device as claimed in claim 9, it is characterised in that the compression parameters include applied field
Scape parameter and/or database schema parameter, then the selecting module, further includes：
First choice submodule, for selecting to be answered with described from least one storage engines of database
With scenario parameters and/or the corresponding target storage engines of database schema parameter；And
Second selection submodule, for being selected from least one compression algorithm of the target storage engines
Corresponding targeted compression algorithm is selected, the target storage engines and its corresponding targeted compression algorithm are constituted
The targeted compression scheme.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201511000457.3A CN106919617B (en) | 2015-12-25 | 2015-12-25 | Compression storage method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201511000457.3A CN106919617B (en) | 2015-12-25 | 2015-12-25 | Compression storage method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106919617A true CN106919617A (en) | 2017-07-04 |
| CN106919617B CN106919617B (en) | 2020-09-04 |
Family
ID=59455660
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201511000457.3A Active CN106919617B (en) | 2015-12-25 | 2015-12-25 | Compression storage method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106919617B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109068183A (en) * | 2018-08-08 | 2018-12-21 | 湖南快乐阳光互动娱乐传媒有限公司 | A kind of live streaming barrage distribution method and device |
| CN111061722A (en) * | 2018-10-16 | 2020-04-24 | 阿里巴巴集团控股有限公司 | Data compression method, data decompression method, device and equipment |
| CN112770114A (en) * | 2020-12-29 | 2021-05-07 | 平安普惠企业管理有限公司 | Image data compression method and device, computer equipment and storage medium |
| GB2605795A (en) * | 2021-04-13 | 2022-10-19 | British Telecomm | Algorithm selection for processor-controlled device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7177944B1 (en) * | 2002-04-13 | 2007-02-13 | Mindspeed Technologies, Inc. | Trans-compression selection and configuration in modem over packet networks |
| CN102761540A (en) * | 2012-05-30 | 2012-10-31 | 北京奇虎科技有限公司 | Data compression method, device and system and server |
| CN103049561A (en) * | 2012-12-31 | 2013-04-17 | 华为技术有限公司 | Data compressing method, storage engine and storage system |
| CN103516369A (en) * | 2013-06-20 | 2014-01-15 | 易乐天 | Method and system for self-adaptation data compression and decompression and storage device |
| US20140317340A1 (en) * | 2013-04-18 | 2014-10-23 | Hitachi Ltd. | Storage system and storage control method |
| CN104462334A (en) * | 2014-12-03 | 2015-03-25 | 天津南大通用数据技术股份有限公司 | Data compression method and device for packing database |
-
2015
- 2015-12-25 CN CN201511000457.3A patent/CN106919617B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7177944B1 (en) * | 2002-04-13 | 2007-02-13 | Mindspeed Technologies, Inc. | Trans-compression selection and configuration in modem over packet networks |
| CN102761540A (en) * | 2012-05-30 | 2012-10-31 | 北京奇虎科技有限公司 | Data compression method, device and system and server |
| CN103049561A (en) * | 2012-12-31 | 2013-04-17 | 华为技术有限公司 | Data compressing method, storage engine and storage system |
| US20140317340A1 (en) * | 2013-04-18 | 2014-10-23 | Hitachi Ltd. | Storage system and storage control method |
| CN103516369A (en) * | 2013-06-20 | 2014-01-15 | 易乐天 | Method and system for self-adaptation data compression and decompression and storage device |
| CN104462334A (en) * | 2014-12-03 | 2015-03-25 | 天津南大通用数据技术股份有限公司 | Data compression method and device for packing database |
Non-Patent Citations (1)
| Title |
|---|
| 王振玺: "DWMS中压缩技术的研究与实现", 《中国优秀硕士学位论文全文数据库》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109068183A (en) * | 2018-08-08 | 2018-12-21 | 湖南快乐阳光互动娱乐传媒有限公司 | A kind of live streaming barrage distribution method and device |
| CN111061722A (en) * | 2018-10-16 | 2020-04-24 | 阿里巴巴集团控股有限公司 | Data compression method, data decompression method, device and equipment |
| CN111061722B (en) * | 2018-10-16 | 2023-06-06 | 阿里巴巴集团控股有限公司 | Data compression and data decompression method, device and equipment |
| CN112770114A (en) * | 2020-12-29 | 2021-05-07 | 平安普惠企业管理有限公司 | Image data compression method and device, computer equipment and storage medium |
| WO2022142039A1 (en) * | 2020-12-29 | 2022-07-07 | 平安普惠企业管理有限公司 | Image data compression method and apparatus, computer device, and storage medium |
| CN112770114B (en) * | 2020-12-29 | 2023-03-28 | 平安普惠企业管理有限公司 | Image data compression method and device, computer equipment and storage medium |
| GB2605795A (en) * | 2021-04-13 | 2022-10-19 | British Telecomm | Algorithm selection for processor-controlled device |
| GB2605795B (en) * | 2021-04-13 | 2023-05-17 | British Telecomm | Algorithm selection for processor-controlled device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106919617B (en) | 2020-09-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104205039B (en) | The interest drive business intelligence system and method for data analysiss are carried out using interest driven data pipeline | |
| CN106919617A (en) | A kind of compression and storage method and device | |
| CN103177061B (en) | Unique value estimation in partition table | |
| CN105550241B (en) | Multi-dimensional database querying method and device | |
| CN103810224B (en) | information persistence and query method and device | |
| US20060026199A1 (en) | Method and system to load information in a general purpose data warehouse database | |
| CN105426375B (en) | A kind of calculation method and device of relational network | |
| CN107016025A (en) | A kind of method for building up and device of non-relational database index | |
| CN105022739B (en) | The storage method and device of data | |
| CN109376139A (en) | Centralized database monitoring method, computer installation and storage medium | |
| WO2015042070A1 (en) | Densely grouping dimensional data | |
| CN106201623B (en) | Game resource file loading method and system based on access order sequence | |
| WO2016174171A1 (en) | Method and database computer system for performing a database query using a bitmap index | |
| CN108288208A (en) | The displaying object of image content-based determines method, apparatus, medium and equipment | |
| CN111932135B (en) | Client risk identification method and device based on distributed database | |
| CN106354536A (en) | Method and device of loading ELF file of Linux system in Windows system | |
| CN106528778A (en) | Method and device for obtaining user retention data | |
| CN107451280B (en) | Data communication method and device and electronic equipment | |
| US7177885B2 (en) | Method and system for reorganizing a tablespace in a database | |
| CN111179068A (en) | Method, device and storage medium for determining enterprise performance credit level | |
| EP2251802A1 (en) | Method and program for generating a subset of data from a database | |
| CN107704568A (en) | A kind of method and device of test data addition | |
| CN107357721A (en) | The method and device of test system | |
| CN114493515A (en) | Power grid asset auditing method and device, storage medium and computer equipment | |
| US20190266163A1 (en) | System and method for behavior-on-read query processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240115 Address after: 100088 room 112, block D, 28 new street, new street, Xicheng District, Beijing (Desheng Park) Patentee after: BEIJING QIHOO TECHNOLOGY Co.,Ltd. Address before: 100088 room 112, block D, 28 new street, new street, Xicheng District, Beijing (Desheng Park) Patentee before: BEIJING QIHOO TECHNOLOGY Co.,Ltd. Patentee before: Qizhi software (Beijing) Co.,Ltd. |
|
| TR01 | Transfer of patent right |