CN106708435A - Data processing method and apparatus - Google Patents
Data processing method and apparatus Download PDFInfo
- Publication number
- CN106708435A CN106708435A CN201611153402.0A CN201611153402A CN106708435A CN 106708435 A CN106708435 A CN 106708435A CN 201611153402 A CN201611153402 A CN 201611153402A CN 106708435 A CN106708435 A CN 106708435A
- Authority
- CN
- China
- Prior art keywords
- data
- hard disk
- buffer zone
- utilization rate
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0611—Improving I/O performance in relation to response time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0616—Improving the reliability of storage systems in relation to life time, e.g. increasing Mean Time Between Failures [MTBF]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0685—Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Signal Processing For Digital Recording And Reproducing (AREA)
Abstract
The present application provides a data processing method and apparatus. The method comprises: determining first target data with the lowest utilization rate in the data stored in a first type of hard disk; and after storing the first target data into a second type of hard disk, deleting the first target data in the first type of hard disk, wherein the data read/write speed of the first type of hard disk is higher than the data read/write speed of the second type of hard disk. According to the method and apparatus provided by the present application, first target data with the lowest utilization rate in the data stored in the first type of hard disk can be stored into the second type of hard disk, namely, the commonly used data is stored in the first type of hard disk with faster data read/write speed, and not commonly used data is stored in the second type of hard disk with slower data read/write speed, so that under the condition of improving part of the cost, the data read/write speed is improved, and the balance between the cost of hard disk and data read/write performance is reached.
Description
Technical field
This disclosure relates to the communications field, more particularly to data processing method and device.
Background technology
At present, the demand of ecommerce increasingly increases, the operation response speed of electric quotient data storehouse system, directly influences electricity
The response speed of sub- business system, has a direct impact to Consumer's Experience.
In correlation technique, can be using solid state hard disc or the data in mechanical hard disk data storage storehouse.If hard using solid-state
Disk, although reading and writing data speed, but price is costly.If using mechanical hard disk, although cost is relatively low, but data
Read or write speed is slower.Balance cannot be reached in terms of the cost of hard disk and data read or write speed.
The content of the invention
In view of this, present disclose provides data processing method and device, to solve the deficiency in correlation technique.
According to the first aspect of the embodiment of the present disclosure, there is provided a kind of data processing method, methods described includes：
The minimum first object data of utilization rate are determined in the data of first kind hard-disc storage；
After by the first object data Cun Chudao Equations of The Second Kind hard disks, first mesh is deleted in the first kind hard disk
Mark data；
Wherein, reading and writing data speed of the reading and writing data speed of the first kind hard disk higher than the Equations of The Second Kind hard disk.
Alternatively, the minimum first object data of determination utilization rate in the data of first kind hard-disc storage, including：
Using the minimum data of utilization rate in the data of at least one buffer zone storage included by first kind hard disk as
First object data.
Alternatively, in the data of at least one buffer zone storage in the following ways included by determination first kind hard disk
The minimum data of utilization rate：
According to algorithm is at least used in the recent period, at interval of the first Preset Time segment mark in the data of each buffer zone storage
Remember and the data of utilization rate highest first；
The second data of least number of times are labeled in statistics predetermined period in the data of each buffer zone storage；
The minimum data of utilization rate in the data that second data are stored as corresponding buffer zone.
Alternatively, when the number of the buffer zone is for multiple, also include：
Determination meets the second target data of data exchange condition in the data of the first buffer zone storage；
By second target data storage to behind the second buffer zone, deleting described the in first buffer zone
Two target datas；
Wherein, the utilization rate of the data for being stored in first buffer zone is less than storage in second buffer zone
The utilization rate of data.
Alternatively, the data exchange condition includes：
Number of times is accessed in the second preset time period and reaches preset value.
Alternatively, also include：
The data stored in the first kind hard disk or the Equations of The Second Kind hard disk are stored using row storage mode.
According to the second aspect of the embodiment of the present disclosure, there is provided a kind of data processing equipment, described device includes：
First determining module, is configured as determining the minimum first object of utilization rate in the data of first kind hard-disc storage
Data；
First data processing module, after being configured as the first object data Cun Chudao Equations of The Second Kind hard disks, described
The first object data are deleted in first kind hard disk；
Wherein, reading and writing data speed of the reading and writing data speed of the first kind hard disk higher than the Equations of The Second Kind hard disk.
Alternatively, first determining module includes：
Determination sub-module, is configured as to make in the data of at least one buffer zone storage included by first kind hard disk
With the minimum data of rate as first object data.
Alternatively, the determination sub-module includes：
Indexing unit, is configured as according to algorithm is at least used in the recent period, every in the data of each buffer zone storage
The data of utilization rate highest first are marked every the first preset time period；
Statistic unit, is configured as counting in predetermined period labeled number of times in the data of each buffer zone storage
The second minimum data；
Determining unit, utilization rate is most in being configured as the data that second data are stored as corresponding buffer zone
Low data.
Alternatively, also include：
Second determining module, is configured as determining to meet data exchange condition in the data of the first buffer zone storage
Second target data；
Second data processing module, after being configured as second target data storage to the second buffer zone, in institute
State and delete second target data in the first buffer zone；
Wherein, the utilization rate of the data for being stored in first buffer zone is less than storage in second buffer zone
The utilization rate of data.
Alternatively, the data exchange condition includes：
Number of times is accessed in the second preset time period and reaches preset value.
Alternatively, also include：
Data memory module, is configured as data of the storage in the first kind hard disk or the Equations of The Second Kind hard disk and uses
Row storage mode is stored.
According to the third aspect of the embodiment of the present disclosure, there is provided a kind of data processing equipment, including：
Processor；
Memory for storing processor-executable instruction；
Wherein, the processor is configured as：
The minimum first object data of utilization rate are determined in the data of first kind hard-disc storage；
After by the first object data Cun Chudao Equations of The Second Kind hard disks, first mesh is deleted in the first kind hard disk
Mark data；
Wherein, reading and writing data speed of the reading and writing data speed of the first kind hard disk higher than the Equations of The Second Kind hard disk.
The technical scheme provided by this disclosed embodiment can include the following benefits：
In the embodiment of the present disclosure, the minimum first object number of utilization rate can be determined in the data of first kind hard-disc storage
According to, and then by the first object data Cun Chudao Equations of The Second Kind hard disks, and delete described first on the first kind hard disk
Target data.Wherein, reading and writing data speed of the reading and writing data speed of the first kind hard disk higher than the Equations of The Second Kind hard disk.It is logical
Said process is crossed, can be hard by the minimum first object data Cun Chudao Equations of The Second Kind of utilization rate in the data of first kind hard-disc storage
In disk, will frequently-used data be stored on the first kind hard disk of reading and writing data speed, the data that will be of little use are stored in data
On the slower Equations of The Second Kind hard disk of read or write speed.In the case where departmental cost is improved, reading and writing data speed is improve, so as to hard
Balance is reached on the cost and data readwrite performance of disk.
In the embodiment of the present disclosure, will can make in the data of at least one buffer zone storage included by first kind hard disk
With the minimum data of rate as first object data.It is alternatively possible to according to algorithm is at least used in the recent period, in each buffer zone
In the data of storage the data of utilization rate highest first are marked at interval of the first preset time period.Count each in predetermined period
The data of least number of times, i.e. the second data are labeled in the data of buffer zone storage.In the embodiment of the present disclosure, second number
According to the data that utilization rate in the data that as corresponding buffer zone stores is minimum.Simplicity is realized, availability is high.
In the embodiment of the present disclosure, it is contemplated that when full table scan is carried out to database, calculated using above-mentioned minimum use in the recent period
Method, mass data that can be by script storage on first kind hard disk is changed to storage to Equations of The Second Kind hard disk, causes high-speed data to be read
Region is write to be contaminated.Can be multiple buffer zones by first kind hard disk partition in order to avoid above-mentioned situation, and according to utilization rate
In placing the data in different buffer zones.
In the embodiment of the present disclosure, the second mesh of data exchange condition is met in the data that the first buffer zone can be stored
Mark data, are changed to store in the second buffer zone.Wherein, the utilization rate of the data for being stored in first buffer zone is less than
The utilization rate of the data stored in second buffer zone, the data exchange condition is to be interviewed in the second preset time period
Ask that number of times reaches preset value.Utilization rate is weighed by being accessed number of times in the second preset time period, is realized according to using
Rate places the data in the purpose in different buffer zones.
In the embodiment of the present disclosure, when e-commerce system scale is smaller, the connection table inquiry relatively low electronics of probability of database
Business system, the data that can now will be stored in the first kind hard disk or the Equations of The Second Kind hard disk use row storage mode
Stored.Further, frequently-used data is stored on the first kind hard disk of reading and writing data speed, by the data that are of little use
It is stored on the slow Equations of The Second Kind hard disk of reading and writing data.In the case where departmental cost is improved, reading and writing data speed is improve
Degree, so as to reach balance on the cost and data readwrite performance of hard disk.
In the embodiment of the present disclosure, can accordingly be adjusted by adjusting first kind hard disk proportion in whole hard disks
Reading and writing data speed.Meanwhile, introduce that data read or write speed is slower but lower-cost Equations of The Second Kind hard disk, so as in the cost of hard disk
Equalization point is found with data readwrite performance.
It should be appreciated that the general description of the above and detailed description hereinafter are only exemplary and explanatory, not
The disclosure can be limited.
Brief description of the drawings
Accompanying drawing herein is merged in specification and constitutes the part of this specification, shows the implementation for meeting the disclosure
Example, and it is used to explain the principle of the disclosure together with specification.
Fig. 1 is a kind of data processing method flow chart of the disclosure according to an exemplary embodiment；
Fig. 2 is another data processing method flow chart of the disclosure according to an exemplary embodiment；
Fig. 3 A to 3B are data processing schematic diagram of a scenario of the disclosure according to an exemplary embodiment；
Fig. 4 is another data processing method flow chart of the disclosure according to an exemplary embodiment；
Fig. 5 is a kind of data processing schematic diagram of a scenario of the disclosure according to an exemplary embodiment；
Fig. 6 is another data processing schematic diagram of a scenario of the disclosure according to an exemplary embodiment；
Fig. 7 is a kind of data processing equipment block diagram of the disclosure according to an exemplary embodiment；
Fig. 8 is another data processing equipment block diagram of the disclosure according to an exemplary embodiment；
Fig. 9 is another data processing equipment block diagram of the disclosure according to an exemplary embodiment；
Figure 10 is another data processing equipment block diagram of the disclosure according to an exemplary embodiment；
Figure 11 is another data processing equipment block diagram of the disclosure according to an exemplary embodiment；
Figure 12 is an a kind of structural representation for data processing equipment of the disclosure according to an exemplary embodiment
Figure.
Specific embodiment
Here exemplary embodiment will be illustrated in detail, its example is illustrated in the accompanying drawings.Following description is related to
During accompanying drawing, unless otherwise indicated, the same numbers in different accompanying drawings represent same or analogous key element.Following exemplary embodiment
Described in implementation method do not represent all implementation methods consistent with the disclosure.Conversely, they be only with it is such as appended
The example of the consistent apparatus and method of some aspects described in detail in claims, the disclosure.
It is the purpose only merely for description specific embodiment in the term of disclosure operation, and is not intended to be limiting the disclosure.
" one kind ", " described " and " being somebody's turn to do " for the singulative run in disclosure and the accompanying claims book is also intended to include majority
Form, unless context clearly shows that other implications.It is also understood that the term "and/or" of operation refers to and wraps herein
May be combined containing one or more associated any or all of project listed.
It will be appreciated that though various information, but this may be described using term first, second, third, etc. in the disclosure
A little information should not necessarily be limited by these terms.These terms are only used for being distinguished from each other open same type of information.For example, not departing from
In the case of disclosure scope, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as
One information.Depending on linguistic context, as word that this is run " if " can be construed to " and ... when " or " when ...
When " or " in response to determining ".
The data processing method that the embodiment of the present disclosure is provided can be used for e-commerce system.As shown in figure 1, Fig. 1 is root
According to a kind of data processing method shown in an exemplary embodiment, comprise the following steps：
In a step 101, the minimum first object data of utilization rate are determined in the data of first kind hard-disc storage.
In the embodiment of the present disclosure, alternatively, the first kind hard disk can be the hard disk of reading and writing data speed, for example
Solid state hard disc.The first kind hard disk includes at least one buffer zone.
In this step, e-commerce system can store at least one buffer zone included by the first kind hard disk
Data in the minimum data of utilization rate as first object data.Optionally it is determined that included by first kind hard disk at least one
The process of the minimum data of utilization rate in the data of individual buffer zone storage, as shown in Fig. 2 Fig. 2 is being implemented shown in earlier figures 1
Another data processing method shown on the basis of example, comprises the following steps：
In step 101-1, according at least using algorithm in the recent period, at interval of the in the data of each buffer zone storage
One preset time period marks the data of utilization rate highest first.
In this step, each buffer zone can be stored using mode is lined up to data, as shown in Figure 3A.According near
Phase at least uses algorithm, and correlation technique can be in the data of each buffer zone storage at interval of the first Preset Time segment mark
Go out the data of utilization rate highest first, it is alternatively possible to first data are moved into queue head, such as shown in Fig. 3 B,
And the number of times increase for being labeled first data is once.
In step 101-2, number of times is labeled most in the data of each buffer zone storage in statistics predetermined period
The second few data.
In this step, can be counted in predetermined period be labeled least number of times in the data of each buffer zone storage
Data, i.e. the second data.
For example buffer zone 1 storage data include data 1, data 2 ... data n.Counted in predetermined period slow
Deposit the least number of times being labeled in the above-mentioned data of the storage of region 1, it is assumed that data m labeled least number of times, then data m is
Second data.
In step 101-3, utilization rate is minimum in the data that second data are stored as corresponding buffer zone
Data.
In this step, in the data that the second data in each buffer zone can be stored as corresponding buffer zone
The minimum data of utilization rate.
For example, first kind hard disk includes 2 buffer zones, the second data determined in buffer zone 1 are data m1, caching
The second data determined in region 2 are data m2, then m1And m2Respectively make in the data of buffer zone 1 and the storage of buffer zone 2
With the data that rate is minimum.
In a step 102, after by the first object data Cun Chudao Equations of The Second Kind hard disks, deleted in the first kind hard disk
Except the first object data.
In the embodiment of the present disclosure, alternatively, the Equations of The Second Kind hard disk can be that reading and writing data speed is less than the first kind
The hard disk of the reading and writing data speed of hard disk, such as mechanical hard disk.The first kind hard disk includes at least one buffer zone.Typically
In the case of, the fast hard disk of read or write speed is relatively costly, i.e., the cost of the cost higher than Equations of The Second Kind hard disk of described first kind hard disk.
In this step, can be according to correlation technique by the first object data Cun Chudao Equations of The Second Kind hard disks, and in institute
State and delete the first object data in first kind hard disk.During the first object data are exactly the data of first kind hard-disc storage
The minimum data of utilization rate.Wherein, if the first kind hard disk includes more than one buffer zone, each buffer zone pair
The first object data answered are changed to be stored on Equations of The Second Kind hard disk.
By said process by the minimum first object data Cun Chudao of utilization rate in the data of first kind hard-disc storage
In two class hard disks, so as to frequently-used data is stored on the first kind hard disk of reading and writing data speed, the data that will be of little use are deposited
It is placed on the slow Equations of The Second Kind hard disk of reading and writing data.
In above-described embodiment, the minimum first object number of utilization rate can be determined in the data of first kind hard-disc storage
According to, and then by the first object data Cun Chudao Equations of The Second Kind hard disks, and delete described first on the first kind hard disk
Target data.Wherein, reading and writing data speed of the reading and writing data speed of the first kind hard disk higher than the Equations of The Second Kind hard disk.It is logical
Said process is crossed, can be hard by the minimum first object data Cun Chudao Equations of The Second Kind of utilization rate in the data of first kind hard-disc storage
In disk, will frequently-used data be stored on the first kind hard disk of reading and writing data speed, the data that will be of little use are stored in data
On the slower Equations of The Second Kind hard disk of read or write speed.In the case where departmental cost is improved, reading and writing data speed is improve, so as to hard
Balance is reached on the cost and data readwrite performance of disk.
It should be noted that when full table scan is carried out to database, algorithm is at least used in the recent period using above-mentioned, can be by original
Substantial amounts of data of this storage on first kind hard disk are changed to storage to Equations of The Second Kind hard disk, cause high-speed data to read and write region quilt
Pollution.Wherein, the full table scan is to search each of the data list process of record, all meets given until finding
Untill the record of condition is returned.Can be multiple buffer zones by first kind hard disk partition in order to avoid above-mentioned situation, and according to
Utilization rate is placed the data in different buffer zones.
Therefore, in the embodiment of the present disclosure, above-mentioned data processing method is as shown in figure 4, Fig. 4 is exemplified according to an implementation
Another data processing method, it is further comprising the steps of：
In step 201, determine to meet the second target of data exchange condition in the data of the first buffer zone storage
Data.
Wherein, the data exchange condition includes：
Number of times is accessed in the second preset time period and reaches preset value.
In the embodiment of the present disclosure, the utilization rate of the data stored in first buffer zone is relatively low.
In this step, making for data can be weighed by the accessed number of times of the data in second preset time period
With rate.It is determined that during second target data, in the data that first buffer zone can be stored when second is default
Between number of times be accessed in section reach the data of preset value as second target data.
Alternatively, the duration and the preset value of second preset time period can be adjusted according to actual conditions.
In step 202., after by second target data storage to the second buffer zone, in first buffer zone
It is middle to delete second target data.
In this step, the second target data that utilization rate in the first buffer zone meets data exchange condition is changed to storage
To in the second buffer zone.Will second target data stored to behind the second buffer zone, described according to correlation technique
Second target data is deleted in first buffer zone.
For example, shown in Fig. 5, data accessed number of times in the second preset time period reaches default in the first buffer zone
, then be changed to for data k store in the second buffer zone by value.
, it is necessary to the data list being maintained in each buffer zone, incites somebody to action so as to reach according to utilization rate in above-described embodiment
Data are stored in the purpose of different buffer zones.
In the embodiment of the present disclosure, the data that can be will be stored in the first kind hard disk or the Equations of The Second Kind hard disk are used
Row storage mode is stored.The row storage is stored by field.
For example tables of data 1 is as follows：
Table 1
| Time buying | User name | Purchase commodity |
| Time 1 | Zhang San | Commodity a |
| Time 2 | Li Si | Commodity b |
| …… | …… | …… |
It is changed into time 1, time 2 ... ... if according to row storage；Zhang San, Li Si ... ...；Commodity a, commodity b ... ....
Row storage mode is not suitable for Transaction Processing generally, that is, be not suitable for recording instant increase, delete
Except, modification and inquiry etc., if e-commerce system is larger, row storage mode just no longer be applicable.Therefore, disclosure reality
The above-mentioned data processing method for applying example offer is applied to the relatively low situation of connection table inquiry probability, and the inquiry of connection table refers to inquiry database
In all data lists, by one query obtain required for result inquiry.That is the less situation of e-commerce system scale.
At this point it is possible to data are stored according to row storage mode.
In the embodiment of the present disclosure, it can be seen that if the number of first kind hard disk is more, reading and writing data speed is also faster,
E-commerce system can just get over fast-response user's request.When but first kind hard disk proportion is excessive, reading and writing data speed
Increase just become less substantially, and cost can increase a lot.Accordingly, it would be desirable to adjust in all hard disks shared by first kind hard disk
Ratio, introduces that data read or write speed is slower but lower-cost Equations of The Second Kind hard disk, so as in the cost and reading and writing data of hard disk
Can on find equalization point.
Above-mentioned data processing method is further illustrated below as follows.
The above-mentioned data processing method that the embodiment of the present disclosure is provided is applied to the relatively low situation of database connection table inquiry probability,
That is the less situation of e-commerce system scale.Assuming that first kind hard disk is solid state hard disc, Equations of The Second Kind hard disk is mechanical hard disk, Gu
The reading and writing data speed of state hard disk is higher than the reading and writing data speed of mechanical hard disk, and the cost of solid state hard disc is higher than mechanical hard disk
Cost.All data are stored according to row storage mode.
Solid state hard disc is divided into two buffer zones, respectively the first buffer zone and the second buffer zone, wherein, institute
State the utilization rate of the utilization rate less than the data stored in second buffer zone of the data stored in the first buffer zone.
If detecting a certain data in the data of the first buffer zone storage, it is accessed in the second preset time period
Number of times reaches preset value, then using the data as the second target data, and be stored in the second buffer zone, such as shown in Fig. 5.
Two are just marked in each buffer zone at interval of the first preset time period using algorithm is at least used in the recent period
The data of utilization rate highest first in the data of buffer zone storage, further, above-mentioned two caching in statistics predetermined period
The second data of least number of times are labeled in the data of region storage, it is hard to machinery to be changed to storage as first object data
In disk, such as shown in Fig. 6.
Above-described embodiment needs the data list being maintained in two buffer zones, and needs according to actual conditions adjustment the
The duration of one preset time period and the second preset time period, it is to avoid because the duration configuration of above-mentioned preset time period is improper, cause number
Reading and writing data speed during according to read or write speed less than only with minimum use algorithm in the recent period.Furthermore it is possible to pass through to adjust the first kind
Hard disk proportion in whole hard disks, accordingly adjusts reading and writing data speed.Meanwhile, introduce data read or write speeds it is relatively slow but into
This relatively low Equations of The Second Kind hard disk, so as to find equalization point on the cost and data readwrite performance of hard disk.
Corresponding with preceding method embodiment, the disclosure additionally provides the embodiment of device.
As shown in fig. 7, Fig. 7 is a kind of data processing equipment block diagram of the disclosure according to an exemplary embodiment, institute
Stating device includes：
First determining module 310, be configured as in the data of first kind hard-disc storage determine utilization rate it is minimum first
Target data；
First data processing module 320, after being configured as the first object data Cun Chudao Equations of The Second Kind hard disks, in institute
State and delete the first object data in first kind hard disk；
Wherein, reading and writing data speed of the reading and writing data speed of the first kind hard disk higher than the Equations of The Second Kind hard disk.
As shown in figure 8, another data processing equipment block diagram of Fig. 8 disclosure according to an exemplary embodiment, should
On the basis of the embodiment of earlier figures 7, first determining module 310 includes embodiment：
Determination sub-module 311, is configured as the data of at least one buffer zone storage included by first kind hard disk
The minimum data of middle utilization rate are used as first object data.
As shown in figure 9, another data processing equipment block diagram of Fig. 9 disclosure according to an exemplary embodiment, should
On the basis of the embodiment of earlier figures 8, the determination sub-module 311 includes embodiment：
Indexing unit 3111, is configured as according to algorithm is at least used in the recent period, in the data of each buffer zone storage
The data of utilization rate highest first are marked at interval of the first preset time period；
Statistic unit 3112, is configured as counting labeled in the data of each buffer zone storage in predetermined period
Second data of least number of times；
Determining unit 3113, is configured as being used in the data for storing second data as corresponding buffer zone
The minimum data of rate.
As shown in Figure 10, another data processing equipment block diagram of Figure 10 disclosure according to an exemplary embodiment,
The embodiment also includes on the basis of the embodiment of earlier figures 7：
Second determining module 330, is configured as determining to meet data exchange bar in the data of the first buffer zone storage
Second target data of part；
Second data processing module 340, after being configured as second target data storage to the second buffer zone,
Second target data is deleted in first buffer zone；
Wherein, the utilization rate of the data for being stored in first buffer zone is less than storage in second buffer zone
The utilization rate of data.
Alternatively, the data exchange condition includes：
Number of times is accessed in the second preset time period and reaches preset value.
As shown in figure 11, another data processing equipment block diagram of Figure 11 disclosure according to an exemplary embodiment,
The embodiment also includes on the basis of the embodiment of earlier figures 7：
Data memory module 350, is configured as data of the storage in the first kind hard disk or the Equations of The Second Kind hard disk
Stored using row storage mode.
For device embodiment, because it corresponds essentially to embodiment of the method, so related part is referring to method reality
Apply the part explanation of example.Device embodiment described above is only schematical, wherein being illustrated as separating component
Unit can be or may not be physically separate, the part shown as unit can be or may not be
Physical location, you can with positioned at a place, or can also be distributed on multiple NEs.Can be according to the actual needs
Some or all of module therein is selected to realize the purpose of disclosure scheme.Those of ordinary skill in the art are not paying wound
In the case that the property made is worked, you can to understand and implement.
Accordingly, the disclosure also provides a kind of data processing equipment, including：
Processor；
Memory for storing processor-executable instruction；
Wherein, the processor is configured as：
The minimum first object data of utilization rate are determined in the data of first kind hard-disc storage；
After by the first object data Cun Chudao Equations of The Second Kind hard disks, first mesh is deleted in the first kind hard disk
Mark data；
Wherein, reading and writing data speed of the reading and writing data speed of the first kind hard disk higher than the Equations of The Second Kind hard disk.
As shown in figure 12, Figure 12 is an a kind of structure of the data processing equipment 1200 according to an exemplary embodiment
Schematic diagram.For example, device 1200 may be provided in e-commerce system.Reference picture 12, device 1200 includes processing assembly
1222, it further includes one or more processors, and the memory resource as representated by memory 1232, for storing
Can be by the instruction of the execution of processing assembly 1222, such as application program.The application program stored in memory 1232 can include
It is one or more each correspond to the module of one group of instruction.Additionally, processing assembly 1222 is configured as execute instruction,
To perform above-mentioned data processing method.
Device 1200 can also include that a power supply module 1226 is configured as the power management of performs device 1200, one
Wired or wireless network interface 1250 is configured as device 1200 being connected to network, and input and output (I/O) interface
1258.Device 1200 can operate the operating system in memory 1232, such as Windows ServerTM, Mac based on storage
OS XTM, UnixTM, LinuxTM, FreeBSDTM or similar.
Those skilled in the art will readily occur to its of the disclosure after considering specification and putting into practice invention disclosed herein
Its embodiment.The disclosure is intended to any modification, purposes or the adaptations of the disclosure, these modifications, purposes or
Person's adaptations follow the general principle of the disclosure and including the undocumented common knowledge in the art of the disclosure
Or conventional techniques.Description and embodiments are considered only as exemplary, and the true scope of the disclosure and spirit are by following
Claim point out.
The preferred embodiment of the disclosure is the foregoing is only, is not used to limit the disclosure, all essences in the disclosure
Within god and principle, any modification, equivalent substitution and improvements done etc. should be included within the scope of disclosure protection.
Claims (13)
1. a kind of data processing method, it is characterised in that methods described includes：
The minimum first object data of utilization rate are determined in the data of first kind hard-disc storage；
After by the first object data Cun Chudao Equations of The Second Kind hard disks, the first object number is deleted in the first kind hard disk
According to；
Wherein, reading and writing data speed of the reading and writing data speed of the first kind hard disk higher than the Equations of The Second Kind hard disk.
2. method according to claim 1, it is characterised in that described to determine to use in the data of first kind hard-disc storage
The minimum first object data of rate, including：
Using the minimum data of utilization rate in the data of at least one buffer zone storage included by first kind hard disk as first
Target data.
3. method according to claim 2, it is characterised in that determine included by first kind hard disk in the following ways extremely
The minimum data of utilization rate in the data of few buffer zone storage：
According to algorithm is at least used in the recent period, marked at interval of the first preset time period in the data of each buffer zone storage
The data of utilization rate highest first；
The second data of least number of times are labeled in statistics predetermined period in the data of each buffer zone storage；
The minimum data of utilization rate in the data that second data are stored as corresponding buffer zone.
4. method according to claim 3, it is characterised in that the number of the buffer zone for it is multiple when, also include：
Determination meets the second target data of data exchange condition in the data of the first buffer zone storage；
After by second target data storage to the second buffer zone, second mesh is deleted in first buffer zone
Mark data；
Wherein, the utilization rate of the data for being stored in first buffer zone is less than the data stored in second buffer zone
Utilization rate.
5. method according to claim 4, it is characterised in that the data exchange condition includes：
Number of times is accessed in the second preset time period and reaches preset value.
6. the method according to claim any one of 1-5, it is characterised in that also include：
The data stored in the first kind hard disk or the Equations of The Second Kind hard disk are stored using row storage mode.
7. a kind of data processing equipment, it is characterised in that described device includes：
First determining module, is configured as determining the minimum first object number of utilization rate in the data of first kind hard-disc storage
According to；
First data processing module, after being configured as the first object data Cun Chudao Equations of The Second Kind hard disks, described first
The first object data are deleted in class hard disk；
Wherein, reading and writing data speed of the reading and writing data speed of the first kind hard disk higher than the Equations of The Second Kind hard disk.
8. device according to claim 7, it is characterised in that first determining module includes：
Determination sub-module, is configured as utilization rate in the data of at least one buffer zone storage included by first kind hard disk
Minimum data are used as first object data.
9. device according to claim 8, it is characterised in that the determination sub-module includes：
Indexing unit, is configured as according at least using algorithm in the recent period, at interval of the in the data that each buffer zone stores
One preset time period marks the data of utilization rate highest first；
Statistic unit, is configured as counting in predetermined period labeled least number of times in the data of each buffer zone storage
The second data；
Determining unit, utilization rate is minimum in being configured as the data that second data are stored as corresponding buffer zone
Data.
10. device according to claim 9, it is characterised in that also include：
Second determining module, is configured as determining to meet the second of data exchange condition in the data of the first buffer zone storage
Target data；
Second data processing module, is configured as second target data storage to behind the second buffer zone, described the
Second target data is deleted in one buffer zone；
Wherein, the utilization rate of the data for being stored in first buffer zone is less than the data stored in second buffer zone
Utilization rate.
11. devices according to claim 10, it is characterised in that the data exchange condition includes：
Number of times is accessed in the second preset time period and reaches preset value.
12. device according to claim any one of 7-11, it is characterised in that also include：
Data memory module, is configured as data of the storage in the first kind hard disk or the Equations of The Second Kind hard disk and is deposited using row
Storage mode is stored.
A kind of 13. data processing equipments, it is characterised in that including：
Processor；
Memory for storing processor-executable instruction；
Wherein, the processor is configured as：
The minimum first object data of utilization rate are determined in the data of first kind hard-disc storage；
After by the first object data Cun Chudao Equations of The Second Kind hard disks, the first object number is deleted in the first kind hard disk
According to；
Wherein, reading and writing data speed of the reading and writing data speed of the first kind hard disk higher than the Equations of The Second Kind hard disk.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611153402.0A CN106708435A (en) | 2016-12-14 | 2016-12-14 | Data processing method and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611153402.0A CN106708435A (en) | 2016-12-14 | 2016-12-14 | Data processing method and apparatus |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106708435A true CN106708435A (en) | 2017-05-24 |
Family
ID=58937106
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611153402.0A Pending CN106708435A (en) | 2016-12-14 | 2016-12-14 | Data processing method and apparatus |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106708435A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110045925A (en) * | 2019-03-05 | 2019-07-23 | 北京字节跳动网络技术有限公司 | Data processing method, device, electronic equipment and computer readable storage medium |
| CN110688350A (en) * | 2018-06-19 | 2020-01-14 | 阿里巴巴集团控股有限公司 | Method and device for storing log |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102117248A (en) * | 2011-03-09 | 2011-07-06 | 浪潮(北京)电子信息产业有限公司 | Caching system and method for caching data in caching system |
| US20120054421A1 (en) * | 2010-08-25 | 2012-03-01 | Hitachi, Ltd. | Information device equipped with cache memories, apparatus and program using the same device |
| CN102520885A (en) * | 2011-12-15 | 2012-06-27 | 清华大学 | Data management system for hybrid hard disk |
| CN103927129A (en) * | 2014-03-14 | 2014-07-16 | 汉柏科技有限公司 | Data storage method |
| CN104008075A (en) * | 2014-05-26 | 2014-08-27 | 华中科技大学 | Request processing method of distributed storage system |
| CN104731523A (en) * | 2013-12-24 | 2015-06-24 | 国际商业机器公司 | Method and controller for collaborative management of non-volatile hierarchical storage system |
| CN105205014A (en) * | 2015-09-28 | 2015-12-30 | 北京百度网讯科技有限公司 | Data storage method and device |
-
2016
- 2016-12-14 CN CN201611153402.0A patent/CN106708435A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120054421A1 (en) * | 2010-08-25 | 2012-03-01 | Hitachi, Ltd. | Information device equipped with cache memories, apparatus and program using the same device |
| CN102117248A (en) * | 2011-03-09 | 2011-07-06 | 浪潮(北京)电子信息产业有限公司 | Caching system and method for caching data in caching system |
| CN102520885A (en) * | 2011-12-15 | 2012-06-27 | 清华大学 | Data management system for hybrid hard disk |
| CN104731523A (en) * | 2013-12-24 | 2015-06-24 | 国际商业机器公司 | Method and controller for collaborative management of non-volatile hierarchical storage system |
| CN103927129A (en) * | 2014-03-14 | 2014-07-16 | 汉柏科技有限公司 | Data storage method |
| CN104008075A (en) * | 2014-05-26 | 2014-08-27 | 华中科技大学 | Request processing method of distributed storage system |
| CN105205014A (en) * | 2015-09-28 | 2015-12-30 | 北京百度网讯科技有限公司 | Data storage method and device |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110688350A (en) * | 2018-06-19 | 2020-01-14 | 阿里巴巴集团控股有限公司 | Method and device for storing log |
| CN110688350B (en) * | 2018-06-19 | 2023-05-09 | 阿里巴巴集团控股有限公司 | Method and device for storing logs |
| CN110045925A (en) * | 2019-03-05 | 2019-07-23 | 北京字节跳动网络技术有限公司 | Data processing method, device, electronic equipment and computer readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8793427B2 (en) | Remote memory for virtual machines | |
| JP2020513176A5 (en) | ||
| JP2018506095A (en) | System for efficient processing of transaction requests related to database accounts | |
| EP3640813B1 (en) | Cluster-based random walk method and apparatus | |
| CN106547784A (en) | A kind of data split storage method and device | |
| CN103995674A (en) | Method, device and equipment for processing access requests | |
| US20230205925A1 (en) | Generating hash values for input strings | |
| CN106888381A (en) | A kind of data resource storage method and device | |
| CN105224560A (en) | Data cached lookup method and device | |
| CN106708435A (en) | Data processing method and apparatus | |
| CN107741961A (en) | Full table scan method and device based on Hbase | |
| CN106775470A (en) | A kind of method and system of data storage | |
| CN109542612A (en) | A kind of hot spot keyword acquisition methods, device and server | |
| CN106325974A (en) | Virtualization IO performance optimization method and system | |
| US20190149478A1 (en) | Systems and methods for allocating shared resources in multi-tenant environments | |
| CN107483565B (en) | Service background identification method, proxy server and computer storage medium | |
| CN107844592A (en) | A kind of method and apparatus of query metadata | |
| CN107807793A (en) | The storage of data trnascription isomery and access method in distributed computer storage system | |
| CN107896194A (en) | A kind of method for searching route, device and storage medium | |
| US9406083B2 (en) | Ranked navigation element | |
| CN111177782A (en) | Method and device for extracting distributed data based on big data and storage medium | |
| CN106339183A (en) | Data retrieving method and server | |
| US10558634B2 (en) | Multi-user probabalistic array | |
| CN104298614A (en) | Method for storing data block in memory device and memory device | |
| CN107562533A (en) | A kind of data loading processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170524 |
|
| RJ01 | Rejection of invention patent application after publication |