CN110262756A - For data cached method and apparatus - Google Patents
For data cached method and apparatus Download PDFInfo
- Publication number
- CN110262756A CN110262756A CN201910547977.8A CN201910547977A CN110262756A CN 110262756 A CN110262756 A CN 110262756A CN 201910547977 A CN201910547977 A CN 201910547977A CN 110262756 A CN110262756 A CN 110262756A
- Authority
- CN
- China
- Prior art keywords
- disk
- utilization rate
- value
- data
- space
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
The embodiment of the present disclosure is disclosed for data cached method and apparatus.One specific embodiment of this method includes: to generate data manipulation task according to the data operation request that the first user sends；Internal storage data spills into the disk as temporary space during in response to determining execution data manipulation task, according to the value of pre-set disk utilization rate parameter, determines that disk is limited with the presence or absence of utilization rate；In response to determine disk limited there are utilization rate and the value of the disk space full scale will of disk be it is false, execute following operating procedure: determining whether the capacity of the remaining space of disk is greater than the data volume of the internal storage data of spilling；If it does, determining whether the prediction utilization rate of disk is greater than the value of disk utilization rate parameter；In response to determining that the prediction utilization rate of disk is not more than the value of disk utilization rate parameter, the internal storage data of spilling is cached to disk.This embodiment improves the stability of database.
Description
Technical field
The embodiment of the present disclosure is related to field of computer technology, and in particular to for data cached method and apparatus.
Background technique
Storage medium of the disk as the bottom, is very important for database, when disk space wretched insufficiency
When, mission failure performed by database may be caused, entire database delay machine can be caused what is more.Traditional is directed to disk
Use space solution to the problem is the monitoring alarm added to disk service condition: working as Insufficient disk space, utilization rate mistake
Gao Shihui issues corresponding alarm notification to operation maintenance personnel, is handled by operation maintenance personnel manpower intervention.
In general, the reason of causing Insufficient disk space to alert is broadly divided into 2 classes: 1) true data volume, log storage are super
Limit；2) it is related to polymerizeing the operations such as (aggregation), table association (join), sequence (sort), can be cached when low memory
Data cause disk utilization rate moment up to 100% to disk, and entire cluster is caused to break down.For magnetic caused by first kind reason
The alarm of disk space deficiency, often can be with look-ahead, therefore can avoid by operations such as dilatation, deleting history data.
Alerted for Insufficient disk space caused by the second class reason, often cannot look-ahead, therefore, it is difficult to obtain effectively
It solves.
Summary of the invention
The embodiment of the present disclosure is proposed for data cached method and apparatus.
In a first aspect, the embodiment of the present disclosure provide it is a kind of for data cached method, this method comprises: according to first
The data operation request that user sends generates data manipulation task；During determining the above-mentioned data manipulation task of execution
Internal storage data spills into the disk as temporary space, according to the value of pre-set disk utilization rate parameter, determines above-mentioned
Disk is limited with the presence or absence of utilization rate；In response to the above-mentioned disk of determination, there are utilization rate limitations and the disk space of above-mentioned disk
The value of full scale will is false, executes following operating procedure: determining whether the capacity of the remaining space of above-mentioned disk is greater than spilling
The data volume of internal storage data；If the capacity of the remaining space of above-mentioned disk is greater than the data volume of the internal storage data overflowed, determine
Whether the prediction utilization rate of above-mentioned disk is greater than the value of above-mentioned disk utilization rate parameter, wherein the prediction of above-mentioned disk uses
Rate is that use space, total disk space determine according to the disk of the data volume of the internal storage data of spilling and above-mentioned disk；
It is not more than the value of above-mentioned disk utilization rate parameter in response to the prediction utilization rate of the above-mentioned disk of determination, by the internal storage data of spilling
It is cached to above-mentioned disk.
In some embodiments, aforesaid operations step further include: be greater than in response to the prediction utilization rate of the above-mentioned disk of determination
The value of above-mentioned disk utilization rate parameter, for above-mentioned data manipulation task execution task rolling back action, and by above-mentioned disk
The value of disk space full scale will be set as true.
In some embodiments, aforesaid operations step further include: if the capacity of the remaining space of above-mentioned disk is not more than
The data volume of the internal storage data of spilling, for above-mentioned data manipulation task execution task rolling back action.
In some embodiments, the above method further include: in response to the above-mentioned disk of determination there are utilization rate limit and on
State the disk space of disk full scale will value be it is true, for above-mentioned data manipulation task execution task rolling back action.
In some embodiments, the above method further include: in response to being directed to above-mentioned data manipulation task execution task rollback
Operation, the terminal used by a user of Xiang Shangshu first send miscue information.
In some embodiments, the above method further include: receive the disk utilization rate parameter setting that second user is sent and ask
It asks, wherein above-mentioned disk utilization rate parameter setting request includes the user information and parameter value of second user；According to above-mentioned second
The user information of user, determines whether above-mentioned second user has parameter setting permission；In response to the above-mentioned second user tool of determination
There is parameter setting permission, determines whether above-mentioned parameter value is virtual value；It, will be above-mentioned in response to determining that above-mentioned parameter value is virtual value
Value of the parameter value as above-mentioned disk utilization rate parameter.
Second aspect, the embodiment of the present disclosure provide a kind of for data cached device, and device includes: generation unit,
It is configured to generate data manipulation task according to the data operation request that the first user sends；Determination unit is configured to respond to
In determine execute above-mentioned data manipulation task during internal storage data spill into the disk as temporary space, according to setting in advance
The value for the disk utilization rate parameter set determines that above-mentioned disk is limited with the presence or absence of utilization rate；Execution unit is configured to respond to
In determine above-mentioned disk there are the value of utilization rate limitation and the disk space full scale will of above-mentioned disk be it is false, execute following behaviour
Make step: determining whether the capacity of the remaining space of above-mentioned disk is greater than the data volume of the internal storage data of spilling；If above-mentioned magnetic
The capacity of the remaining space of disk is greater than the data volume of the internal storage data overflowed, determines whether the prediction utilization rate of above-mentioned disk is greater than
The value of above-mentioned disk utilization rate parameter, wherein the prediction utilization rate of above-mentioned disk is the data according to the internal storage data of spilling
Use space, total disk space determine the disk of amount and above-mentioned disk；Prediction in response to the above-mentioned disk of determination uses
Rate is not more than the value of above-mentioned disk utilization rate parameter, and the internal storage data of spilling is cached to above-mentioned disk.
In some embodiments, aforesaid operations step further include: be greater than in response to the prediction utilization rate of the above-mentioned disk of determination
The value of above-mentioned disk utilization rate parameter, for above-mentioned data manipulation task execution task rolling back action, and by above-mentioned disk
The value of disk space full scale will be set as true.
In some embodiments, aforesaid operations step further include: if the capacity of the remaining space of above-mentioned disk is not more than
The data volume of the internal storage data of spilling, for above-mentioned data manipulation task execution task rolling back action.
In some embodiments, above-mentioned apparatus further include: rolling back action execution unit is configured in response to determine above-mentioned
Disk limited there are utilization rate and the value of the disk space full scale will of above-mentioned disk be it is true, for above-mentioned data manipulation task
Execution task rolling back action.
In some embodiments, above-mentioned apparatus further include: transmission unit is configured in response to for above-mentioned data manipulation
Task execution task rolling back action, the terminal used by a user of Xiang Shangshu first send miscue information.
In some embodiments, above-mentioned apparatus further include: receiving unit is configured to receive the disk of second user transmission
Utilization rate parameter setting request, wherein above-mentioned disk utilization rate parameter setting request includes the user information and ginseng of second user
Numerical value；Permission determination unit is configured to the user information according to above-mentioned second user, determines whether above-mentioned second user has
Parameter setting permission；Virtual value determination unit is configured in response to determine that above-mentioned second user has parameter setting permission, really
Determine whether above-mentioned parameter value is virtual value；Assignment unit is configured in response to determine that above-mentioned parameter value is virtual value, will be above-mentioned
Value of the parameter value as above-mentioned disk utilization rate parameter.
The third aspect, the embodiment of the present disclosure provide a kind of server, which includes: one or more processors；
Storage device is stored thereon with one or more programs, when said one or multiple programs are by said one or multiple processors
When execution, so that said one or multiple processors realize the method as described in implementation any in first aspect.
Fourth aspect, the embodiment of the present disclosure provide a kind of computer-readable medium, are stored thereon with computer program,
In, the method as described in implementation any in first aspect is realized when which is executed by processor.
What the embodiment of the present disclosure provided is used for data cached method and apparatus, the data manipulation sent according to the first user
Request generates data manipulation task, if it is determined that internal storage data is spilt into as interim sky during executing data manipulation task
Between disk determine that disk is limited with the presence or absence of utilization rate according to the value of pre-set disk utilization rate parameter.If magnetic
Take inventory and limited in utilization rate, and the value of the disk space of disk full scale will be it is false, then execute following operating procedure: firstly, really
Determine the remaining space of disk capacity whether be greater than spilling internal storage data data volume.If the capacity of the remaining space of disk
Greater than the data volume of the internal storage data of spilling, then further determine that whether the prediction utilization rate of disk is greater than disk utilization rate parameter
Value.If it is not greater, then the internal storage data of spilling is cached to disk, thus taking by setting disk utilization rate parameter
Value, after guaranteeing the internal storage data that disk buffering overflows, the disk utilization rate of disk will not be greater than taking for disk utilization rate parameter
The problem of being worth, and then avoid since internal storage data spilling leads to Insufficient disk space, and then causing database delay machine, to mention
The high stability of database.
Detailed description of the invention
By reading a detailed description of non-restrictive embodiments in the light of the attached drawings below, the disclosure is other
Feature, objects and advantages will become more apparent upon:
Fig. 1 is that one embodiment of the disclosure can be applied to exemplary system architecture figure therein；
Fig. 2 is the flow chart according to one embodiment for data cached method of the disclosure；
Fig. 3 is the schematic diagram according to an application scenarios for data cached method for the disclosure；
Fig. 4 is the flow chart according to another embodiment for data cached method of the disclosure；
Fig. 5 is the structural schematic diagram according to one embodiment for data cached device of the disclosure；
Fig. 6 is adapted for the structural schematic diagram for the computer system for realizing the server of the embodiment of the present disclosure.
Specific embodiment
The disclosure is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched
The specific embodiment stated is used only for explaining related invention, rather than the restriction to the invention.It also should be noted that in order to
Convenient for description, part relevant to related invention is illustrated only in attached drawing.
It should be noted that in the absence of conflict, the feature in embodiment and embodiment in the disclosure can phase
Mutually combination.The disclosure is described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
Fig. 1 is shown can be using the embodiment of the present disclosure for data cached method or for data cached device
Exemplary system architecture 100.
As shown in Figure 1, system architecture 100 may include terminal device 101,102,103, network 104 and server 105.
Network 104 between terminal device 101,102,103 and server 105 to provide the medium of communication link.Network 104 can be with
Including various connection types, such as wired, wireless communication link or fiber optic cables etc..
User can be used terminal device 101,102,103 and be interacted by network 104 with server 105, to receive or send out
Send message etc..Various telecommunication customer end applications can be installed, such as web browser is answered on terminal device 101,102,103
With, shopping class application, searching class application, instant messaging tools, mailbox client, social platform software etc..
Terminal device 101,102,103 can be hardware, be also possible to software.When terminal device 101,102,103 is hard
When part, it can be the various electronic equipments with display screen and information input, including but not limited to smart phone, plate electricity
Brain, pocket computer on knee and desktop computer etc..When terminal device 101,102,103 is software, may be mounted at
In above-mentioned cited electronic equipment.Multiple softwares or software module may be implemented into (such as providing distributed clothes in it
Business), single software or software module also may be implemented into.It is not specifically limited herein.
Server 105 can be to provide the server of various services, such as database server.Database server can be with
It receives the data operation request of user, and corresponding data manipulation task is executed according to data operation request, and by implementing result
Feed back to terminal device 101,102,103.It may include disk in server 105.
It should be noted that server 105 can be hardware, it is also possible to software.It, can when server 105 is hardware
To be implemented as the distributed server cluster that multiple servers form, individual server also may be implemented into.When server 105 is
When software, multiple softwares or software module (such as providing Distributed Services) may be implemented into, also may be implemented into single
Software or software module.It is not specifically limited herein.
It should be understood that the number of terminal device, network and server in Fig. 1 is only schematical.According to realization need
It wants, can have any number of terminal device, network and server.
It should be noted that generally being held by server 105 provided by the embodiment of the present disclosure for data cached method
Row, correspondingly, is generally positioned in server 105 for data cached device.
With continued reference to Fig. 2, the process of one embodiment for data cached method according to the disclosure is shown
200.This is used for data cached method, comprising the following steps:
Step 201, data manipulation task is generated according to the data operation request that the first user sends.
It in the present embodiment, can be with for the executing subject of data cached method (such as server 105 shown in FIG. 1)
Carry out terminal (such as Fig. 1 institute of information input using it from the first user by wired connection mode or radio connection
The terminal device 101 that shows, 102,103) receive data operation request.As an example, data operation request may include that data are new
Increase request, data inquiry request, data update request etc..Executing subject can be generated according to the data operation request received
Data manipulation task, the data manipulation task is for responding data operation request.It is that data are looked into data operation request
For asking request, data manipulation task can be from data base querying data, and return to query result.
Step 202, internal storage data is spilt into as temporary space during executing data manipulation task in response to determination
Disk determine that disk is limited with the presence or absence of utilization rate according to the value of pre-set disk utilization rate parameter.
In the present embodiment, executing subject is during executing data manipulation task, if being related to polymerizeing
(aggregation), the big operation of data volumes such as table association (join), sequence (sort), may result in low memory, at this time
Internal storage data can spill into the disk as temporary space.Here, temporary space can refer to when low memory, for caching
The space of data.If it is determined that internal storage data spills into the disk as temporary space during executing data manipulation task,
Then executing subject can judge that disk is limited with the presence or absence of utilization rate according to the value of pre-set disk utilization rate parameter.This
In, disk utilization rate parameter can be previously provided with inside executing subject, the value range of the disk utilization rate parameter are as follows: 0 arrives
100.When disk utilization rate parameter is 0 or 100, the limitation of diskless utilization rate is indicated；When disk utilization rate parameter is big
In 0 and when less than 100, indicate that there are utilization rate limitations for disk.In this way, the memory during determining execution data manipulation task
When data spill into disk, executing subject can judge disk with the presence or absence of utilization rate according to the value of disk utilization rate parameter
Limitation.
Here, if it is determined that the limitation of diskless utilization rate, then executing subject can be directly by the internal storage data of spilling
Disk is written.
Step 203, in response to determining that disk is limited there are utilization rate and the value of the disk space full scale will of disk is
Vacation executes following operating procedure 2031~2033.
In the present embodiment, disk space full scale will can be set for disk, full scale will can for the disk space
For indicating whether disk space is full.Specifically, full scale will can be set to true (true) or vacation to disk space
(false), when full scale will is set as true to disk space, indicate that disk space has been expired；When disk space has expired traffic sign placement
For fictitious time, indicate that disk space is less than.If it is determined that there are utilization rate limitations for disk, and full scale will is the disk space of disk
Vacation, then executing subject can execute following steps 2031~2033.
Step 2031, determine whether the capacity of the remaining space of disk is greater than the data volume of the internal storage data of spilling.
In the present embodiment, executing subject can determine whether the capacity of the remaining space of disk is greater than the memory number of spilling
According to data volume.
Step 2032, if the capacity of the remaining space of disk is greater than the data volume of the internal storage data overflowed, disk is determined
Prediction utilization rate whether be greater than the value of disk utilization rate parameter.
In the present embodiment, if it is determined that the capacity of the remaining space of disk is greater than the data volume of the internal storage data overflowed,
Then executing subject may further determine that whether the prediction utilization rate of disk is greater than the value of above-mentioned disk utilization rate parameter.Its
In, the prediction utilization rate of disk is according to the disk of the data volume of the internal storage data of spilling and disk use space, disk
What gross space determined.As an example, executing subject can pass through formula: the prediction utilization rate of disk=(disk use space+
The data volume of the internal storage data of spilling)/total disk space, calculate the prediction utilization rate of disk.Here, the prediction utilization rate of disk
After can referring to the internal storage data plus the spilling that will be cached, disk expects utilization rate to be achieved.
Step 2033, it in response to determining that the prediction utilization rate of disk is not more than the value of disk utilization rate parameter, will overflow
Internal storage data be cached to disk.
In the present embodiment, if the prediction utilization rate of disk no more than disk use parameter value, show even if
The internal storage data of spilling is cached to disk, it is contemplated that the utilization rate of disk is also no more than the value of disk utilization rate parameter.At this point,
The internal storage data of spilling can be cached to disk by executing subject.
In some optional implementations of the present embodiment, if aforesaid operations step can also include: the surplus of disk
The capacity of complementary space is not more than the data volume of the internal storage data overflowed, for data manipulation task execution task rolling back action.
In this implementation, if the capacity of the remaining space of disk is not more than the data volume of the internal storage data overflowed,
Then executing subject can be directed to data manipulation task execution task rolling back action.
In some optional implementations of the present embodiment, it is above-mentioned for data cached method can also include: sound
Should in determine disk there are the value of utilization rate limitation and the disk space full scale will of disk be it is true, for data manipulation task
Execution task rolling back action.
In this implementation, if it is determined that disk is there are utilization rate limitation, and the disk space of disk full scale will
Value is very that then executing subject can be directed to data manipulation task execution task rolling back action.
In some optional implementations of the present embodiment, it is above-mentioned for data cached method can also include: sound
Ying Yu is directed to data manipulation task execution task rolling back action, sends miscue information to the first user.
In this implementation, if being directed to data manipulation task execution task rolling back action, executing subject can be incited somebody to action
First terminal used by a user sends miscue information, to prompt the first user disk space use to transfinite.
In some optional implementations of the present embodiment, it is above-mentioned for data cached method can also include following
Content:
Firstly, receiving the disk utilization rate parameter setting request that second user is sent.
In this implementation, executing subject can receive the disk utilization rate that second user is sent by institute's using terminal
Parameter setting request, wherein the request of disk utilization rate parameter setting may include the user information and parameter value of second user.This
In, the user information of second user can include but is not limited to user name, password etc..
Secondly, determining whether second user has parameter setting permission according to the user information of second user.
In this implementation, the user information of manager can be previously stored in executing subject, here, manager can
To refer to the user with disk utilization rate parameter setting permission.In this way, executing subject can be by the user information of second user
It is matched with the user information of the manager prestored, if the user information of second user and the user of the manager prestored believe
Manner of breathing matches (such as identical), then executing subject can determine that second user has parameter setting permission.
Then, in response to determining that second user has parameter setting permission, determine whether parameter value is virtual value.
In this implementation, if it is determined that second user has parameter setting permission, then executing subject can be further
Determine whether the parameter value in the request of disk utilization rate parameter setting is virtual value.Here, the value model of disk utilization rate parameter
It encloses are as follows: 0 to 100.If the parameter value in the request of disk utilization rate parameter setting meets condition: 0≤parameter value≤100, can
To determine parameter value as virtual value.
Finally, in response to determining that parameter value is virtual value, using parameter value as the value of disk utilization rate parameter.
In this implementation, if it is determined that parameter value is virtual value, then executing subject can be by disk utilization rate parameter
Value of the parameter value in request as disk utilization rate parameter is set.
With continued reference to the signal that Fig. 3, Fig. 3 are according to the application scenarios for data cached method of the present embodiment
Figure.In the application scenarios of Fig. 3, the first user sends data manipulation to database server 302 by terminal device 301 and asks
It asks, database server 302 generates data manipulation task according to data operation request.If executing the process of data manipulation task
Middle internal storage data spills into the disk (not shown) as temporary space, and database server 302 can be according to setting in advance
The value 95% for the disk utilization rate parameter set determines that disk is limited with the presence or absence of utilization rate.Make in response to determining that disk exists
Value with rate limitation and the disk space full scale will of disk be it is false, database server 302 can execute following operating procedure:
Firstly, determining whether the capacity of the remaining space of disk is greater than the data volume of the internal storage data of spilling.If the residue of disk is empty
Between capacity be greater than overflow internal storage data data volume, then further determine that whether the prediction utilization rate of disk makes greater than disk
With the value 95% of rate parameter.In response to determining that the prediction utilization rate of disk is not more than the value 95% of disk utilization rate parameter,
The internal storage data of spilling is cached to disk.
The method provided by the above embodiment of the disclosure guarantees disk buffering by the value of setting disk utilization rate parameter
After the internal storage data of spilling, the disk utilization rate of disk will not be greater than the value of disk utilization rate parameter, so avoid by
The problem of leading to Insufficient disk space in internal storage data spilling, and then causing database delay machine, to improve the steady of database
It is qualitative.
With further reference to Fig. 4, it illustrates the processes 400 of another embodiment for data cached method.The use
In the process 400 of data cached method, comprising the following steps:
Step 401, data manipulation task is generated according to the data operation request that the first user sends.
In the present embodiment, step 401 is similar with the step 201 of embodiment illustrated in fig. 2, and details are not described herein again.
Step 402, internal storage data is spilt into as temporary space during executing data manipulation task in response to determination
Disk determine that disk is limited with the presence or absence of utilization rate according to the value of pre-set disk utilization rate parameter.
In the present embodiment, step 402 is similar with the step 202 of embodiment illustrated in fig. 2, and details are not described herein again.
Step 403, in response to determining that disk is limited there are utilization rate and the value of the disk space full scale will of disk is
Vacation executes following operating procedure 4031~4034.
In the present embodiment, if it is determined that disk is there are utilization rate limitation, and the value of the disk space of disk full scale will
It is vacation, then executing subject can execute following operating procedure 4031~4034.
Step 4031, determine whether the capacity of the remaining space of disk is greater than the data volume of the internal storage data of spilling.
In the present embodiment, step 4031 is similar with the step 2031 of embodiment illustrated in fig. 2, and details are not described herein again.
Step 4032, if the capacity of the remaining space of disk is greater than the data volume of the internal storage data overflowed, disk is determined
Prediction utilization rate whether be greater than the value of disk utilization rate parameter.
In the present embodiment, step 4032 is similar with the step 2032 of embodiment illustrated in fig. 2, and details are not described herein again.
Step 4033, it in response to determining that the prediction utilization rate of disk is not more than the value of disk utilization rate parameter, will overflow
Internal storage data be cached to disk.
In the present embodiment, step 4033 is similar with the step 2033 of embodiment illustrated in fig. 2, and details are not described herein again.
Step 4034, in response to determining that the prediction utilization rate of disk is greater than the value of disk utilization rate parameter, for number
Task rolling back action is executed according to operation task, and sets true for the value of the disk space of disk full scale will.
In this implementation, the prediction utilization rate of disk is greater than the value of disk utilization rate parameter, then shows if will
The internal storage data of spilling is cached to disk, then the utilization rate of estimated disk will be greater than the value of disk utilization rate parameter, this feelings
Condition may result in disk space wretched insufficiency.Therefore, executing subject can be grasped for the task rollback of data manipulation task execution
Make.Meanwhile executing subject can set true for the value of the disk space of disk full scale will.
Figure 4, it is seen that being used for data cached method in the present embodiment compared with the corresponding embodiment of Fig. 2
Process 400 highlight disk prediction utilization rate be greater than disk utilization rate parameter value in the case where, for data manipulation
Task execution task rolling back action, and genuine step is set by the value of the disk space of disk full scale will.This reality as a result,
The scheme for applying example description can avoid leading to Insufficient disk space, Jin Eryin since internal storage data overflows by task rolling back action
The problem of playing database delay machine, to improve the stability of database.Meanwhile by by the disk space of disk full scale will
Value be set as very, can to avoid next time execute data manipulation task when carry out again once cache attempt and waste time.
With further reference to Fig. 5, as the realization to method shown in above-mentioned each figure, present disclose provides one kind for caching number
According to device one embodiment, the Installation practice is corresponding with embodiment of the method shown in Fig. 2, which can specifically answer
For in various electronic equipments.
As shown in figure 5, the present embodiment includes: generation unit 501, determination unit 502 for data cached device 500
With execution unit 503.Wherein, generation unit 501 is configured to generate data according to the data operation request that the first user sends
Operation task；Determination unit 502 is configured in response to internal storage data during determining the above-mentioned data manipulation task of execution and overflows
The disk as temporary space is arrived out, according to the value of pre-set disk utilization rate parameter, determines whether above-mentioned disk is deposited
It is limited in utilization rate；Execution unit 503 is configured in response to determine above-mentioned disk, and there are utilization rate limitation and above-mentioned disks
Disk space full scale will value be it is false, execute following operating procedure: determine the remaining space of above-mentioned disk capacity whether
Greater than the data volume of the internal storage data of spilling；If the capacity of the remaining space of above-mentioned disk is greater than the number of the internal storage data overflowed
According to amount, determine whether the prediction utilization rate of above-mentioned disk is greater than the value of above-mentioned disk utilization rate parameter, wherein above-mentioned disk
Prediction utilization rate is according to the disk of the data volume of the internal storage data of spilling and above-mentioned disk use space, total disk space
Determining；It is not more than the value of above-mentioned disk utilization rate parameter in response to the prediction utilization rate of the above-mentioned disk of determination, by spilling
Internal storage data is cached to above-mentioned disk.
In the present embodiment, for the generation unit 501, determination unit 502 and execution unit of data cached device 500
503 specific processing and its brought technical effect can be respectively with reference to step 201, step 202 and steps in Fig. 2 corresponding embodiment
Rapid 203 related description, details are not described herein.
In some optional implementations of the present embodiment, aforesaid operations step further include: in response to the above-mentioned magnetic of determination
The prediction utilization rate of disk is greater than the value of above-mentioned disk utilization rate parameter, grasps for above-mentioned data manipulation task execution task rollback
Make, and sets true for the value of the disk space of above-mentioned disk full scale will.
In some optional implementations of the present embodiment, aforesaid operations step further include: if above-mentioned disk is surplus
The capacity of complementary space is grasped no more than the data volume of the internal storage data overflowed for above-mentioned data manipulation task execution task rollback
Make.
In some optional implementations of the present embodiment, above-mentioned apparatus 500 further include: rolling back action execution unit
(not shown) is configured in response to determine above-mentioned disk there are the disk space of utilization rate limitation and above-mentioned disk
The value of full scale will is very, for above-mentioned data manipulation task execution task rolling back action.
In some optional implementations of the present embodiment, above-mentioned apparatus 500 further include: transmission unit (does not show in figure
Out), it is configured in response to for above-mentioned data manipulation task execution task rolling back action, Xiang Shangshu first is used by a user
Terminal sends miscue information.
In some optional implementations of the present embodiment, above-mentioned apparatus 500 further include: receiving unit (does not show in figure
Out), it is configured to receive the disk utilization rate parameter setting request of second user transmission, wherein above-mentioned disk utilization rate parameter
The user information and parameter value that request includes second user are set；Permission determination unit (not shown), is configured to basis
The user information of above-mentioned second user, determines whether above-mentioned second user has parameter setting permission；Virtual value determination unit (figure
In be not shown), be configured in response to determine that above-mentioned second user has parameter setting permission, determine above-mentioned parameter value whether be
Virtual value；Assignment unit (not shown) is configured in response to determine that above-mentioned parameter value is virtual value, by above-mentioned parameter value
Value as above-mentioned disk utilization rate parameter.
Below with reference to Fig. 6, it illustrates the electronic equipment that is suitable for being used to realize embodiment of the disclosure, (example is as shown in figure 1
Server) 600 structural schematic diagram.Server shown in Fig. 6 is only an example, should not be to the function of embodiment of the disclosure
Any restrictions can be brought with use scope.
As shown in fig. 6, electronic equipment 600 may include processing unit (such as central processing unit, graphics processor etc.)
601, random access can be loaded into according to the program being stored in read-only memory (ROM) 602 or from storage device 608
Program in memory (RAM) 603 and execute various movements appropriate and processing.In RAM 603, it is also stored with electronic equipment
Various programs and data needed for 600 operations.Processing unit 601, ROM 602 and RAM 603 pass through the phase each other of bus 604
Even.Input/output (I/O) interface 605 is also connected to bus 604.
In general, following device can connect to I/O interface 605: including such as touch screen, touch tablet, keyboard, mouse, taking the photograph
As the input unit 606 of head, microphone, accelerometer, gyroscope etc.；Including such as liquid crystal display (LCD), loudspeaker, vibration
The output device 607 of dynamic device etc.；Storage device 608 including such as tape, hard disk etc.；And communication device 609.Communication device
609, which can permit electronic equipment 600, is wirelessly or non-wirelessly communicated with other equipment to exchange data.Although Fig. 6 shows tool
There is the electronic equipment 600 of various devices, it should be understood that being not required for implementing or having all devices shown.It can be with
Alternatively implement or have more or fewer devices.Each box shown in Fig. 6 can represent a device, can also root
According to needing to represent multiple devices.
Particularly, in accordance with an embodiment of the present disclosure, it may be implemented as computer above with reference to the process of flow chart description
Software program.For example, embodiment of the disclosure includes a kind of computer program product comprising be carried on computer-readable medium
On computer program, which includes the program code for method shown in execution flow chart.In such reality
It applies in example, which can be downloaded and installed from network by communication device 609, or from storage device 608
It is mounted, or is mounted from ROM 602.When the computer program is executed by processing unit 601, the implementation of the disclosure is executed
The above-mentioned function of being limited in the method for example.
It is situated between it should be noted that computer-readable medium described in embodiment of the disclosure can be computer-readable signal
Matter or computer readable storage medium either the two any combination.Computer readable storage medium for example can be with
System, device or the device of --- but being not limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor, or it is any more than
Combination.The more specific example of computer readable storage medium can include but is not limited to: have one or more conducting wires
Electrical connection, portable computer diskette, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type are programmable
Read-only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic are deposited
Memory device or above-mentioned any appropriate combination.In embodiment of the disclosure, computer readable storage medium, which can be, appoints
What include or the tangible medium of storage program that the program can be commanded execution system, device or device use or and its
It is used in combination.And in embodiment of the disclosure, computer-readable signal media may include in a base band or as carrier wave
The data-signal that a part is propagated, wherein carrying computer-readable program code.The data-signal of this propagation can be adopted
With diversified forms, including but not limited to electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal is situated between
Matter can also be any computer-readable medium other than computer readable storage medium, which can be with
It sends, propagate or transmits for by the use of instruction execution system, device or device or program in connection.Meter
The program code for including on calculation machine readable medium can transmit with any suitable medium, including but not limited to: electric wire, optical cable,
RF (radio frequency) etc. or above-mentioned any appropriate combination.
Above-mentioned computer-readable medium can be included in above-mentioned electronic equipment；It is also possible to individualism, and not
It is fitted into the electronic equipment.Above-mentioned computer-readable medium carries one or more program, when said one or more
When a program is executed by the electronic equipment, so that the electronic equipment: generating number according to the data operation request that the first user sends
According to operation task；Internal storage data is spilt into as temporary space during in response to determining the above-mentioned data manipulation task of execution
Disk determines that above-mentioned disk is limited with the presence or absence of utilization rate according to the value of pre-set disk utilization rate parameter；In response to
Determine above-mentioned disk there are the value of utilization rate limitation and the disk space full scale will of above-mentioned disk be it is false, execute following operation
Step: determine whether the capacity of the remaining space of above-mentioned disk is greater than the data volume of the internal storage data of spilling；If above-mentioned disk
Remaining space capacity be greater than overflow internal storage data data volume, determine whether the prediction utilization rate of above-mentioned disk is greater than
State the value of disk utilization rate parameter, wherein the prediction utilization rate of above-mentioned disk is the data volume according to the internal storage data of spilling
And use space, total disk space determine the disk of above-mentioned disk；In response to the prediction utilization rate of the above-mentioned disk of determination
No more than the value of above-mentioned disk utilization rate parameter, the internal storage data of spilling is cached to above-mentioned disk.
The behaviour for executing embodiment of the disclosure can be write with one or more programming languages or combinations thereof
The computer program code of work, described program design language include object oriented program language-such as Java,
Smalltalk, C++ further include conventional procedural programming language-such as " C " language or similar program design language
Speech.Program code can be executed fully on the user computer, partly be executed on the user computer, as an independence
Software package execute, part on the user computer part execute on the remote computer or completely in remote computer or
It is executed on server.In situations involving remote computers, remote computer can pass through the network of any kind --- packet
It includes local area network (LAN) or wide area network (WAN)-is connected to subscriber computer, or, it may be connected to outer computer (such as benefit
It is connected with ISP by internet).
Flow chart and block diagram in attached drawing are illustrated according to the system of the various embodiments of the disclosure, method and computer journey
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
A part of one module, program segment or code of table, a part of the module, program segment or code include one or more use
The executable instruction of the logic function as defined in realizing.It should also be noted that in some implementations as replacements, being marked in box
The function of note can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are actually
It can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it to infuse
Meaning, the combination of each box in block diagram and or flow chart and the box in block diagram and or flow chart can be with holding
The dedicated hardware based system of functions or operations as defined in row is realized, or can use specialized hardware and computer instruction
Combination realize.
Being described in unit involved in embodiment of the disclosure can be realized by way of software, can also be passed through
The mode of hardware is realized.Described unit also can be set in the processor, for example, can be described as: a kind of processor
Including generation unit, determination unit and execution unit.Wherein, the title of these units is not constituted under certain conditions to the list
The restriction of member itself, for example, generation unit is also described as " generating number according to the data operation request that the first user sends
According to the unit of operation task ".
Above description is only the preferred embodiment of the disclosure and the explanation to institute's application technology principle.Those skilled in the art
Member it should be appreciated that embodiment of the disclosure involved in invention scope, however it is not limited to the specific combination of above-mentioned technical characteristic and
At technical solution, while should also cover do not depart from foregoing invention design in the case where, by above-mentioned technical characteristic or its be equal
Feature carries out any combination and other technical solutions for being formed.Such as disclosed in features described above and embodiment of the disclosure (but
It is not limited to) technical characteristic with similar functions is replaced mutually and the technical solution that is formed.
Claims (14)
1. a kind of for data cached method, comprising:
Data manipulation task is generated according to the data operation request that the first user sends；
Internal storage data spills into the disk as temporary space, root during in response to determining the execution data manipulation task
According to the value of pre-set disk utilization rate parameter, determine that the disk is limited with the presence or absence of utilization rate；
Limited in response to the determination disk there are utilization rate and the value of the disk space full scale will of the disk be it is false, hold
The following operating procedure of row: determine whether the capacity of the remaining space of the disk is greater than the data volume of the internal storage data of spilling；Such as
The capacity of the remaining space of disk described in fruit is greater than the data volume of the internal storage data overflowed, determines the prediction utilization rate of the disk
Whether the value of the disk utilization rate parameter is greater than, wherein the prediction utilization rate of the disk is the memory number according to spilling
According to data volume and the disk disk use space, total disk space determine；In response to the determination disk
It predicts that utilization rate is not more than the value of the disk utilization rate parameter, the internal storage data of spilling is cached to the disk.
2. according to the method described in claim 1, wherein, the operating procedure further include:
It is greater than the value of the disk utilization rate parameter in response to the prediction utilization rate of the determination disk, is grasped for the data
Make task execution task rolling back action, and sets true for the value of the disk space of disk full scale will.
3. according to the method described in claim 1, wherein, the operating procedure further include:
If the capacity of the remaining space of the disk is no more than the data volume of the internal storage data overflowed, for the data manipulation
Task execution task rolling back action.
4. according to the method described in claim 1, wherein, the method also includes:
It is limited in response to the determination disk there are utilization rate and the value of the disk space full scale will of the disk is true, needle
To the data manipulation task execution task rolling back action.
5. according to any method of claim 2-4, wherein the method also includes:
In response to being directed to the data manipulation task execution task rolling back action, the terminal used by a user of Xiang Suoshu first is sent
Miscue information.
6. according to the method described in claim 1, wherein, the method also includes:
Receive the disk utilization rate parameter setting request that second user is sent, wherein the disk utilization rate parameter setting request
User information and parameter value including second user；
According to the user information of the second user, determine whether the second user has parameter setting permission；
There is parameter setting permission in response to the determination second user, determine whether the parameter value is virtual value；
It is virtual value in response to the determination parameter value, using the parameter value as the value of the disk utilization rate parameter.
7. a kind of for data cached device, comprising:
Generation unit is configured to generate data manipulation task according to the data operation request that the first user sends；
Determination unit is configured in response to internal storage data during determining the execution data manipulation task and spills into conduct
The disk of temporary space determines the disk with the presence or absence of utilization rate according to the value of pre-set disk utilization rate parameter
Limitation；
Execution unit is configured in response to determine the disk there are the disk space of utilization rate limitation and the disk
The value of full scale will is vacation, executes following operating procedure: determining whether the capacity of the remaining space of the disk is greater than the interior of spilling
The data volume of deposit data；If the capacity of the remaining space of the disk is greater than the data volume of the internal storage data overflowed, institute is determined
Whether the prediction utilization rate for stating disk is greater than the value of the disk utilization rate parameter, wherein the prediction utilization rate of the disk
It is that use space, total disk space determine according to the disk of the data volume of the internal storage data of spilling and the disk；It rings
The value of the disk utilization rate parameter should be not more than in determining the prediction utilization rate of the disk, the internal storage data of spilling is delayed
It is stored to the disk.
8. device according to claim 7, wherein the operating procedure further include:
It is greater than the value of the disk utilization rate parameter in response to the prediction utilization rate of the determination disk, is grasped for the data
Make task execution task rolling back action, and sets true for the value of the disk space of disk full scale will.
9. device according to claim 7, wherein the operating procedure further include:
If the capacity of the remaining space of the disk is no more than the data volume of the internal storage data overflowed, for the data manipulation
Task execution task rolling back action.
10. device according to claim 7, wherein described device further include:
Rolling back action execution unit, being configured in response to determine the disk, there are utilization rate limitation and the magnetic of the disk
The value of disk space full scale will be it is true, for the data manipulation task execution task rolling back action.
11. according to any method of claim 8-10, wherein described device further include:
Transmission unit is configured in response to for the data manipulation task execution task rolling back action, and Xiang Suoshu first is used
Terminal used in family sends miscue information.
12. device according to claim 7, wherein described device further include:
Receiving unit is configured to receive the disk utilization rate parameter setting request of second user transmission, wherein the disk makes
User information and parameter value including second user are requested with rate parameter setting；
Permission determination unit is configured to the user information according to the second user, determines whether the second user has
Parameter setting permission；
Virtual value determination unit is configured in response to determine that the second user has parameter setting permission, determines the ginseng
Whether numerical value is virtual value；
Assignment unit is configured in response to determine that the parameter value is virtual value, make the parameter value as the disk
With the value of rate parameter.
13. a kind of server, comprising:
One or more processors；
Storage device is stored thereon with one or more programs,
When one or more of programs are executed by one or more of processors, so that one or more of processors are real
Now such as method as claimed in any one of claims 1 to 6.
14. a kind of computer-readable medium, is stored thereon with computer program, wherein real when described program is executed by processor
Now such as method as claimed in any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910547977.8A CN110262756B (en) | 2019-06-24 | 2019-06-24 | Method and device for caching data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910547977.8A CN110262756B (en) | 2019-06-24 | 2019-06-24 | Method and device for caching data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110262756A true CN110262756A (en) | 2019-09-20 |
| CN110262756B CN110262756B (en) | 2023-03-28 |
Family
ID=67920714
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910547977.8A Active CN110262756B (en) | 2019-06-24 | 2019-06-24 | Method and device for caching data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110262756B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105868025A (en) * | 2016-03-30 | 2016-08-17 | 华中科技大学 | System for settling fierce competition of memory resources in big data processing system |
| CN107155081A (en) * | 2017-05-15 | 2017-09-12 | 青岛海信电器股份有限公司 | Method of program recording and device |
| US20170293468A1 (en) * | 2016-04-10 | 2017-10-12 | International Business Machines Corporation | Efficient handling of sort payload in a column organized relational database |
| CN107992432A (en) * | 2017-11-28 | 2018-05-04 | 福建中金在线信息科技有限公司 | The method and terminal device of a kind of data buffer storage |
| CN109831508A (en) * | 2019-02-15 | 2019-05-31 | 网宿科技股份有限公司 | A kind of caching method and equipment, storage medium |
| CN109916022A (en) * | 2017-12-13 | 2019-06-21 | 广东美的制冷设备有限公司 | Control method, device and the computer readable storage medium of air regulator |
-
2019
- 2019-06-24 CN CN201910547977.8A patent/CN110262756B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105868025A (en) * | 2016-03-30 | 2016-08-17 | 华中科技大学 | System for settling fierce competition of memory resources in big data processing system |
| US20170293468A1 (en) * | 2016-04-10 | 2017-10-12 | International Business Machines Corporation | Efficient handling of sort payload in a column organized relational database |
| CN107155081A (en) * | 2017-05-15 | 2017-09-12 | 青岛海信电器股份有限公司 | Method of program recording and device |
| CN107992432A (en) * | 2017-11-28 | 2018-05-04 | 福建中金在线信息科技有限公司 | The method and terminal device of a kind of data buffer storage |
| CN109916022A (en) * | 2017-12-13 | 2019-06-21 | 广东美的制冷设备有限公司 | Control method, device and the computer readable storage medium of air regulator |
| CN109831508A (en) * | 2019-02-15 | 2019-05-31 | 网宿科技股份有限公司 | A kind of caching method and equipment, storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 吴恩慈: "广播机制解决Shuffle过程数据倾斜的方法", 《计算机系统应用》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110262756B (en) | 2023-03-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109299348A (en) | A kind of data query method, apparatus, electronic equipment and storage medium | |
| CN109558246A (en) | A kind of load-balancing method, device, electronic equipment and storage medium | |
| CN109936635A (en) | Load-balancing method and device | |
| CN109408205A (en) | Method for scheduling task and device based on hadoop cluster | |
| CN110391938A (en) | Method and apparatus for deployment services | |
| CN109257320A (en) | Message storage method and device | |
| CN109710859A (en) | Data query method and apparatus | |
| CN109657174A (en) | Method and apparatus for more new data | |
| CN109446309A (en) | Problem feedback method and device | |
| CN110334109A (en) | Relational database data query method, system, medium and electronic equipment | |
| CN111427701A (en) | Workflow engine system and business processing method | |
| CN109525855A (en) | Method and apparatus for handling information | |
| CN109885593A (en) | Method and apparatus for handling information | |
| CN110365745A (en) | Data processing system, method, apparatus, server and computer-readable medium | |
| CN109656799A (en) | Test method and device | |
| CN109491721A (en) | Method and apparatus for load information | |
| CN109218041A (en) | Request processing method and device for server system | |
| CN110223179A (en) | The data processing method of fund, device, system, medium | |
| CN109933508A (en) | Method and apparatus for sending information | |
| CN109885564A (en) | Method and apparatus for sending information | |
| CN113010238A (en) | Permission determination method, device and system for micro application call interface | |
| CN108763881A (en) | Method and apparatus for controlling user right | |
| CN109150928A (en) | Method and apparatus for handling request | |
| CN115022106B (en) | Group information processing method, device, equipment and medium | |
| CN110262756A (en) | For data cached method and apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |