CN108984589A - A kind of method for writing data and server - Google Patents
A kind of method for writing data and server Download PDFInfo
- Publication number
- CN108984589A CN108984589A CN201810532139.9A CN201810532139A CN108984589A CN 108984589 A CN108984589 A CN 108984589A CN 201810532139 A CN201810532139 A CN 201810532139A CN 108984589 A CN108984589 A CN 108984589A
- Authority
- CN
- China
- Prior art keywords
- data
- distribution type
- type file
- file storing
- library
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of method for writing data server and computer readable storage mediums, by at least one performance indicator for monitoring distribution type file storing data library, and when performance indicator reaches pre-set level threshold value, Redis service is enabled, in the data write-in Redis that the terminal received is sent.In this way, server is in distribution type file storing data library scarce capacity, data can be written by Redis, the distribution type file storing data library of scarce capacity is got around, to ensure the write performance of server, the concurrency of server side ensure that, while the data for also ensuring that server can in time send terminal perform corresponding processing, corresponding data is returned into terminal, guarantees the smoothness of the application of user side.
Description
Technical field
The present invention relates to field of terminal technology, more specifically to a kind of method for writing data and server.
Background technique
With the continuous increase of user volume, the server side of application program all employs greatly distribution type file storing data library
(MongoDB) storage or transfer of high concurrent data are carried out.But the number received with distribution type file storing data library
According to the continuous increase of amount, influenced by distribution type file storing data library self-ability, it can in distribution type file storing data library
There can be the problems such as data can not be recycled from memory in time, which results in server write performances degradation occurs, influences
The concurrency of server side, while also causing server that can not handle in time the interface requests of user, to cause user side
Using Caton.
Summary of the invention
The technical problem to be solved in the present invention is that: the data volume received with distribution type file storing data library is not
Disconnected to increase, the problems such as can not recycling from memory in time in distribution type file storing data library there may be data, this is resulted in
There is degradation in server write performance, influences the concurrency of server side, while server also being caused not handle in time
The interface requests of user, to cause the application Caton of user side.For the technical problem, the present invention provides a kind of data write-in
Method and server.
In order to solve the above technical problems, the present invention provides a kind of method for writing data, the method for writing data includes:
Monitor at least one performance indicator in distribution type file storing data library；
When the performance indicator reaches pre-set level threshold value, Redis (reaffirming) service is enabled；
The Redis service includes:
Receive the data that terminal is sent；
The data received are written in Redis.
Optionally, the performance indicator includes at least one of data write performance, data record rate.
Optionally, in distribution type file storing data library be monitored performance indicator at least there are two types of；
It is described when the performance indicator reaches pre-set level threshold value, enable Redis service include: in any one a performance
When index reaches the performance indicator corresponding pre-set level threshold value, Redis service is enabled.
Optionally, after enabling Redis service, further includes:
Determine the non-thermal point data in distribution type file storing data library；
By the non-hot data compression；
It will be in the compressed non-thermal point data write-in disk.
Optionally, the non-hot data in determination distribution type file storing data library include: by with lower section
At least one of formula determines the non-thermal point data in distribution type file storing data library:
Mode one: the reading times of each data in distribution type file storing data library are detected；Reading times are small
It is determined as non-thermal point data in the data of the first reading times threshold value；
Mode two: the corresponding data of each data storage areas detected in distribution type file storing data library are always read
Number；The total reading times of the data are determined as less than all data in the data storage areas of the second reading times threshold value
Non-thermal point data；
Mode three: the corresponding data of all types of data in distribution type file storing data library are detected and are always read time
Number；The all types of data that the total reading times of the data are less than third reading times threshold value are determined as non-thermal point data.
Optionally, the first reading times threshold value is the reading of each data in distribution type file storing data library
The average value of number；
The second reading times threshold value is that each data storage areas is corresponding in distribution type file storing data library
The average value of the total reading times of data；
The third reading times threshold value is the corresponding data of data all types of in distribution type file storing data library
The average value of total reading times.
Optionally, after described by the compressed non-thermal point data write-in disk, further includes:
It will be in the data write-in distribution type file storing data library in the Redis.
Optionally, described to include: in the compressed non-thermal point data write-in disk
If size of data of the vacant memory space of the disk less than the compressed non-thermal point data or the magnetic
The vacant memory space of disk is less than preset spare space threshold value, applies for disk service automatically；The disk service includes for institute
It states disk and distributes bigger memory space；
In the disk after the compressed non-hot data compression write-in to have been applied to the disk service.
Optionally, when described by the non-hot data compression, further includes: establish the index of the non-thermal point data；
It is described by the compressed non-thermal point data write-in disk after, further includes:
When receiving the read requests to the non-thermal point data, according to the index of the non-thermal point data in disk
Find compressed non-thermal point data；
By the compressed non-hot data decompression, it is requested to return to read requests described in the non-thermal point data
Data.
Further, the present invention also provides a kind of server, the server includes processor, memory and communication
Bus；
The communication bus is for realizing the connection communication between the processor and memory；
The processor is for executing one or more program stored in the memory, to realize any of the above-described kind
The step of method for writing data.
Further, the present invention also provides a kind of computer readable storage medium, the computer readable storage mediums
It is stored with one or more program, one or more of programs can be executed by one or more processor, to realize
The step of any of the above-described kind of method for writing data.
Beneficial effect
The present invention provides a kind of method for writing data server and computer readable storage medium, for now with distribution
The continuous increase for the data volume that formula document storing data library receives, in distribution type file storing data library there may be data without
Situations such as method is recycled from memory in time results in server write performance and degradation occurs, influence the concurrency of server side,
Also cause server that can not handle in time the interface requests of user simultaneously, so that the problem of causing the application Caton of user side, leads to
At least one performance indicator in monitoring distribution type file storing data library is crossed, and when performance indicator reaches pre-set level threshold value,
Redis service is enabled, in the data write-in Redis that the terminal received is sent.In this way, server is stored in distribution type file
When database capability deficiency, data can be written by Redis, get around the distribution type file storing data of scarce capacity
Library ensure that the concurrency of server side to ensure the write performance of server, while also ensure that server can be timely
The data sent to terminal perform corresponding processing, and corresponding data is returned to terminal, guarantees the smoothness of the application of user side.This
Outside, data are written by Redis in distribution type file storing data library scarce capacity for server, also give distribution type file
Storing data library reserves the time for carrying out recovery capability.
Detailed description of the invention
Present invention will be further explained below with reference to the attached drawings and examples, in attached drawing:
The structural schematic diagram of Fig. 1 optional server of each embodiment to realize the present invention；
Fig. 2 is a kind of method for writing data basic flow chart that first embodiment of the invention provides；
Fig. 3 is a kind of flow chart for Redis service that first embodiment of the invention provides；
Fig. 4 is the data interaction schematic diagram between a kind of terminal and server that first embodiment of the invention provides；
Fig. 5 is the data interaction schematic diagram between another terminal and server that first embodiment of the invention provides；
Fig. 6 is a kind of non-hot flow chart of data processing figure that first embodiment of the invention provides；
Fig. 7 is a kind of flow diagram for method for writing data that second embodiment of the invention provides；
Fig. 8 is a kind of server architecture schematic diagram that third embodiment of the invention provides.
Specific embodiment
It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not intended to limit the present invention.
As shown in Figure 1, the structural schematic diagram of each embodiment one optional server to realize the present invention, the server
It includes at least: input and output (IO) bus 11, processor 12, memory 13, memory 14 and communication device 15.Wherein,
Input and output (IO) bus 11 respectively with itself belonging to server other components (processor 12, memory 13,
Memory 14 and communication device 15) connection, and transmission lines are provided for other components.
Processor 12 usually controls the overall operation of the server belonging to itself.For example, processor 12 execute calculate and really
The operation such as recognize.Wherein, processor 12 can be central processing unit (CPU).
Communication device 15, generally includes one or more components, allows server and wireless communication system belonging to itself
Radio communication between system or network.
The software code that 13 storage processor of memory is readable, processor is executable, it includes be used for control processor 12
Execute the instruction (i.e. software execution function) of functions described herein.
Based on above-mentioned server hardware structure, each embodiment of the method for the present invention is proposed.
First embodiment
For the continuous increase for solving the data volume received now with distribution type file storing data library, distribution type file
Situations such as can not recycling from memory in time in storing data library there may be data, results in server write performance and occurs seriously
Decline, influences the concurrency of server side, while also causing server that can not handle in time the interface requests of user, to cause
The problem of application Caton of user side, present embodiments provide a kind of method for writing data.Referring to fig. 2, Fig. 2 mentions for the present embodiment
The method for writing data basic flow chart of confession, comprising:
S201: at least one performance indicator in monitoring distribution type file storing data library；
In the present embodiment, performance indicator includes at least one of data write performance, data record rate.The present embodiment
Middle data write performance can by the writing speed in distribution type file storing data library, data currently writable amount etc. come
It is embodied.Correspondingly, data record rate can data record duration by distribution type file storing data library, data record
Amount etc. is embodied.
It should be noted that data record referred to deletes data from memory in distribution type file storing data library
It removes.When deleting data from memory, data can be marked in server, and after a certain time delete the data being marked
It removes.Therefore, the data that data record duration described in the present embodiment refers to are from this duration for being tagged to deletion, the present embodiment
Described in the data volume deleted from memory in certain time length section that refers to of data record amount.
It should be understood that when the performance indicator in monitoring server distribution type file storing data library, monitored performance
The type of index is to predefine well.The a certain performance that server can only detect distribution type file storing data library refers to
Mark, also can detecte certain several performance indicator in distribution type file storing data library.For example, server can only monitoring data return
Yield, can also simultaneously monitoring data write performance and the data rate of recovery.Specifically monitoring any or which kind performance indicator can
To be preselected by related management personnel.
It is to be further understood that when the performance indicator of monitoring server includes data write performance, the practical inspection of server
What is surveyed is the specific targets of writing speed and/or the data currently embodiment such as writable amount data write performance.For example, service
Device is in monitoring data write performance, the writing speed sum number that can be distribution type file storing data library data of actual monitoring
According to current writable amount.
Correspondingly, when the performance indicator of monitoring server includes data record rate, server it is actually detected be data
Recycle the specific targets that duration and/or data record amount etc. embody data record rate.For example, server is in monitoring data write-in property
When energy, actual monitoring can be the corresponding data record duration in distribution type file storing data library and/or data record amount.
S202: when performance indicator reaches pre-set level threshold value, Redis service is enabled.
It should be understood that metrics-thresholds in the present embodiment can be by related management personnel based on practical experience or greatly
It is next preset to measure examination.Preset metrics-thresholds can be the critical value of each performance indicator in the present embodiment, i.e., super
Cross threshold value will lead to the ability rapid decrease in distribution type file storing data library later, influence user experience.
In the present embodiment, preset metrics-thresholds should be corresponding with the selected performance indicator to be monitored
's.For example, corresponding preset metrics-thresholds should when the performance indicator to be monitored includes data write performance
Including data write performance threshold value.In the present embodiment, since data write performance is by distribution type file storing data library
Writing speed, currently writable amount etc. is embodied data, therefore data write performance threshold value should be correspondence
The threshold value of the specific targets of data write performance is embodied, which can be one or multiple, data write performance threshold
The threshold number of value is determined by the number of the specific targets of the embodiment data write performance of actual monitoring.For example, server is supervised
Measured data write performance is that currently writable amount is realized by monitoring data, then the data write performance threshold being correspondingly arranged
The value as writable amount threshold value of data, only one.In another example monitoring server data write performance is write by monitoring data
Entering speed and data, currently writable amount is realized, then the data write performance threshold value that is correspondingly arranged is data write-in speed
Threshold value and the writable amount threshold value of data are spent, there are two.
Correspondingly, in the present embodiment, since data record rate is returned by the data in distribution type file storing data library
Time receiving length, data record amount etc. embodied, therefore data record rate threshold value should be the corresponding data record rate that embodies
The threshold value of specific targets, the threshold value can be for one or multiple, and the threshold number of data record rate threshold value is by actually supervising
The numbers of the specific targets of the embodiment data record rate of survey determines.For example, monitoring server data record rate is to pass through monitoring
Data record duration realizes, then the data record rate threshold value that is correspondingly arranged is data record duration threshold value, only one
It is a.In another example monitoring server data record rate be recycle duration and data yield by monitoring data to realize, then
The data record rate threshold value being correspondingly arranged is data record duration threshold value and data yield threshold value, there are two.
It should be understood that in a kind of specific embodiment of the present embodiment, in the corresponding index of a performance indicator
Threshold value reaches the corresponding finger of the specific targets in any one specific targets of the performance indicator there are when two or more
When marking threshold value, that is, determine that the performance indicator reaches pre-set level threshold value, enabling reaffirms that Redis is serviced.
It should be understood that certain specific targets may belong to just in the present embodiment for different specific targets
Index, and certain specific targets may belong to negative index.For direct index, sentence when being more than or equal to corresponding metrics-thresholds
The fixed index reaches corresponding metrics-thresholds, and enabling reaffirms that Redis is serviced, such as data record duration belongs to direct index.And
For negative index, determines that the index reaches corresponding metrics-thresholds when being less than or equal to corresponding metrics-thresholds, enable weight
Shen Redis service, such as currently writable amount, data record amount belong to negative index for writing speed, data.
In the present embodiment, if monitoring is that two or more the performance in distribution type file storing data library refers to
It is corresponding can to reach the performance indicator in any one a performance indicator then in a kind of specific embodiment of the present embodiment for mark
Pre-set level threshold value when, just enable Redis service.Certainly, it in a kind of specific embodiment of the present embodiment, can also set
When being scheduled on all properties index and reaching the corresponding pre-set level threshold value of each performance indicator, Redis service is just enabled；Or setting
When there is a performance indicator of N (N be more than or equal to 2) all to reach the corresponding pre-set level threshold value of each performance indicator, Redis clothes are just enabled
Business.
In the present embodiment, shown in Figure 3, Redis service includes:
S301: the data that terminal is sent are received；
It should be noted that the data that terminal is sent in the present embodiment may include user's request and relevant user
Preference or behavioural information.User's request refers to that the requirement server that terminal is sent returns to the solicited message number of certain data
According to, such as can be and enter request etc. using unlatching request, purchase request, the page.In the present embodiment, user preference or behavior letter
Cease the information of the preference that can embody user or behavior that refer to, such as the game that the merchandise news of user's browsing, user are opened
Information, news information of user's browsing etc..In fact, user preference or behavioural information can be requested by a user carrying.Such as eventually
End sends a goods purchase request, then the merchandise news to be bought of user is often carried in the request.
It should be understood that server can also receive the data of terminal transmission, but at that time in not enabled Redis service
Server is that the data for sending terminal are write direct in distribution type file storing data library.For example, see terminal shown in Fig. 4
Data interaction schematic diagram between server, in not enabled Redis service, terminal 41 is the distribution text with server 42
Shelves storing data library carries out data interaction.Data interaction schematic diagram between terminal and server shown in Figure 5, is enabling
When Redis is serviced, terminal 51 is to carry out data interaction with the Redis of server 52.
S302: the data received are written in Redis.
It should be noted that Redis is a high performance key-value database.It supports the value type of storage
Including string (character string), list (chained list), set (set) and zset (ordered set).These data types are all supported
Push/pop, add/remove and intersection union and difference set and richer operation are taken, and these operations are all atomicities.
On this basis, redis supports the sequence of various different modes.For guaranteed efficiency, data are all to cache in memory.
The data of update periodically can be written disk or modification is operated in the additional record file of write-in by redis.
In the present embodiment, after enabling Redis service, server can intercept the number called from user interface automatically
According to, and by write-in Redis in.
In the present embodiment, it after enabling Redis service, may refer to shown in Fig. 6, further includes:
S601: the non-thermal point data in distribution type file storing data library is determined；
In the present embodiment, it can determine in distribution type file storing data library at least one of in the following manner
Non-thermal point data:
Mode one: the reading times of each data in detection distribution type file storing data library；By reading times less than
The data of one reading times threshold value are determined as non-thermal point data.
In the present embodiment, the first reading times threshold value can be by related management personnel based on practical experience or largely
Test comes preset；It may also be what dynamic was calculated.
In a kind of specific embodiment of the present embodiment, the first reading times threshold value is distribution type file storing data library
In each data reading times average value.The practical reading times of data and the reading times of each data can be passed through at this time
Average value come determine which data be non-thermal point data.M number is shared in distribution type file storing data library for example, setting
According to corresponding total reading times are m, then the first reading times threshold value is (m/M), and all reading times can be less than by server
(m/M) data are determined as non-thermal point data.
In the present embodiment, particularly, for the data of certain new additions, it is possible its as caused by time relationship
The data for being no more than K for write-in duration can be set less than the first reading times threshold value in reading times, first without non-
The determination of hot spot data.K can be tested based on practical experience or largely to set in advance by related management personnel in the present embodiment
It is fixed.
Mode two: the corresponding data of each data storage areas in detection distribution type file storing data library always read secondary
Number；The total reading times of data are determined as less than all data in the data storage areas of the second reading times threshold value non-hot
Data.
In the present embodiment, the second reading times threshold value can be by related management personnel based on practical experience or largely
Test comes preset；It may also be what dynamic was calculated.
It should be noted that distribution type file storing data library (MongoDB) is a kind of to be stored based on distributed document
Database Systems, system architecture are distributed type assemblies framework, and data dispersion is stored in the clustered node of Mongodb cluster.
In the present embodiment, multiple clustered nodes can also be made using each clustered node as a data storage areas
For a data storage areas.
In a kind of specific embodiment of the present embodiment, the second reading times threshold value is distribution type file storing data library
In the corresponding total reading times of data of each data storage areas average value.The corresponding reality of data storage areas can be passed through at this time
Which data the average value of the total reading times of border data and the total reading times of the corresponding data of each data storage areas determines
For non-thermal point data.P data storage areas, P data storage areas are shared in distribution type file storing data library for example, setting
Total reading times be p (the sum of the reading times of all data are p i.e. in distribution type file storing data library), then second read
Frequency threshold value is (p/P), and server can be true less than all data in the data storage areas of (p/P) by all reading times
It is set to non-thermal point data.
It should be understood that the corresponding total reading times of data of a data storage areas are are as follows: the data storage areas
In each data the sum of reading times.
Mode three: the total reading times of the corresponding data of all types of data in detection distribution type file storing data library；It will
The all types of data that the total reading times of data are less than third reading times threshold value are determined as non-thermal point data.
In the present embodiment, third reading times threshold value can be by related management personnel based on practical experience or largely
Test comes preset；It may also be what dynamic was calculated.
It should be noted that often there are many kinds of type, examples for the data of storage in distribution type file storing data library
Such as game data, web data, system data.
In a kind of specific embodiment of the present embodiment, third reading times threshold value is distribution type file storing data library
In the corresponding total reading times of data of all types of data average value.The corresponding real data of all types of data can be passed through at this time
The average value of total reading times total reading times of data corresponding with each categorical data determines which data is non-hot
Data.Q kind data are shared in distribution type file storing data library for example, setting, total reading times of this Q kind data are that q (is distributed
The sum of reading times of all data are that q), then third reading times threshold value is (q/Q), server in formula document storing data library
All data for the data type that all reading times are less than (q/Q) can be determined as non-thermal point data, such as game data
Total reading times be less than (q/Q), then all game datas are all determined as non-thermal point data by server.
S602: by non-hot data compression；
In the present embodiment, after detecting each time, all non-hot data compressions that can be will test become
One compressed file, but multiple compressed files can also be collapsed into.For example, determining non-thermal point data according to mode two
It afterwards, can be by the non-hot data compression of each data storage areas at a compressed file；For example determine data storage area
Data in domain 1 are non-thermal point data, determine that the data in data storage areas 2 are non-thermal point data, then by data storage area
Data compression in domain 1 is at a compressed file, by the data compression in data storage areas 2 at a compressed file.Example again
Such as, after determining non-thermal point data according to mode three, the non-hot data compression of each type can be compressed at one
File；For example determine that game data is non-thermal point data, determines that web data is non-thermal point data, then game data is compressed into
Web data is compressed into a compressed file by one compressed file.
It should be understood that cost is relatively low for disk service for server, therefore in the transfer of non-thermal point data
In the process, if the corresponding Insufficient disk space in distribution type file storing data library, it can apply for that bigger disk service comes
Guarantee the smooth of non-thermal point data transfer.
If specifically, can be set disk vacant memory space be less than compressed non-thermal point data size of data,
Server is just automatic to apply for disk service；Or be arranged in disk vacant memory space be less than preset spare space threshold value,
Server is just automatic to apply for disk service.It should be understood that disk service described in the present embodiment refers to for disk distribution more
Big memory space.
After having applied for disk service, the vacant memory space of disk can meet the needs of non-thermal point data transfer,
Compressed non-hot data compression can be written in the disk after having applied for disk service for server.
S603: will be in compressed non-thermal point data write-in disk.
In the present embodiment, when user accesses non-hot request of data later in order to prevent, server can not find correspondence
Data, therefore can just establish the index of non-thermal point data when compressing to non-thermal point data.
In the present embodiment, rope corresponding to compressed file where the index of non-thermal point data can be non-thermal point data
Draw.For example, the non-thermal non-thermal point data 10000 of point data 1- has been compressed into a compressed file, server can be this
Compressed file establishes an index, requests access in this way in user any one in the non-thermal point data 10000 of non-thermal point data 1-
When a, this compressed file can be found by the index, and then the compressed file is decompressed to obtain what user wanted access to
That data.It should be understood that should can be corresponded to for the index that compressed file is established non-hot in compressed file
Data.For example, server can establish a packed record table, record in table by the non-thermal non-thermal point data of point data 1-
10000 compressions are for compressed file A (A is the index that can be considered compressed file).
In the present embodiment, the index of non-thermal point data is also possible to index corresponding to non-thermal point data.I.e. one non-
The corresponding index of hot spot data, the index can be directed toward the compressed file of non-thermal point data or the compression text of non-thermal point data
Storage location of the part in disk.
In the present embodiment, after the index for establishing non-thermal point data, the reading of non-thermal point data is asked if receiving
It asks, then can find compressed non-thermal point data in disk according to the index of non-thermal point data；And then it will be compressed non-
Hot spot data decompression, returns to the requested data of read requests in non-thermal point data.
In the present embodiment, after compressed non-thermal point data being written in disk, distribution type file storing data library
The i.e. vacant data write-in distribution type file storage number that in addition to a large amount of memory space, will can be written before in Redis at this time
According in library.Meanwhile server can be automatically deleted the data being had been written into distribution type file storing data library in Redis.
It should be understood that after by compressed non-thermal point data write-in disk, distribution type file storing data
Library, that is, vacant in addition to a large amount of memory space, the ability in distribution type file storing data library is relatively by compressed non-thermal points at this time
According to a degree of promotion is had before in write-in disk, if the performance indicator in the distribution type file storing data library of monitoring at this time
Not up to pre-set level threshold value, the i.e. ability in distribution type file storing data library are sufficient for server and want to the processing of data
It asks, server can discharge Redis service automatically, and the terminal data received is write direct distribution type file storing data
In library, to reduce system cost.
It should be understood that each step in method for writing data provided in this embodiment can be by as shown in Figure 1
Server is independently realized.Specifically, can realize the one or more of above-mentioned each step by storing in memory 13
Program executes the program by processor 12, to realize each step of above-mentioned method for writing data.
Method for writing data provided in this embodiment, by at least one performance for monitoring distribution type file storing data library
Index, and when performance indicator reaches pre-set level threshold value, Redis service is enabled, the data that the terminal received is sent are write
Enter in Redis.In this way, in distribution type file storing data library scarce capacity, number can be written by Redis in server
According to the distribution type file storing data library for having got around scarce capacity ensure that server to ensure the write performance of server
The concurrency of side, while the data for also ensuring that server can in time send terminal perform corresponding processing, by respective counts
According to terminal is returned to, guarantee the smoothness of the application of user side.
It, can will be non-thermal in distribution type file storing data library in addition, server is when by Redis data are written
Point data compression is stored into disk, to restore the ability in distribution type file storing data library to a certain extent, and is being divided
Cloth document storing data library ability restore to meet server to be automatically closed after the requirement of data processing Redis service, after
It is continuous that data are written using distribution type file storing data library, it ensure that server to the cost of data processing.
In addition, server when compressing non-thermal point data, can establish index for non-thermal point data, to ensure that user i.e.
Make to be intended to access non-thermal point data, server can also be quickly found out user and want access to non-thermal point data and return to use
Family.
Second embodiment
The present embodiment is on the basis of first embodiment, by taking a kind of more specific data writing process as an example, to this hair
Bright work further illustrates.
Shown in Figure 7, Fig. 7 is the flow diagram of method for writing data provided by the present embodiment, comprising:
S701: the data write performance and the data rate of recovery in monitoring distribution type file storing data library；
Data write performance is current by the writing speed in distribution type file storing data library, data in the present embodiment
Writable amount is embodied.Data record rate passes through the data record duration in distribution type file storing data library, data record
Amount is to be embodied.Server is substantially the data write-in speed in the distribution type file storing data library of monitoring when being monitored
Degree, data currently writable amount, data record duration, data record amount.
It should be noted that data record referred to deletes data from memory in distribution type file storing data library
It removes.When deleting data from memory, data can be marked in server, and after a certain time delete the data being marked
It removes.Therefore, the data that data record duration described in the present embodiment refers to are from this duration for being tagged to deletion, the present embodiment
Described in the data volume deleted from memory in certain time length section that refers to of data record amount.
S702: reach preset data write performance threshold value in data write performance or data record rate reaches preset data and returns
When yield threshold value, enable in data write-in Redis of the Redis service to send the terminal received；
In the present embodiment, the writing speed in distribution type file storing data library and data are currently in writable amount
When any one has reached corresponding metrics-thresholds, i.e. judgement data write performance reaches preset data write performance threshold value.Together
It manages, any one in the data record duration and data yield in distribution type file storing data library has reached its corresponding finger
When marking threshold value, i.e. the judgement data rate of recovery reaches preset data rate of recovery threshold value.I.e. in the present embodiment, distribution type file stores
Currently writable amount, data record duration, any one in data record amount reach for the writing speed of database, data
When its corresponding metrics-thresholds, server is to enable data of the Redis service to send the terminal that receives to be written
In Redis, to guarantee concurrency and to the quick response of terminal.
It should be understood that certain specific targets may belong to just in the present embodiment for different specific targets
Index, and certain specific targets may belong to negative index.For direct index, sentence when being more than or equal to corresponding metrics-thresholds
The fixed index reaches corresponding metrics-thresholds, and enabling reaffirms that Redis is serviced, such as data record duration belongs to direct index.And
For negative index, determines that the index reaches corresponding metrics-thresholds when being less than or equal to corresponding metrics-thresholds, enable weight
Shen Redis service, such as currently writable amount, data record amount belong to negative index for writing speed, data.
In the present embodiment, each metrics-thresholds can be the critical value of each performance indicator, that is, be more than i.e. meeting after threshold value
The ability rapid decrease for leading to distribution type file storing data library, influences user experience.
S703: the non-thermal point data in scanning distribution type file storing data library；
In the present embodiment, non-hot judgment criteria is data in a data storage areas in a certain period of time
The number being inside read into.Server is according to all readings of the data in each data storage areas in distribution type file storing data library
Number is taken, it is secondary lower than the reading of each data storage areas in distribution type file storing data library to find the reading times within the time cycle
All data in these data storage areas are all determined as non-thermal point data by the data storage areas of number average value.
S704: it will be written in disk after non-hot data compression；
In the present embodiment, index is established for this non-thermal point data in compression, to guarantee that it is non-that user needs to access this part
When hot spot data, server can be quickly found out its corresponding compressed file in magnetic according to the index of the non-thermal point data of required access
Position in disk, and then compressed file decompression is found into the corresponding non-thermal point data of the index, to quickly user be needed to visit
The non-thermal point data asked returns to user.In this way, server both can guarantee that distribution type file storing data library was accessed in user
All data, and can sufficiently improve the running efficiency and space utilization rate of whole system.
In addition, cost is relatively low for disk service for server, so during the transfer of non-thermal point data,
If server disk insufficient space, server can apply for bigger disk service automatically.
S705: will be in the data write-in distribution type file storing data library in Redis；
Will by non-hot data compression after be written disk in after, distribution type file storing data library, that is, vacant in addition to a large amount of
Memory space, the data in Redis can be written in distribution type file storing data library at this time, and discharge in Redis and write
The data entered guarantee the performance of Redis.
S706: distribution type file storing data library is written in release Redis service, the data that the terminal received is sent
In.
After by compressed non-thermal point data write-in disk, distribution type file storing data library, that is, vacant in addition to big
The memory space of amount, the ability in distribution type file storing data library is relatively by it in compressed non-thermal point data write-in disk at this time
Before have a degree of promotion, if the performance indicator in distribution type file storing data library of monitoring is not up to pre-set level at this time
Threshold value, the i.e. ability in distribution type file storing data library are sufficient for server to the processing requirement of data, and server can
To discharge Redis service automatically, the terminal data received is write direct in distribution type file storing data library, is to reduce
System cost.
A kind of method for writing data is present embodiments provided, the performance by monitoring distribution type file storing data library refers to
Mark, and when performance indicator reaches pre-set level threshold value, Redis service is enabled, the data that the terminal received is sent are written
In Redis.In this way, in distribution type file storing data library scarce capacity, data can be written by Redis in server,
The distribution type file storing data library for having got around scarce capacity ensure that server side to ensure the write performance of server
Concurrency, while also ensure server can in time to terminal send data perform corresponding processing, by corresponding data
Terminal is returned to, guarantees the smoothness of the application of user side.
It, can will be non-thermal in distribution type file storing data library in addition, server is when by Redis data are written
Point data compression is stored into disk, to restore the ability in distribution type file storing data library to a certain extent, and is being divided
Cloth document storing data library ability restore to meet server to be automatically closed after the requirement of data processing Redis service, after
It is continuous that data are written using distribution type file storing data library, it ensure that server to the cost of data processing.
In addition, server when compressing non-thermal point data, can establish index for non-thermal point data, so that server was both
It can guarantee that all data in distribution type file storing data library are accessed in user, and can sufficiently improve the running efficiency of whole system
With space utilization rate.
3rd embodiment
A kind of server is present embodiments provided, it is shown in Figure 8 comprising processor 81, memory 82 and communication
Bus 83.Wherein:
Communication bus 83 is for realizing the connection communication between processor 81 and memory 82.
Processor 81 is for executing the one or more programs stored in memory 82, to realize above-mentioned first embodiment
And/or each step of method for writing data described in second embodiment.
Meanwhile the present embodiment also provides a kind of computer readable storage medium, as floppy disk, CD, hard disk, flash memory, USB flash disk,
CF card, SD card, mmc card etc. are stored with one for realizing above-mentioned each step or more in the computer readable storage medium
A program, this one or more program can be executed by one or more processor, with realize above-mentioned first embodiment and/or
Each step of method for writing data described in second embodiment.Details are not described herein.
It should be noted that, in this document, the terms "include", "comprise" or its any other variant are intended to non-row
His property includes, so that the process, method, article or the device that include a series of elements not only include those elements, and
And further include other elements that are not explicitly listed, or further include for this process, method, article or device institute it is intrinsic
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including being somebody's turn to do
There is also other identical elements in the process, method of element, article or device.
The serial number of the above embodiments of the invention is only for description, does not represent the advantages or disadvantages of the embodiments.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side
Method can be realized by means of software and necessary general hardware platform, naturally it is also possible to by hardware, but in many cases
The former is more preferably embodiment.Based on this understanding, technical solution of the present invention substantially in other words does the prior art
The part contributed out can be embodied in the form of software products, which is stored in a storage medium
In (such as ROM/RAM, magnetic disk, CD), including some instructions are used so that a terminal (can be mobile phone, computer, service
Device, air conditioner or network equipment etc.) execute method described in each embodiment of the present invention.
The embodiment of the present invention is described with above attached drawing, but the invention is not limited to above-mentioned specific
Embodiment, the above mentioned embodiment is only schematical, rather than restrictive, those skilled in the art
Under the inspiration of the present invention, without breaking away from the scope protected by the purposes and claims of the present invention, it can also make very much
Form, all of these belong to the protection of the present invention.
Claims (10)
1. a kind of method for writing data, which is characterized in that the method for writing data includes:
Monitor at least one performance indicator in distribution type file storing data library；
When the performance indicator reaches pre-set level threshold value, enabling reaffirms that (Redis) is serviced；
The Redis service includes:
Receive the data that terminal is sent；
The data received are written in Redis.
2. method for writing data as described in claim 1, which is characterized in that the performance indicator include data write performance,
At least one of data record rate.
3. method for writing data as described in claim 1, which is characterized in that the distribution type file storing data is supervised in library
Survey performance indicator at least there are two types of；
It is described when the performance indicator reaches pre-set level threshold value, enable Redis service include: in any one a performance indicator
When reaching the corresponding pre-set level threshold value of the performance indicator, Redis service is enabled.
4. method for writing data as described in any one of claims 1-3, which is characterized in that after enabling Redis service, also
Include:
Determine the non-thermal point data in distribution type file storing data library；
By the non-hot data compression；
It will be in the compressed non-thermal point data write-in disk.
5. method for writing data as claimed in claim 4, which is characterized in that the determination distribution type file storing data
Non-hot data in library include: to determine in distribution type file storing data library at least one of in the following manner
Non-thermal point data:
Mode one: the reading times of each data in distribution type file storing data library are detected；By reading times less than
The data of one reading times threshold value are determined as non-thermal point data；
Mode two: the corresponding data of each data storage areas in distribution type file storing data library are detected and are always read time
Number；The total reading times of the data are determined as less than all data in the data storage areas of the second reading times threshold value non-
Hot spot data；
Mode three: the total reading times of the corresponding data of all types of data in distribution type file storing data library are detected；It will
The all types of data that the total reading times of data are less than third reading times threshold value are determined as non-thermal point data.
6. method for writing data as claimed in claim 5, which is characterized in that the first reading times threshold value is the distribution
The average value of the reading times of each data in formula document storing data library；
The second reading times threshold value is the corresponding data of data storage areas each in distribution type file storing data library
The average value of total reading times；
The third reading times threshold value is always read for the corresponding data of data all types of in distribution type file storing data library
Take the average value of number.
7. method for writing data as claimed in claim 4, which is characterized in that described by the compressed non-thermal point data
After being written in disk, further includes:
It will be in the data write-in distribution type file storing data library in the Redis.
8. method for writing data as claimed in claim 4, which is characterized in that described to write the compressed non-thermal point data
Enter in disk and includes:
If the vacant memory space of the disk be less than the compressed non-thermal point data size of data or the disk
Vacant memory space is less than preset spare space threshold value, applies for disk service automatically；The disk service includes for the magnetic
Disk distributes bigger memory space；
In the disk after the compressed non-hot data compression write-in to have been applied to the disk service.
9. method for writing data as claimed in claim 4, which is characterized in that it is described by the non-hot data compression when,
Further include: establish the index of the non-thermal point data；
It is described by the compressed non-thermal point data write-in disk after, further includes:
When receiving the read requests to the non-thermal point data, found in disk according to the index of the non-thermal point data
Compressed non-thermal point data；
By the compressed non-hot data decompression, the requested number of read requests described in the non-thermal point data is returned
According to.
10. a kind of server, which is characterized in that the server includes processor, memory and communication bus；
The communication bus is for realizing the connection communication between the processor and memory；
The processor is for executing one or more program stored in the memory, to realize such as claim 1-9
The step of described in any item method for writing data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810532139.9A CN108984589A (en) | 2018-05-29 | 2018-05-29 | A kind of method for writing data and server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810532139.9A CN108984589A (en) | 2018-05-29 | 2018-05-29 | A kind of method for writing data and server |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108984589A true CN108984589A (en) | 2018-12-11 |
Family
ID=64542748
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810532139.9A Pending CN108984589A (en) | 2018-05-29 | 2018-05-29 | A kind of method for writing data and server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108984589A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110007878A (en) * | 2019-04-17 | 2019-07-12 | Oppo广东移动通信有限公司 | Storage card processing method, storage card processing unit and mobile terminal |
| CN112351111A (en) * | 2020-12-30 | 2021-02-09 | 常州微亿智造科技有限公司 | Near-real-time progress monitoring method for uploading industrial quality inspection file to server terminal |
| CN114356225A (en) * | 2021-12-17 | 2022-04-15 | 得一微电子股份有限公司 | Data storage method and device of memory, terminal equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103617007A (en) * | 2013-12-04 | 2014-03-05 | 深圳市华成峰实业有限公司 | Method and system for achieving multilevel intelligent storage |
| CN104539713A (en) * | 2014-12-31 | 2015-04-22 | 北京奇虎科技有限公司 | Service request processing method and device |
| CN104965850A (en) * | 2015-04-29 | 2015-10-07 | 云南电网有限责任公司 | Database high-available implementation method based on open source technology |
| CN106411578A (en) * | 2016-09-12 | 2017-02-15 | 国网山东省电力公司电力科学研究院 | Website monitoring system and method applicable to power industry |
| CN107391633A (en) * | 2017-06-30 | 2017-11-24 | 北京奇虎科技有限公司 | Data-base cluster Automatic Optimal processing method, device and server |
| CN107729217A (en) * | 2017-10-30 | 2018-02-23 | 郑州云海信息技术有限公司 | A kind of database abnormality eliminating method and terminal |
-
2018
- 2018-05-29 CN CN201810532139.9A patent/CN108984589A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103617007A (en) * | 2013-12-04 | 2014-03-05 | 深圳市华成峰实业有限公司 | Method and system for achieving multilevel intelligent storage |
| CN104539713A (en) * | 2014-12-31 | 2015-04-22 | 北京奇虎科技有限公司 | Service request processing method and device |
| CN104965850A (en) * | 2015-04-29 | 2015-10-07 | 云南电网有限责任公司 | Database high-available implementation method based on open source technology |
| CN106411578A (en) * | 2016-09-12 | 2017-02-15 | 国网山东省电力公司电力科学研究院 | Website monitoring system and method applicable to power industry |
| CN107391633A (en) * | 2017-06-30 | 2017-11-24 | 北京奇虎科技有限公司 | Data-base cluster Automatic Optimal processing method, device and server |
| CN107729217A (en) * | 2017-10-30 | 2018-02-23 | 郑州云海信息技术有限公司 | A kind of database abnormality eliminating method and terminal |
Non-Patent Citations (1)
| Title |
|---|
| 奈文: "MongoDB+Redis任务队列性能瓶颈", 《SEGMENTFAULT：HTTPS://SEGMENTFAULT.COM/Q/1010000000318681》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110007878A (en) * | 2019-04-17 | 2019-07-12 | Oppo广东移动通信有限公司 | Storage card processing method, storage card processing unit and mobile terminal |
| CN112351111A (en) * | 2020-12-30 | 2021-02-09 | 常州微亿智造科技有限公司 | Near-real-time progress monitoring method for uploading industrial quality inspection file to server terminal |
| CN114356225A (en) * | 2021-12-17 | 2022-04-15 | 得一微电子股份有限公司 | Data storage method and device of memory, terminal equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8521986B2 (en) | Allocating storage memory based on future file size or use estimates | |
| CN105224528B (en) | Big data processing method and device based on graph calculation | |
| CN108984589A (en) | A kind of method for writing data and server | |
| CN104272266A (en) | Management system which manages computer system having plurality of devices to be monitored | |
| WO2016115957A1 (en) | Method and device for accelerating computers and intelligent devices for users and applications | |
| CN104765794B (en) | A kind of method for partitioning database and device | |
| CN109086141B (en) | Memory management method and device and computer readable storage medium | |
| CN109491928A (en) | Buffer control method, device, terminal and storage medium | |
| CN110989937B (en) | Data storage method, device and equipment and computer readable storage medium | |
| CN109284220A (en) | Clustering fault restores duration evaluation method, device, equipment and storage medium | |
| US20050198298A1 (en) | System monitoring method | |
| WO2020000492A1 (en) | Storage fragment managing method and terminal | |
| CN108255620A (en) | A kind of business logic processing method, apparatus, service server and system | |
| CN108694188A (en) | A kind of newer method of index data and relevant apparatus | |
| KR20160096313A (en) | Apparatus and method for monitoring analysis application for analyzing big data | |
| JP2012247901A (en) | Database management method, database management device, and program | |
| CN110245129A (en) | Distributed global data deduplication method and device | |
| CN109656479A (en) | A kind of method and device constructing memory command sequence | |
| CN109144431A (en) | Caching method, device, equipment and the storage medium of data block | |
| CN114077690A (en) | Vector data processing method, device, equipment and storage medium | |
| CN108132759A (en) | A kind of method and apparatus that data are managed in file system | |
| US11194861B2 (en) | Graph partitioning method and apparatus | |
| CN104699513B (en) | A kind of document handling method and device | |
| CN106326400A (en) | Multi-dimension data set-based data processing system | |
| CN112783417A (en) | Data reduction method and device, computing equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181211 |
|
| RJ01 | Rejection of invention patent application after publication |