CN108121507A - A kind of data processing method, device and electronic equipment - Google Patents
A kind of data processing method, device and electronic equipment Download PDFInfo
- Publication number
- CN108121507A CN108121507A CN201711274747.6A CN201711274747A CN108121507A CN 108121507 A CN108121507 A CN 108121507A CN 201711274747 A CN201711274747 A CN 201711274747A CN 108121507 A CN108121507 A CN 108121507A
- Authority
- CN
- China
- Prior art keywords
- agent
- data
- data volume
- water conservancy
- conservancy diversion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0626—Reducing size or complexity of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
An embodiment of the present invention provides a kind of data processing method, device and electronic equipment, wherein, the data processing method includes：According to preset period of time, the first data volume for treating water conservancy diversion data cached in current slot in agency service Agent is obtained；It obtained in a current slot adjacent upper period, what is cached in the Agent treats the second data volume of water conservancy diversion data；According to first data volume and second data volume, the number change mode of the Agent is determined, the number change mode includes, and deletes, increases or constant；Number according to the Agent changes mode, and corresponding change is currently accessed the number of Agent.The embodiment of the present invention realizes reasonable employment Agent and carries out data water conservancy diversion, reduces the wasting of resources.
Description
Technical field
The present invention relates to field of computer technology, more particularly to a kind of data processing method, device and electronic equipment.
Background technology
In actual production, needs are frequently encountered data flow water conservancy diversion to the situation in another storage system.Existing skill
In art, can by agency service Agent by data flow from a system water conservancy diversion to another storage system.In order to ensure data
The flow vertex value of data flow is usually set to the fixation throughput value of water conservancy diversion, i.e., handled up according to the fixation by the real-time of water conservancy diversion
Magnitude sets the number of the Agent needed, and the number of the Agent is unalterable.
However, inventor has found in the implementation of the present invention, at least there are the following problems for the prior art：
During actual water conservancy diversion, the uninterrupted of data flow is continually changing, is become such as the quantity of online user
Change and cyclically-varying is presented or changes because of other reasons.And it is to immobilize to carry out the Agent that data water conservancy diversion uses
, this may cause the Agent wastings of resources.
The content of the invention
The embodiment of the present invention is designed to provide a kind of data processing method, device and electronic equipment, is closed with realizing
Reason carries out data water conservancy diversion using Agent, reduces the wasting of resources.Specific technical solution is as follows：
To achieve the above object of the invention, the embodiment of the invention discloses a kind of data processing method, including：
According to preset period of time, treat water conservancy diversion data the cached in current slot in agency service Agent is obtained
One data volume；
It obtained in a current slot adjacent upper period, what is cached in the Agent treats the of water conservancy diversion data
Two data volumes；
According to first data volume and second data volume, the number change mode of the Agent is determined, it is described a
Number change modes include, and delete, increase or constant；
Number according to the Agent changes mode, and corresponding change is currently accessed the number of Agent.
Optionally, caching treats that the process of water conservancy diversion data includes in the Agent：
Acquisition is treated into water conservancy diversion data sending to during recipient, is sent when obtaining the rate for the treatment of water conservancy diversion data and being more than
When the rate of water conservancy diversion data, caching treats water conservancy diversion data in the Agent.
Optionally, it is described according to first data volume and second data volume, determine the number change of the Agent
Mode, including：
First data volume and second data volume are done into difference operation, and by the difference operation result with it is described
The handling capacity of Agent does business's computing, obtains the functional relation between first data volume and second data volume；
According to the functional relation, determine that the number of the Agent changes mode.
Optionally, it is described according to the functional relation, determine the number change mode of the Agent, including：
Calculate the end value of the functional relation；
When the end value is positive number, the number change mode of the Agent is determined to increase corresponding number
Agent；
Alternatively, when the end value is zero and first data volume is not zero equal to second data volume, determine
The number change mode of the Agent is constant；
Alternatively, when the end value is zero and first data volume equal to second data volume is zero, institute is determined
The number change mode of Agent is stated to delete an Agent；
Alternatively, when the end value is negative, the number change mode of the Agent is determined to delete corresponding number
Agent。
To achieve the above object of the invention, the embodiment of the invention also discloses a kind of data processing equipment, including：
First data volume acquisition module, for according to preset period of time, obtaining agency service Agent in current slot
The first data volume for treating water conservancy diversion data of middle caching；
Second data volume acquisition module, for obtaining in a current slot adjacent upper period, the Agent
The second data volume for treating water conservancy diversion data of middle caching；
Change mode determining module, for according to first data volume and second data volume, determining the Agent
Number change mode, number change mode includes, and deletes, increases or constant；
Agent changes module, and for changing mode according to the number of the Agent, corresponding change is currently accessed Agent's
Number.
Optionally, described device further includes：
It treats water conservancy diversion data cache module, treats that water conservancy diversion data sending to during recipient, works as acquisition for what will be obtained
When the rate of water conservancy diversion data is more than and sends when the rate of water conservancy diversion data, caching treats water conservancy diversion data in the Agent.
Optionally, the change mode determining module, including：
Functional relation determination sub-module, for first data volume and second data volume to be done difference operation, and
The handling capacity of the difference operation result and the Agent are done into business's computing, obtain first data volume and the described second number
According to the functional relation between amount；
Change mode determination sub-module, for according to the functional relation, determining that the number of the Agent changes mode.
Optionally, the change mode determination sub-module, specifically for calculating the end value of the functional relation；When described
When end value is positive number, the number change mode of the Agent is determined to increase the Agent of corresponding number；Alternatively, when the knot
When fruit value is zero and first data volume is not zero equal to second data volume, the number change side of the Agent is determined
Formula is constant；Alternatively, when the end value is zero and first data volume equal to second data volume is zero, institute is determined
The number change mode of Agent is stated to delete an Agent；Alternatively, when the end value is negative, the Agent is determined
Number change mode to delete the Agent of corresponding number.
To achieve the above object of the invention, the embodiment of the invention discloses a kind of electronic equipment, connect including processor, communication
Mouth, memory and communication bus, wherein, the processor, the communication interface, the memory are complete by the communication bus
Into mutual communication；
The memory, for storing computer program；
The processor during for performing the program stored on the memory, realizes any of the above-described data
Processing method.
To achieve the above object of the invention, the embodiment of the invention also discloses a kind of computer readable storage medium, the meters
Instruction is stored in calculation machine readable storage medium storing program for executing, when run on a computer so that computer performs any of the above-described described
Data processing method.
To achieve the above object of the invention, the embodiment of the invention also discloses it is a kind of comprising instruction computer program product,
When run on a computer so that computer performs any of the above-described data processing method.
A kind of data processing method provided in an embodiment of the present invention, device and electronic equipment, realize reasonable employment
Agent carries out data water conservancy diversion, reduces the waste of resource.Specifically, obtain in current time Agent and cache according to the time cycle
The data volume for treating water conservancy diversion data, be the first data volume；And obtained in the upper period adjacent with the current slot,
What is cached in the Agent treats the data volume of water conservancy diversion data, is the second data volume；Pass through the first data volume and the second data volume
Relation obtains the number of current desired Agent, and then the number of change Agent is corresponded to according to the real-time demands of Agent.It can
See, the embodiment of the present invention can be according to the real-time change of flow, and then determines the number of required Agent, reaches dynamic and timely adjusts
The number of whole Agent, and then effectively avoid the waste of resource.In addition, the data processing method of the embodiment of the present invention can be realized certainly
The number of the dynamic current Agent of adjustment, without manpower intervention, improves work efficiency.
Description of the drawings
It in order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is attached drawing needed in technology description to be briefly described.
Fig. 1 is a kind of data processing method flow chart of the embodiment of the present invention；
Fig. 2 be the embodiment of the present invention a kind of data processing method in Agent number change mode definite method flow
Figure；
Fig. 3 is a kind of data processing method system construction drawing of the embodiment of the present invention；
Fig. 4 is another data processing method flow chart of the embodiment of the present invention；
Fig. 5 is a kind of data processing equipment structure diagram of the embodiment of the present invention；
Fig. 6 is a kind of electronic equipment structure diagram of the embodiment of the present invention.
Specific embodiment
Below in conjunction with the attached drawing in the embodiment of the present invention, the technical solution in the embodiment of the present invention is described.
With the fast development of information technology, data message is also more and more huger, how that the data of a system are quick
Accurate water conservancy diversion stored into other system and data handling procedure in the problem of needing concern.In the prior art
There is the method by data flow from a system water conservancy diversion to another storage system by agency service Agent.In order to ensure data
The flow vertex value of data flow is usually set to the fixation throughput value of water conservancy diversion, i.e., handled up according to the fixation by the real-time of water conservancy diversion
Magnitude sets the number of the Agent needed, and the number of the Agent is unalterable.However existing for this diversion mode
Problem is that the flow of current data stream causes the Agent wastings of resources when smaller.
To solve the above-mentioned problems, the embodiment of the invention discloses a kind of data processing method, device and electronic equipment,
By periodically obtaining the data volume of caching, and then the number for changing Agent reaches conjunction to adapt to the data volume of current water conservancy diversion
Reason carries out data water conservancy diversion using Agent, reduces the purpose of the wasting of resources.Specific embodiment is as follows：
To achieve the above object of the invention, the embodiment of the invention discloses a kind of data processing method, as shown in Figure 1.Fig. 1 is
A kind of data processing method flow chart of the embodiment of the present invention, including：
S101, according to preset period of time, what is cached in acquisition current slot in agency service Agent treats water conservancy diversion data
The first data volume.
In embodiments of the present invention, by periodically detect cached in agency service Agent treat water conservancy diversion data and
Pass between the data volume for treating water conservancy diversion data cached in the upper period adjacent with the period of the current detection in Agent
System determines whether to need the number for changing Agent.
It should be noted that the executive agent of the embodiment of the present invention is the software for being provided with the data processing method of the present invention
The hardware device of program or data processing method with the embodiment of the present invention.It will be appreciated by persons skilled in the art that
The executive agent of following steps is same executive agent, therefore is no longer repeated one by one.
In this step, preset period of time can be set, and then when reaching preset period of time, executive agent obtains
What current slot cached in Agent treats water conservancy diversion data volume, is the first data volume of the embodiment of the present invention.For example, when default
Between the cycle be arranged to 30 minutes, current time 12:00, then it can obtain 11 in each Agent:30-12:00 caching treats water conservancy diversion
Data volume.
S102 was obtained in a current slot adjacent upper period, and what is cached in Agent treats the second number of water conservancy diversion data
According to amount.
It is above-mentioned cached in all Agent of current slot are obtained treat water conservancy diversion data volume, be the embodiment of the present invention
The first data volume after, pass through executive agent and obtain what is cached in upper period adjacent with the current slot in Agent
It treats water conservancy diversion data volume, is the second data volume of the embodiment of the present invention.For example, preset period of time is arranged to 30 minutes, currently
Time is 12:00, then it obtains 11 in each Agent:00-11:30 cachings treat water conservancy diversion data volume.
S103 according to the first data volume and the second data volume, determines that the number of Agent changes mode, number change mode
Including deleting, increase or constant.
It is above-mentioned after the first data volume and the second data volume is obtained, by the first data volume and the second data volume and carry
The handling capacity of each Agent of preceding setting, it may be determined that go out the number change mode of current time Agent.The number of the Agent becomes
More mode is reduced to increase the number of Agent on the basis of original Agent number on the basis of original Agent number
The number of Agent and these three constant situations of the number of original Agent.
Specifically, can it is pre- to be calculated by preset function relation determined by the first data volume and the second data volume for this
If the end value of functional relation, when the result is positive number, corresponding change mode is to increase the number of Agent；When the result
Be worth for -1 when, corresponding change mode for Agent number it is constant；When end value is non-1 negative, corresponding change mode
To delete Agent.
S104 changes mode according to the number of Agent, and corresponding change is currently accessed the number of Agent.
It is above-mentioned after the number change mode of the Agent corresponding to current time is determined, according to above-mentioned definite Agent
Number change mode, it is corresponding to increase or reduction and the end value corresponding number on the basis of original accesses Agent number
Agent。
A kind of data processing method provided in an embodiment of the present invention realizes reasonable employment Agent and carries out data water conservancy diversion, subtracts
The waste of resource is lacked.Specifically, the data for treating water conservancy diversion data cached in current time Agent are obtained according to the time cycle
Amount is the first data volume；And obtained in the upper period adjacent with the current slot, what is cached in the Agent treats
The data volume of water conservancy diversion data is the second data volume；By the first data volume and the relation of the second data volume, obtain current desired
The number of Agent, and then the number for changing Agent is corresponded to according to the real-time demands of Agent.As it can be seen that the embodiment of the present invention can
It according to the real-time change of flow, and then determines the number of required Agent, reaches the number of dynamic adjustment Agent, Jin Eryou
Effect avoids the waste of resource.In addition, the data processing method of the embodiment of the present invention can realize the number for automatically adjusting current Agent
Mesh without manpower intervention, improves work efficiency.
Optionally, in a kind of embodiment of the data processing method of the embodiment of the present invention, caching treats water conservancy diversion in Agent
The process of data includes：
Acquisition is treated into water conservancy diversion data sending to during recipient, is sent when obtaining the rate for the treatment of water conservancy diversion data and being more than
When the rate of water conservancy diversion data, caching treats water conservancy diversion data in Agent.
Agent technologies are a kind of computer systems packed under certain environment.It has independence, interactivity, anti-
Ying Xing, initiative should additionally possess the characteristic that some mankind just have, such as knowledge, conviction, obligation, intention.
The independence of Agent is mainly reflected in：Agent, which has, belongs to the computing resource of its own and part in the row of itself
Mechanism in order to control, can be direct operated in no external world, according to its internal state and the environmental information perceived, determines
Determine and control the behavior of itself.For example, SNMP (Simple NetworkManagement Protocol, simple network management association
View) in Agent be exactly independent operating in managed unit from host process；The interactivity of Agent is mainly reflected in：
Agent can implement flexile interaction with other Agent (including people) Agent communication languages, can effectively with its
His Agent cooperates.For example, the user on an Internet is needed using Agent communication languages to the Agent that takes the initiative in offering a hand
State information requirement.
The reactivity of Agent is mainly reflected in：It (may be physical world, manipulation figure that Agent, which can perceive residing environment,
The user at shape interface or other Agent etc.), and in due course reaction is made to dependent event.For example, the Agent of a simulation aircraft
In due course reaction can be made to the manipulation of user；The initiative of Agent is mainly reflected in：Agent can follow promise and take master
Dynamic action, shows object-oriented behavior.For example, the Agent that takes the initiative in offering a hand on an Internet, is obtaining new information
It can actively be submitted to the users of needs according to agreement afterwards, a Workflow Management Agent can be according to arrange will most
New job progress is actively notified to related work station.
The embodiment of the present invention is by the above-mentioned characteristics of Agent technologies, in the reception treated water conservancy diversion data terminal Yu treat water conservancy diversion data
Agent technologies between side are set and each Agent is managed by flexible resource pond, so as to realize data water conservancy diversion by Agent,
And the caching that water conservancy diversion data are treated in each Agent realizations is managed by flexible resource pond.Flexible resource pond is similar based on distribution
Formula resource management framework, such as mesos can easily be applied, discharge, the service of the usage amount of flexible resource at any time.
There are three module inside Agent, pulling data is each responsible for, it is data cached, send data.
In embodiments of the present invention, Agent will treat water conservancy diversion data sending to during recipient, and water conservancy diversion data are treated in acquisition
Rate and transmission treat the rate of water conservancy diversion data, and then both compare the relation of the size of rate.When water conservancy diversion data are treated in acquisition
Rate be more than send when the rate of water conservancy diversion data, remaining data volume will be generated in Agent, that is, occurs caching and waits to lead
The situation of flow data.
Acquisition is treated into water conservancy diversion data sending to during recipient, is sent when obtaining the rate for the treatment of water conservancy diversion data and being more than
When the rate of water conservancy diversion data, caching treats water conservancy diversion data in Agent.
As it can be seen that by the embodiment of the present invention, it can be achieved that detecting the efficiency of service of Agent in real time, you can detection Agent is obtained
The rate for treating water conservancy diversion data and Agent is taken to send the rate for treating water conservancy diversion data, so also can determine whether be in current time Agent
No caching needs water conservancy diversion data.
Optionally, in a kind of embodiment of the data processing method of the embodiment of the present invention, according to the first data volume and the
Two data volumes determine the number change mode of Agent, can be just like a kind of data processing side of the embodiment of the present invention shown in Fig. 2
The definite method flow diagram of Agent number change mode in method, including：
First data volume and the second data volume are done difference operation by S201, and handling up difference operation result and Agent
Amount does business's computing, obtains the functional relation between the first data volume and the second data volume.
The embodiment of the present invention carries out data water conservancy diversion using flexible resource pond deployment Agent, by the fortune for periodically obtaining Agent
Row state, and then decide whether to apply for new Agent to flexible resource pond or destroy one or more Agent.
In this step, it may be determined that go out the functional relation between the first data volume and the second data volume.Specific implementation process
For：The difference of the first data volume and the second data volume is first calculated, then business's fortune is done with the handling capacity of Agent on the basis of the difference
It calculates, the functional relation which being determined as between the first data volume and the second data volume.
Assuming that the first data volume is expressed as S0；Second data volume is expressed as S1 because the second data volume S1 represent with it is current
Data volume in adjacent upper period period, then the second data volume S1 is history given data；The scale of handling up of Agent
Q is shown as, handling capacity Q is the known quantity set in advance.Then the functional relation between the first data volume and the second data volume can table
Show as follows：
Num=(S0-S1)/Q, wherein, num represents that the functional relation between the first data volume S0 and the second data volume S1 is true
Fixed end value, it is the amount changed with the variation of the first data volume S0 and the second data volume S1.
S202 calculates the end value of functional relation.
Specifically, according to the functional relation between above-mentioned first data volume and the second data volume, obtained by service pre-
If what is cached in the current slot corresponding to the time cycle in Agent treats the first data volume S0 of water conservancy diversion data, obtain and work as
The the second data volume S1 for treating water conservancy diversion data that was cached in adjacent upper period preceding period in Agent and obtain Agent
Handling capacity Q (each reporting interval can send Q datas), and then result of calculation value num.
S2031 when end value is positive number, determines the number change mode of Agent to increase the Agent of corresponding number；
S2032, when end value be zero and first data volume be not zero equal to the second data volume when, determine the number of Agent
Change mode is constant；
S2033, when end value be zero and first data volume equal to the second data volume be zero when, determine that the number of Agent becomes
More mode is one Agent of deletion；
S2034 when end value is negative, determines the number change mode of Agent to delete the Agent of corresponding number.
It is above-mentioned the end value of num is calculated after, the end value of the num is corresponding with 3 kinds of situations：Positive number, zero and negative
Number.It then can determine that the number change mode of current time Agent.
That is the num>When 0, illustrate that the transmission rate of current Agent is slower, there is more buffer data size in Agent, need
Increase Agent that will be appropriate ensures that the data volume of caching excessively influences the performance of Agent, then can determine the number change of Agent
Mode corresponds to the Agent of number to increase.
As the num=0, there are two types of situations：When the first data volume is not equal to the second data volume in above-mentioned functional relation
When zero, illustrate that the data volume that current time caches in Agent is identical with history quantity amount, then can determine that the number of Agent becomes
More mode is constant；When the first data volume equal to the second data volume is zero in above-mentioned functional relation, illustrate current transmission speed
Rate is larger, the data without caching in Agent, then can determine the number change mode of Agent to delete an Agent；When this
num<When 0, illustrate that the transmission rate of current Agent is very fast, the data volume cached in Agent is less, it is necessary to appropriate reduction
The quantity of Agent then can determine the number change mode of Agent to delete corresponding number in order to avoid causing the waste of resource
Agent。
As it can be seen that by the embodiment of the present invention, the usage quantity situation of current Agent can be learnt in time, and then automatically
The number of current desired Agent is changed, stock number caused by solving the problems, such as flow trough is taken.
It, can be real just like the present invention shown in Fig. 3 in order to better illustrate a kind of data processing method of the embodiment of the present invention
Apply a kind of data processing method system construction drawing of example.The data processing method system construction drawing includes：Treat water conservancy diversion data queue
301st, agency service Agent302, flexible resource pond 303, calculating service service304 and treat water conservancy diversion data receiver 305.
In embodiments of the present invention, obtain and send by agency service Agent302 and treat water conservancy diversion data queue 301, send out
It gives and treats that water conservancy diversion data receiver's 305 treats water conservancy diversion data.In preset period of time, calculate service service304 and obtain respectively
The first data volume for treating water conservancy diversion data and obtain and the current time that current slot caches in agency service Agent302
The second data volume for treating water conservancy diversion data cached in a upper period of Duan Xianglin in Agent.Service service304 is calculated to lead to
The functional relation between the first data volume and the second data volume is crossed, determines the number change mode of current time Agent, Jin Er
Apply in flexible resource pond 303 or cancel one or more agency service Agent302.
It should be noted that the data processing method system construction drawing of the embodiment of the present invention is only comprising at data of the present invention
A kind of system construction drawing of reason method, other include the system construction drawing of data processing method of the present invention, fall within of the invention real
The protection domain of example is applied, is no longer repeated one by one herein.
In order to better illustrate a kind of data processing method of the embodiment of the present invention, a kind of number is disclosed in the present embodiment
According to processing method, as shown in Figure 4.Fig. 4 be the embodiment of the present invention another data processing method flow chart, specific implementation step
It is as follows：
S401, according to preset period of time, what is cached in acquisition current slot in agency service Agent treats water conservancy diversion data
The first data volume；
S402 was obtained in a current slot adjacent upper period, and what is cached in Agent treats the second number of water conservancy diversion data
According to amount；
First data volume and the second data volume are done difference operation by S403, and handling up difference operation result and Agent
Amount does business's computing, obtains the functional relation between the first data volume and the second data volume；
S404 calculates the end value of functional relation；
S405 when end value is positive number, determines the number change mode of Agent to increase the Agent of corresponding number；When
End value is zero and first data volume when being not zero equal to the second data volume, and the number change mode for determining Agent is constant；When
End value is zero and first data volume when equal to the second data volume being zero, determines the number change mode of Agent to delete one
Agent；When end value is negative, the number change mode of Agent is determined to delete the Agent of corresponding number.
S406 changes mode according to the number of Agent, and corresponding change is currently accessed the number of Agent.
It, can be by calculating the required Agent of current time in real time, and then to bullet as it can be seen that by the embodiment of the present invention
Property resource pool application or revocation Agent, number that is dynamic, timely adjusting Agent is realized, so as to avoid flow trough unrestrained
Take resource.
To achieve the above object of the invention, the embodiment of the invention also discloses a kind of data processing equipment, as shown in Figure 5.Fig. 5
For a kind of data processing equipment structure diagram of the embodiment of the present invention, including：
First data volume acquisition module 501, for according to preset period of time, obtaining agency service in current slot
What is cached in Agent treats the first data volume of water conservancy diversion data；
Second data volume acquisition module 502 for obtaining in a current slot adjacent upper period, delays in Agent
That deposits treats the second data volume of water conservancy diversion data；
Change mode determining module 503, for according to the first data volume and the second data volume, determining that the number of Agent becomes
More mode, number change mode include, delete, increase or constant；
Agent changes module 504, and for changing mode according to the number of Agent, corresponding change is currently accessed Agent's
Number.
A kind of data processing equipment provided in an embodiment of the present invention realizes reasonable employment Agent and carries out data water conservancy diversion, subtracts
The waste of resource is lacked.Specifically, the data for treating water conservancy diversion data cached in current time Agent are obtained according to the time cycle
Amount is the first data volume；And obtained in the upper period adjacent with the current slot, what is cached in the Agent treats
The data volume of water conservancy diversion data is the second data volume；By the first data volume and the relation of the second data volume, obtain current desired
The number of Agent, and then the number for changing Agent is corresponded to according to the real-time demands of Agent.As it can be seen that the embodiment of the present invention can
It according to the real-time change of flow, and then determines the number of required Agent, reaches the number that dynamic timely adjusts Agent, and then
Effectively avoid the waste of resource.In addition, the data processing method of the embodiment of the present invention, which can be realized, automatically adjusts current Agent's
Number without manpower intervention, improves work efficiency.
It should be noted that the device of the embodiment of the present invention is using a kind of above-mentioned device of data processing method, then on
All embodiments for stating a kind of data processing method are suitable for the device, and can reach the same or similar advantageous effect.
Optionally, in a kind of embodiment of the data processing equipment of the embodiment of the present invention, described device further includes：
It treats water conservancy diversion data cache module, treats that water conservancy diversion data sending to during recipient, works as acquisition for what will be obtained
When the rate of water conservancy diversion data is more than and sends when the rate of water conservancy diversion data, caching treats water conservancy diversion data in Agent.
Optionally, in a kind of embodiment of the data processing equipment of the embodiment of the present invention, mode determining module is changed
503, including：
Functional relation determination sub-module for the first data volume and the second data volume to be done difference operation, and difference is transported
The handling capacity for calculating result and Agent does business's computing, obtains the functional relation between the first data volume and the second data volume；
Change mode determination sub-module, for according to functional relation, determining that the number of Agent changes mode.
Optionally, in a kind of embodiment of the data processing equipment of the embodiment of the present invention, mode determination sub-module is changed,
Specifically for calculating the end value of functional relation；When end value is positive number, the number change mode for determining Agent is increase pair
Answer the Agent of number；When end value be zero and first data volume be not zero equal to the second data volume when, determine the number of Agent
Change mode is constant；When end value be zero and first data volume equal to the second data volume be zero when, determine that the number of Agent becomes
More mode is one Agent of deletion；When end value is negative, the number change mode of Agent is determined to delete corresponding number
Agent.
To achieve the above object of the invention, the embodiment of the invention discloses a kind of electronic equipment, as shown in Figure 6.Fig. 6 is this hair
The a kind of electronic equipment structure diagram of bright embodiment, it is total including processor 601, communication interface 602, memory 603 and communication
Line 604, wherein, processor 601, communication interface 602, memory 603 complete mutual communication by communication bus 604；
Memory 603, for storing computer program；
Processor 601 during for performing the program stored on memory, realizes following method and step：
According to preset period of time, treat water conservancy diversion data the cached in current slot in agency service Agent is obtained
One data volume；
It obtained in a current slot adjacent upper period, what is cached in Agent treats the second data volume of water conservancy diversion data；
According to the first data volume and the second data volume, the number change mode of Agent is determined, number, which changes mode, to be included,
It deletes, increase or constant；
Number according to Agent changes mode, and corresponding change is currently accessed the number of Agent.
The communication bus 604 that above-mentioned electronic equipment is mentioned can be Peripheral Component Interconnect standard
(PeripheralComponent Interconnect, PCI) bus or expanding the industrial standard structure (Extended
IndustryStandard Architecture, EISA) bus etc..It is total that the communication bus can be divided into address bus, data
Line, controlling bus etc..For ease of representing, only represented in figure with a thick line, it is not intended that an only bus or a species
The bus of type.
Communication interface 602 is for the communication between above-mentioned electronic equipment and other equipment.
Memory 603 can include random access memory (Random Access Memory, RAM), can also include
Nonvolatile memory (Non-Volatile Memory, NVM), for example, at least a magnetic disk storage.Optionally, memory
603 can also be at least one storage device for being located remotely from aforementioned processor 601.
Above-mentioned processor 601 can be general processor, including central processing unit (Central
ProcessingUnit, CPU), network processing unit (Network Processor, NP) etc.；It can also be digital signal processor
(Digital Signal Processing, DSP), application-specific integrated circuit (Application SpecificIntegrated
Circuit, ASIC), field programmable gate array (Field-Programmable Gate Array, FPGA) or other can
Programmed logic device, discrete gate or transistor logic, discrete hardware components.
A kind of electronic equipment provided in an embodiment of the present invention realizes reasonable employment Agent and carries out data water conservancy diversion, reduces
The waste of resource.Specifically, the data volume for treating water conservancy diversion data cached in current time Agent is obtained according to the time cycle, i.e.,
For the first data volume；And obtained in the upper period adjacent with the current slot, what is cached in the Agent treats water conservancy diversion number
According to data volume, be the second data volume；By the first data volume and the relation of the second data volume, current desired Agent is obtained
Number, and then correspond to according to the real-time demands of Agent the number of change Agent.As it can be seen that the embodiment of the present invention can be according to stream
The real-time change of amount, and then determine the number of required Agent, reach dynamic and timely adjust the number of Agent, and then effectively keep away
Exempt from the waste of resource.In addition, the data processing method of the embodiment of the present invention can realize the number for automatically adjusting current Agent,
Without manpower intervention, improve work efficiency.
To achieve the above object of the invention, the embodiment of the invention also discloses a kind of computer readable storage medium, computers
Readable storage medium storing program for executing memory contains computer program, when computer program is executed by processor, realizes any in above-described embodiment
The step of described data processing method.
A kind of computer readable storage medium provided in an embodiment of the present invention realizes reasonable employment Agent and carries out data
Water conservancy diversion reduces the waste of resource.Specifically, water conservancy diversion data are treated according to what is cached in time cycle acquisition current time Agent
Data volume, be the first data volume；And obtained in the upper period adjacent with the current slot, delay in the Agent
That deposits treats the data volume of water conservancy diversion data, is the second data volume；By the first data volume and the relation of the second data volume, worked as
The number of Agent needed for preceding, and then the number for changing Agent is corresponded to according to the real-time demands of Agent.As it can be seen that the present invention is real
Applying example can be according to the real-time change of flow, and then determines the number of required Agent, reaches the number that dynamic timely adjusts Agent
Mesh, and then effectively avoid the waste of resource.In addition, the data processing method of the embodiment of the present invention can realize that automatically adjustment is current
The number of Agent without manpower intervention, improves work efficiency.
To achieve the above object of the invention, the embodiment of the invention also discloses it is a kind of comprising instruction computer program product,
When run on a computer so that computer performs any data processing method in above-described embodiment.
It is provided in an embodiment of the present invention it is a kind of comprising instruction computer program product, realize reasonable employment Agent into
Row data water conservancy diversion, reduces the waste of resource.Specifically, wait to lead according to what is cached in time cycle acquisition current time Agent
The data volume of flow data is the first data volume；And obtained in the upper period adjacent with the current slot, it should
What is cached in Agent treats the data volume of water conservancy diversion data, is the second data volume；Pass through the pass of the first data volume and the second data volume
System obtains the number of current desired Agent, and then the number of change Agent is corresponded to according to the real-time demands of Agent.As it can be seen that
The embodiment of the present invention can be according to the real-time change of flow, and then determines the number of required Agent, reaches dynamic and timely adjusts
The number of Agent, and then effectively avoid the waste of resource.In addition, the data processing method of the embodiment of the present invention can be realized automatically
The current Agent of adjustment number, without manpower intervention, improve work efficiency.
In the above-described embodiments, can come wholly or partly by software, hardware, firmware or its any combination real
It is existing.When implemented in software, can entirely or partly realize in the form of a computer program product.Computer program product
Including one or more computer instructions.When loading on computers and performing computer program instructions, all or part of real estate
Raw flow or function according to the embodiment of the present invention.Computer can be all-purpose computer, special purpose computer, computer network,
Or other programmable devices.Computer instruction can be stored in a computer-readable storage medium or from a computer
Readable storage medium storing program for executing to another computer readable storage medium transmit, for example, computer instruction can from a web-site,
Computer, server or data center by wired (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as
Infrared, wireless, microwave etc.) mode is transmitted to another web-site, computer, server or data center.Computer
Readable storage medium storing program for executing can be that any usable medium that computer can access either includes one or more usable medium collection
Into the data storage devices such as server, data center.Usable medium can be magnetic medium, (for example, floppy disk, hard disk, magnetic
Band), optical medium (for example, DVD) or semiconductor medium (such as solid state disk Solid State Disk (SSD)) etc..
It should be noted that herein, relational terms such as first and second and the like are used merely to a reality
Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation
In any this actual relation or order.Moreover, term " comprising ", "comprising" or its any other variant are intended to
Non-exclusive inclusion, so that process, method, article or equipment including a series of elements not only will including those
Element, but also including other elements that are not explicitly listed or further include as this process, method, article or equipment
Intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that
Also there are other identical elements in process, method, article or equipment including element.
Each embodiment in this specification is described using relevant mode, identical similar portion between each embodiment
Point just to refer each other, and the highlights of each of the examples are difference from other examples.Especially for device,
For electronic equipment, computer readable storage medium and computer program product embodiments, since it is substantially similar to method
Embodiment, so description is fairly simple, the relevent part can refer to the partial explaination of embodiments of method.
The foregoing is merely a prefered embodiment of the invention, is not intended to limit the scope of the present invention.It is all in this hair
Any modifications, equivalent replacements and improvements are made within bright spirit and principle, are all contained in protection scope of the present invention.
Claims (10)
1. a kind of data processing method, which is characterized in that including：
According to preset period of time, the first number for treating water conservancy diversion data cached in current slot in agency service Agent is obtained
According to amount；
It obtained in a current slot adjacent upper period, what is cached in the Agent treats the second number of water conservancy diversion data
According to amount；
According to first data volume and second data volume, determine that the number of the Agent changes mode, the number becomes
More mode includes, and deletes, increases or constant；
Number according to the Agent changes mode, and corresponding change is currently accessed the number of Agent.
2. according to the method described in claim 1, it is characterized in that, caching treats the process bag of water conservancy diversion data in the Agent
It includes：
Acquisition is treated into water conservancy diversion data sending to during recipient, waits to lead when obtaining the rate for the treatment of water conservancy diversion data and being more than to send
During the rate of flow data, caching treats water conservancy diversion data in the Agent.
It is 3. according to the method described in claim 1, it is characterized in that, described according to first data volume and second data
Amount determines the number change mode of the Agent, including：
First data volume and second data volume are done into difference operation, and by the difference operation result with it is described
The handling capacity of Agent does business's computing, obtains the functional relation between first data volume and second data volume；
According to the functional relation, determine that the number of the Agent changes mode.
4. according to the method described in claim 3, it is characterized in that, described according to the functional relation, determine the Agent's
Number changes mode, including：
Calculate the end value of the functional relation；
When the end value is positive number, the number change mode of the Agent is determined to increase the Agent of corresponding number；
Alternatively, when the end value is zero and first data volume is not zero equal to second data volume, determine described
The number change mode of Agent is constant；
Alternatively, when the end value is zero and first data volume equal to second data volume is zero, determine described
The number change mode of Agent is one Agent of deletion；
Alternatively, when the end value is negative, the number change mode of the Agent is determined to delete corresponding number
Agent。
5. a kind of data processing equipment, which is characterized in that including：
First data volume acquisition module, for according to preset period of time, obtaining in current slot and delaying in agency service Agent
That deposits treats the first data volume of water conservancy diversion data；
Second data volume acquisition module for obtaining in a current slot adjacent upper period, delays in the Agent
That deposits treats the second data volume of water conservancy diversion data；
Change mode determining module, for according to first data volume and second data volume, determining that the Agent's is a
Number change modes, the number change mode include, and delete, increase or constant；
Agent changes module, and for changing mode according to the number of the Agent, corresponding change is currently accessed of Agent
Number.
6. device according to claim 5, which is characterized in that described device further includes：
It treats water conservancy diversion data cache module, treats that water conservancy diversion data sending to during recipient, is waited to lead when obtaining for what will be obtained
The rate of flow data, which is more than, to be sent when the rate of water conservancy diversion data, and caching treats water conservancy diversion data in the Agent.
7. device according to claim 5, which is characterized in that the change mode determining module, including：
Functional relation determination sub-module, for first data volume and second data volume to be done difference operation, and by institute
It states difference operation result and the handling capacity of the Agent does business's computing, obtain first data volume and second data volume
Between functional relation；
Change mode determination sub-module, for according to the functional relation, determining that the number of the Agent changes mode.
8. device according to claim 7, which is characterized in that the change mode determination sub-module, specifically for calculating
The end value of the functional relation；When the end value is positive number, the number change mode for determining the Agent is increase pair
Answer the Agent of number；Alternatively, when the end value is zero and first data volume is not zero equal to second data volume
When, the number change mode for determining the Agent is constant；Alternatively, when the end value is zero and first data volume etc.
When second data volume is zero, the number change mode of the Agent is determined to delete an Agent；Alternatively, when described
When end value is negative, the number change mode of the Agent is determined to delete the Agent of corresponding number.
9. a kind of electronic equipment, which is characterized in that including processor, communication interface, memory and communication bus, wherein, it is described
Processor, the communication interface, the memory complete mutual communication by the communication bus；
The memory, for storing computer program；
The processor during for performing the program stored on the memory, realizes any sides of claim 1-4
Method step.
10. a kind of computer readable storage medium, which is characterized in that the computer readable storage medium memory contains computer
Program realizes claim 1-4 any method and steps when the computer program is executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711274747.6A CN108121507B (en) | 2017-12-06 | 2017-12-06 | Data processing method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711274747.6A CN108121507B (en) | 2017-12-06 | 2017-12-06 | Data processing method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108121507A true CN108121507A (en) | 2018-06-05 |
| CN108121507B CN108121507B (en) | 2021-04-02 |
Family
ID=62229733
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711274747.6A Active CN108121507B (en) | 2017-12-06 | 2017-12-06 | Data processing method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108121507B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111552711A (en) * | 2020-04-27 | 2020-08-18 | 中国银行股份有限公司 | Streaming data processing method and device and electronic equipment |
| CN112632163A (en) * | 2020-12-18 | 2021-04-09 | 平安普惠企业管理有限公司 | Big data report exporting method and related equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101184031A (en) * | 2007-12-03 | 2008-05-21 | 浙江大学 | Client terminal active load balancing method facing to network multimedia transmission service |
| CN101198034A (en) * | 2007-12-29 | 2008-06-11 | 北京航空航天大学 | Network video monitoring system and its data exchanging method |
| KR20100048491A (en) * | 2008-10-31 | 2010-05-11 | 에스케이텔레콤 주식회사 | System and method for multimedia streaming of distributed contents using mobile agent |
| CN105933391A (en) * | 2016-04-11 | 2016-09-07 | 青岛海信传媒网络技术有限公司 | Node capacity expansion method, device and system |
| CN106202370A (en) * | 2016-07-07 | 2016-12-07 | 乐视控股（北京）有限公司 | A kind of method and system carrying out water conservancy diversion in cloud object stores |
| CN106911778A (en) * | 2017-02-27 | 2017-06-30 | 网宿科技股份有限公司 | A kind of flow bootstrap technique and system |
| CN107295040A (en) * | 2016-03-31 | 2017-10-24 | 阿里巴巴集团控股有限公司 | Data uploading method, apparatus and system |
-
2017
- 2017-12-06 CN CN201711274747.6A patent/CN108121507B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101184031A (en) * | 2007-12-03 | 2008-05-21 | 浙江大学 | Client terminal active load balancing method facing to network multimedia transmission service |
| CN101198034A (en) * | 2007-12-29 | 2008-06-11 | 北京航空航天大学 | Network video monitoring system and its data exchanging method |
| KR20100048491A (en) * | 2008-10-31 | 2010-05-11 | 에스케이텔레콤 주식회사 | System and method for multimedia streaming of distributed contents using mobile agent |
| CN107295040A (en) * | 2016-03-31 | 2017-10-24 | 阿里巴巴集团控股有限公司 | Data uploading method, apparatus and system |
| CN105933391A (en) * | 2016-04-11 | 2016-09-07 | 青岛海信传媒网络技术有限公司 | Node capacity expansion method, device and system |
| CN106202370A (en) * | 2016-07-07 | 2016-12-07 | 乐视控股（北京）有限公司 | A kind of method and system carrying out water conservancy diversion in cloud object stores |
| CN106911778A (en) * | 2017-02-27 | 2017-06-30 | 网宿科技股份有限公司 | A kind of flow bootstrap technique and system |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111552711A (en) * | 2020-04-27 | 2020-08-18 | 中国银行股份有限公司 | Streaming data processing method and device and electronic equipment |
| CN112632163A (en) * | 2020-12-18 | 2021-04-09 | 平安普惠企业管理有限公司 | Big data report exporting method and related equipment |
| CN112632163B (en) * | 2020-12-18 | 2024-05-17 | 北京神州数码云科信息技术有限公司 | Big data report export method and related equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108121507B (en) | 2021-04-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230283681A1 (en) | System and method for throttling service requests having non-uniform workloads | |
| CN110851529B (en) | Calculation power scheduling method and related equipment | |
| US8953453B1 (en) | System and method for throttling service requests using work-based tokens | |
| CN112165691B (en) | Content delivery network scheduling method, device, server and medium | |
| CN109076024A (en) | data control method and terminal device | |
| US9405725B2 (en) | Writing message to controller memory space | |
| US20210377340A1 (en) | Using reinforcement learning to scale queue-based services | |
| US11799901B2 (en) | Predictive rate limiting system for cloud computing services | |
| CN108121507A (en) | A kind of data processing method, device and electronic equipment | |
| US10609206B1 (en) | Auto-repairing mobile communication device data streaming architecture | |
| WO2020142867A1 (en) | Traffic shaping method and related device | |
| WO2020034729A1 (en) | Data processing method, related device, and computer storage medium | |
| CN105978821B (en) | The method and device that network congestion avoids | |
| CN108234149A (en) | Network request management method and device | |
| US8429348B2 (en) | Method and mechanism for delaying writing updates to a data cache | |
| US10540217B2 (en) | Message cache sizing | |
| WO2015188495A1 (en) | Data transmission method and device | |
| US8379525B2 (en) | Techniques to support large numbers of subscribers to a real-time event | |
| CN109981487B (en) | Data scheduling method and device, electronic equipment and readable storage medium | |
| CN107689969A (en) | A kind of determination method and device of cache policy | |
| CN113590666B (en) | Data caching method, system, equipment and computer medium in AI cluster | |
| WO2021244441A1 (en) | Service configuration method and apparatus therefor | |
| Kang et al. | Evaluating DDS, MQTT, and ZeroMQ Under Different IoT Traffic Conditions | |
| WO2020221300A1 (en) | Protocol data unit session processing method and apparatus, and electronic device | |
| US11632319B2 (en) | Processing device and moving method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |