CN107168681A - A kind of real-time task processing method and processing device - Google Patents
A kind of real-time task processing method and processing device Download PDFInfo
- Publication number
- CN107168681A CN107168681A CN201710210712.XA CN201710210712A CN107168681A CN 107168681 A CN107168681 A CN 107168681A CN 201710210712 A CN201710210712 A CN 201710210712A CN 107168681 A CN107168681 A CN 107168681A
- Authority
- CN
- China
- Prior art keywords
- aggregate function
- operation object
- stream processing
- data packet
- task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
- G06F9/30003—Arrangements for executing specific machine instructions
- G06F9/3005—Arrangements for executing specific machine instructions to perform operations for flow control
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
- G06F9/30098—Register arrangements
- G06F9/30101—Special purpose registers
Abstract
The embodiment of the invention discloses a kind of real-time task processing method and processing device, the real-time task processing method includes：When Stream Processing task currently performs aggregate function, the encapsulated data packet of the aggregate function of the Stream Processing tasks carrying stored is loaded, wherein, the encapsulated data packet includes：Operation object, the functional relation of the operation object of the aggregate function of the Stream Processing task；According to the aggregate function and the encapsulated data packet, to the current value of the operation object of the aggregate function, according to the functional relation of the operation object, the aggregate function operation is performed.The embodiment of the present invention, which is realized, to be managed real time data processing status of processes, and is easily updated operation, the time of reduction system processing and the workload of developer, improves the development efficiency of real-time task.
Description
Technical field
The present invention relates to big data processing technology field, more particularly to a kind of real-time task processing method and processing device.
Background technology
With the lasting rise of the prosperity of Internet industry, especially mobile Internet, the data that machine, user produce are got over
Come more, while all kinds of big data processing systems have also been expedited the emergence of, such as Hadoop, Spark, Storm.These systems are a bit
For the batch processing engine of batch processing task, such as Hadoop MapReduce, some are for the real-time of stream data processing
Handle engine, such as Storm, also some systems can realize simultaneously batch processing task and stream data processing real-time processing appoint
Business, such as Spark.
The big data processing system of batch processing is that all data are just already prepared to when starting to calculate, etc.
It is to be entered to calculate；And stream data processing engine, data are not fully prepared at the very start, but pushing away over time
Move, engender, as the user behaviors log of user, be characterized in as flowing water, be not once to come, but bit by bit " flow "
Come over.And the data that " stream " comes are also bit by bit to be processed in real-time system processing.The main distinction of two types task is：
Batch processing task has global data, easily carries out the converging operation of data；And the real-time processing of stream data does not possess entirely
Office data, the intermediateness in polymerization.The real-time processing engine of existing stream data processing, is handled in real time in streaming task
When, the processing procedure and function logics relation of different time or different conditions for same operation are, it is necessary to which developer is real
Shi Chongxin writes.Error number in such as one log stream, in a period of time error number be one constantly cumulative numeral, it is necessary to
It is temporarily stored in external storage database, as MySQL, Redis etc..Within statistics a period of time first time during error number, open
The program that originator is write the logical function of associative operation and is connected with database, is deposited using program statistics external storage system
Error number in the current time of storage, within statistics a period of time next time during error number, developer needs to write related behaviour again
The logical function of work and the program being connected with database, using the program count again external storage system storage it is current when
Interior error number.
The method that existing streaming task is handled in real time, different time for same or similar logical relationship operations or
The processing procedure of different conditions, adds the time of system processing and adds the workload of developer, whole development efficiency
It is not high.
The content of the invention
The purpose of the embodiment of the present invention is to provide a kind of real-time task processing method and processing device, provided with realizing to real-time
The condition managing of data handling procedure, and easily it is updated operation, the time of reduction system processing and the work of developer
Measure, improve the development efficiency of real-time task.Concrete technical scheme is as follows：
To achieve the above object of the invention, the embodiment of the invention discloses a kind of real-time task processing method, including：
When Stream Processing task currently performs aggregate function, the institute of the Stream Processing tasks carrying stored is loaded
The encapsulated data packet of aggregate function is stated, wherein, the encapsulated data packet includes：The aggregate function of the Stream Processing task
Operation object, the functional relation of the operation object；
According to the aggregate function and the encapsulated data packet, to the current of the operation object of the aggregate function
Numerical value, according to the functional relation of the operation object, performs the aggregate function operation.
Alternatively, it is described currently perform aggregate function in Stream Processing task when, load the Stream Processing task and hold
Before the encapsulated data packet of the capable aggregate function, methods described also includes：
When the Stream Processing task performs the aggregate function first, to the operation object of the aggregate function
Numerical value, operated according to the functional relation of the operation object, the aggregate function for performing the Stream Processing task；
At the end of the Stream Processing task performs the aggregate function first, by the Stream Processing tasks carrying
The operation object, the functional relation of the operation object of the aggregate function, are encapsulated in the encapsulated data packet；
Store the encapsulated data packet.
Alternatively, in the numerical value of the operation object to the aggregate function, closed according to the function of the operation object
It is, performs after the operation of the aggregate function of the Stream Processing task, described method also includes：
By to the numerical value of the operation object of the aggregate function, institute is performed according to the functional relation of the operation object
The aggregate function operation of Stream Processing task is stated, the result of the numerical value of the operation object is obtained, is stored in database.
Alternatively, described according to the aggregate function and the encapsulated data packet, to described in the aggregate function
The current value of operation object, it is described before the functional relation of the operation object, the execution aggregate function operation
Method also includes：
The accumulative Stream Processing task performs the aggregate function and currently performs institute to the Stream Processing task first
Updating the data for the state the aggregate function period, operation object, the Stream Processing times is accumulated to by described update the data
Business perform first the aggregate function the operation object it is described numerically, obtain the Stream Processing task and currently perform
The current value of the operation object of the aggregate function；
Load the encapsulated data packet of the aggregate function of the Stream Processing task, and the operation object
The current value；
Accordingly, operation object, the operation object of the aggregate function according in the encapsulated data packet
Functional relation, to the current value of the operation object of the aggregate function, according to the functional relation of the operation object,
Corresponding operation is performed, including：
By the encapsulated data packet, the operation object, described of the aggregate function in the encapsulated data packet is obtained
The functional relation of operation object；
To the current value of the operation object of the aggregate function, closed according to the function of the operation object
It is, performs the aggregate function operation.
Alternatively, it is described according to the aggregate function and the encapsulated data packet, to the behaviour of the aggregate function
Make the current value of object, according to the functional relation of the operation object, perform the aggregate function operation, including：
By the encapsulated data packet, the operation object, described of the aggregate function in the encapsulated data packet is obtained
The functional relation of operation object, and get in database the current value of the operation object of the aggregate function；
To the current value of the operation object of the aggregate function, closed according to the function of the operation object
It is, performs the aggregate function operation.
To achieve the above object of the invention, the embodiment of the invention also discloses a kind of real-time task processing unit, including：
State load module, for when Stream Processing task currently performs aggregate function, loading the stream stored
The encapsulated data packet of the aggregate function of formula processing tasks carrying, wherein, the encapsulated data packet includes：The Stream Processing
Operation object, the functional relation of the operation object of the aggregate function of task；
Performing module, for according to the aggregate function and the encapsulated data packet, to described in the aggregate function
The current value of operation object, according to the functional relation of the operation object, performs the aggregate function operation.
Alternatively, described device also includes：
First performing module, for when the Stream Processing task performs the aggregate function first, to the polymerization
The numerical value of the operation object of function, according to the institute of the functional relation of the operation object, the execution Stream Processing task
State aggregate function operation；
Package module, at the end of performing the aggregate function first in the Stream Processing task, by the streaming
The operation object, the functional relation of the operation object of the aggregate function of tasks carrying are handled, the envelope is encapsulated in
Fill in packet；
First memory module, for storing the encapsulated data packet.
Alternatively, described device also includes：
Second memory module, for by the numerical value of the operation object of the aggregate function, according to the operation pair
The functional relation of elephant performs the aggregate function operation of the Stream Processing task, obtains the knot of the numerical value of the operation object
Really, it is stored in database.
Alternatively, described device also includes：
Accumulation module, the aggregate function is performed to the Stream Processing times for adding up the Stream Processing task first
Business currently perform it is the aggregate function period, the operation object update the data, described update the data is accumulated to institute
State Stream Processing task perform first the aggregate function the operation object it is described numerically, obtain the Stream Processing
Task currently performs the current value of the operation object of the aggregate function；
Packet load-on module, the encapsulation of data of the aggregate function for loading the Stream Processing task
Bag, and the operation object the current value；
Accordingly, the performing module, including：
Acquisition submodule, for by the encapsulated data packet, obtaining the aggregate function in the encapsulated data packet
Operation object, the functional relation of the operation object；
Implementation sub-module, for the current value of the operation object to the aggregate function, according to the behaviour
Make the functional relation of object, perform the aggregate function operation.
Alternatively, the performing module, is additionally operable to, by the encapsulated data packet, obtains in the encapsulated data packet
The functional relation of the operation object of the aggregate function, the operation object, and get in database the polymerization letter
The current value of several operation objects；To the current value of the operation object of the aggregate function, according to the behaviour
Make the functional relation of object, perform the aggregate function operation.
The embodiment of the invention discloses a kind of real-time task processing method and processing device, by by the polymerization of Stream Processing task
The functional relation packing sealed storage of the operation object of function and the operation object, when performing same Stream Processing in real time
During the aggregate function of task, directly obtain the operation object of the aggregate function of the Stream Processing task stored and be somebody's turn to do
There is provided the functional relation of the operation object for same operation function and the operation object for the functional relation of operation object
The method of logical relation encapsulation caching.Do not need each function interface with regard to same problem of developer repeatedly to write reading and writing data to connect
Mouthful, reduce the time of the real time data processing business for same operation and improve whole development efficiency.In addition, will be by
Functional relation according to operation object performs the aggregate function operation of Stream Processing task, obtains the result of the numerical value of operation object,
It is stored in database, realizes the storage method of the various data modes of real-time task processing.The embodiment of the present invention is finally real
Show and real time data processing status of processes has been managed, and be easily updated operation, improved the development efficiency of real-time task.
Certainly, implementing any product or method of the present invention must be not necessarily required to while reaching all the above advantage.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is the accompanying drawing used required in technology description to be briefly described, it should be apparent that, drawings in the following description are only this
Some embodiments of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can be with
Other accompanying drawings are obtained according to these accompanying drawings.
Fig. 1 is a kind of real-time task process flow figure of the embodiment of the present invention；
Fig. 2 is a kind of encapsulated data packet implementation process figure of real-time task processing method of the embodiment of the present invention；
Fig. 3 is a kind of execution aggregate function operational flowchart of real-time task processing method of the embodiment of the present invention；
Fig. 4 is a kind of real-time task processing unit figure of the embodiment of the present invention；
Fig. 5 is another real-time task processing unit figure of the embodiment of the present invention；
Fig. 6 implements figure for a kind of real-time task processing unit of the embodiment of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation is described, it is clear that described embodiment is only a part of embodiment of the invention, rather than whole embodiments.It is based on
Embodiment in the present invention, it is every other that those of ordinary skill in the art are obtained under the premise of creative work is not made
Embodiment, belongs to the scope of protection of the invention.
To achieve the above object of the invention, the embodiment of the invention discloses a kind of real-time task processing method, as shown in Figure 1.
Fig. 1 is a kind of real-time task process flow figure of the embodiment of the present invention, including：
S101, when Stream Processing task currently performs aggregate function, the Stream Processing tasks carrying that loading has been stored
The encapsulated data packet of aggregate function, wherein, encapsulated data packet includes：The operation object of the aggregate function of Stream Processing task, behaviour
Make the functional relation of object；
Specifically, when big data handles streaming task, it is necessary to handle the different time and different conditions of same task
Under task.The embodiment of the invention discloses by big data handle streaming tasks carrying same type, identical function polymerization letter
Number operation is stored in database, when the aggregate function operation of same type identical function is performed in next time, is added from database
Carry stored logical relation.
Aggregate function is that the function for calculating and returning single value is performed to a class value, and it is often with selecting select sentences
Packet group by clauses be used together.Aggregate function includes：The average value in designated groups is returned to, null value is ignored；Return
The quantity of project in designated groups；Return to the maximum for specifying data；Return to the minimum value for specifying data；Return and specify data
Be only used for sum, null value is ignored；Return to the number of entry in designated groups；An additional row are produced, when with piecemeal
Cube operators or curling rollup operators are when adding line, and output valve is 1, when added it is capable be not by piecemeal cube or volume
When bent rollup is produced, output valve is 0；The binary system check value calculated the row in table or expression list is returned to, for examining
Survey the change of row in table；The check value for specifying data is returned to, null value is ignored；Return on the row of table or on expression list
The check value of calculating, for generating hash index；Return to the SS deviation of all values in given expression formula；Return to given table
Up to the filling SS deviation of all values in formula；Return to the statistical variance of all values in given expression formula；Return to given table
The statistical variance of the filling of all values up in formula.
Specific practice is, when Stream Processing task currently performs aggregate function, to load what is stored first from database
The letter of the aggregate function operation object and operation object is packaged with the encapsulated data packet of the aggregate function, the encapsulated data packet
Number relation.The encapsulated data packet is decapsulated to the functional relation for getting the aggregate function operation object and operation object.
S102, according to aggregate function and encapsulated data packet, to the current value of the operation object of aggregate function, according to behaviour
Make the functional relation of object, perform aggregate function operation.
Specifically, the current value of the current operation object for performing aggregate function is obtained in database, is obtained by above-mentioned
After the encapsulated data packet decapsulation arrived, the aggregate function operation object and the functional relation of operation object of acquisition, according to
The functional relation of the aggregate function operation object, to the current value of the operation object of aggregate function, performs specific polymerization letter
Number operation.Aggregate function operation can be the one or several kinds in the above-mentioned aggregate function enumerated.
The embodiment of the invention discloses a kind of real-time task processing method, by by the aggregate function of Stream Processing task
The functional relation packing sealed storage of operation object and the operation object, when performing same Stream Processing task in real time
During aggregate function, operation object and operation pair of the aggregate function of the Stream Processing task stored are directly obtained
There is provided the logic of the operation object for same operation function and the functional relation of operation object pass for the functional relation of elephant
The method of system's encapsulation caching.Do not need developer to repeat to write data read and write interface with regard to the function interface of same problem every time, drop
The time of the low real time data processing business for same operation and improve whole development efficiency.The embodiment of the present invention is most
Realize eventually and real time data processing status of processes is managed, and be easily updated operation, improve the exploitation of real-time task
Efficiency.
Alternatively, in a kind of embodiment of the real-time task processing method of the embodiment of the present invention, in Stream Processing task
During current execution aggregate function, before the encapsulated data packet for the aggregate function for loading Stream Processing tasks carrying, method also includes,
The encapsulated data packet implementation process of real-time task processing method, as shown in Figure 2.Fig. 2 appoints in real time for one kind of the embodiment of the present invention
The encapsulated data packet implementation process figure for processing method of being engaged in, including：
S201, when Stream Processing task performs aggregate function first, to the numerical value of the operation object of aggregate function, according to
The functional relation of operation object, the aggregate function operation for performing Stream Processing task；
Specifically, when Stream Processing task performs aggregate function first, according to the logical relation for the aggregate function write,
The operation object for performing the aggregate function first and the functional relation of the aggregate function operation object are got, in database
The numerical value of the operation object is obtained, the numerical value of the operation object is performed according to the functional relation of the aggregate function operation object should
Aggregate function is operated.
For example, in the average value during Stream Processing task performs a web site traffic designated groups first, using for the first time
The logical relation function write, gets the numerical value of the web site traffic in database, according to the logical relation function, to the net
The numerical value for business of standing performs the operation averaged.
S202, at the end of Stream Processing task performs aggregate function first, by the polymerization letter of Stream Processing tasks carrying
Several operation object, the functional relation of operation object, are encapsulated in encapsulated data packet；
Specifically, at the end of above-mentioned Stream Processing task performs aggregate function first, by the Stream Processing tasks carrying
Aggregate function operation object, the functional relation of operation object, an organic whole is formed, by data and peration data
Source code is organically combined, and is formed " class ", final to realize the operation pair of the aggregate function of the Stream Processing tasks carrying
As the functional relation of, operation object is packaged into encapsulated data packet.
For example, at the end of the average value during Stream Processing task performs a web site traffic designated groups first, will write
The operation object that the logical relation function averaged is performed to the numerical value of the web site traffic, the functional relation of operation object seals
In encapsulated data packet.
S203, storage enclosure packet.
Specifically, the encapsulated data packet of the aggregate function of above-mentioned formation is stored in database, the database can be use
The customized database for storing the aggregate function logical relation in family, or MySQL, the Redis commonly used etc., specifically can basis
Depending on performance.
It can be seen that, in a kind of embodiment of the real-time task processing method of the embodiment of the present invention, by some conventional polymerizations
Function operation is packaged so that when Stream Processing task performs same polymeric function in real time, it is not necessary to manipulate external number by hand
According to when being operated using identical aggregate function next time, directly obtaining the packaged encapsulated data packet, perform corresponding poly-
Close function operation.There is provided the operation object and the logical relation of the functional relation of the operation object for same operation function
The method for encapsulating caching so that each Stream Processing task has corresponding state therewith when performing same polymeric function in real time
Association performs the storage of same polymeric function result in real time there is provided encapsulated data packet and Stream Processing task so that conveniently look into
Look for real-time streaming to handle task and maintain the institute of Stream Processing task stateful.
Alternatively, in a kind of embodiment of the real-time task processing method of the embodiment of the present invention, to aggregate function
The numerical value of operation object, after the functional relation of operation object, the aggregate function operation of execution Stream Processing task, method
Also include：
By to the numerical value of the operation object of aggregate function, Stream Processing task is performed according to the functional relation of operation object
Aggregate function is operated, and is obtained the result of the numerical value of operation object, is stored in database.
Specifically, at the end of when Stream Processing task performs aggregate function first, by the operation obtained in database
Object numerical value, the aggregate function for performing Stream Processing task according to the functional relation of operation object is operated, the knot after being operated
Really, it is stored in database.
For example, when Stream Processing task performs a web site traffic and specifies the maximum of data first, using for the first time
The logical relation function write, gets the maximum of period or the time point web site traffic in database, by acquisition
Maximum is inputted, and the maximum of acquisition is stored in database again.When next time upon execution between section or time point
When the web site traffic specifies the maximum of data, it is only necessary to search the Stream Processing task and perform the specified number of the web site traffic first
According to maximum terminate after to current point in time this period maximum, then by the execution first of preservation, the web site traffic refers to
The maximum of fixed number evidence is made comparisons, and just can show that total period or the time point web site traffic specify the maximum of data.
It can be seen that, in a kind of embodiment of the real-time task processing method of the embodiment of the present invention, by will be to aggregate function
Operation object numerical value, according to operation object functional relation perform Stream Processing task aggregate function operate, grasped
The result for making the numerical value of object is stored in database, when Stream Processing task performs aggregate function again, can directly be obtained
Stream Processing task performs the result of the numerical value of the operation object of aggregate function first, can first be performed with Stream Processing task poly-
Terminate when closing function as time point, calculate the number for performing the aggregate function period after the time point again to the Stream Processing task
According to the time of reduction processing data and the efficiency for improving real-time task processing method entirety.In addition, will be according to operation object
Functional relation perform Stream Processing task aggregate function operation, obtain the result of the numerical value of operation object, be stored in data
In storehouse, the storage method of the various data modes of real-time task processing is realized.
Alternatively, in a kind of embodiment of the real-time task processing method of the embodiment of the present invention, according to aggregate function
And encapsulated data packet, to the current value of the operation object of aggregate function, according to the functional relation of operation object, perform polymerization
Before function operation, method also includes：
Accumulative Stream Processing task performs aggregate function and currently performs the aggregate function period to Stream Processing task first
, operation object update the data, will update the data and be accumulated to the operation object that Stream Processing task performs aggregate function first
Numerically, the current value that Stream Processing task currently performs the operation object of aggregate function is obtained；
Specifically, aggregate function is performed as starting point using Stream Processing task first, currently performed with Stream Processing task
The aggregate function is end point, obtains a period.The loss for adding up the period in database handles the behaviour of task
Make updating the data for object, obtain the period, operation object the amount of updating the data.If the Stream Processing task of the period
The data for performing aggregate function are in incremental form, and Stream Processing task is performed to the number of the operation object of the aggregate function first
It is worth for radix, the currently execution aggregate function period, the operation object amount of updating the data that will obtain is added to the Stream Processing
Task performs the operation object of the aggregate function numerically first, obtains the Stream Processing task and currently performs the aggregate function
Operation object current value.If the form that the data of the Stream Processing tasks carrying aggregate function of the period taper off,
The numerical value of the operation object of the aggregate function is performed first with Stream Processing task, when subtracting obtained current execution aggregate function
Between section, the operation object amount of updating the data, obtain the operation object that the Stream Processing task currently performs the aggregate function
Current value.
For example, when needing to polymerize some index in Stream Processing task, such as, and scan text data flow
In error-critical word, and sum, the error-critical word summation occurred in system gone out with real-time statistics.It is first with Stream Processing task
The error-critical word summation occurred in secondary execution statistical system is starting point, is currently performed with Stream Processing task in statistical system
The error-critical word summation of generation is end point, obtains a period.The mistake pass occurred in the system for adding up the period
Key word is updated the data, the amount of updating the data of the error-critical word occurred in the system for obtaining the period.Stream Processing is appointed
It is radix that business performs the error-critical word summation occurred in statistical system first, and the mistake occurred in the system of the period is closed
The amount of updating the data of key word is added to the Stream Processing task and performed first in the error-critical word summation occurred in statistical system,
Obtain the current value that Stream Processing task currently performs the error-critical word occurred in statistical system.
Load Stream Processing task aggregate function encapsulated data packet, and operation object current value；
Specifically, by the encapsulated data packet of the aggregate function stored in database, and the current value of operation object adds
It is downloaded in real-time task actuator.
For example, occur in real-time statistics system error-critical word summation when, held first in above-mentioned Stream Processing task
The encapsulated data packet of the error-critical word summation occurred in row statistical system, and Stream Processing task obtained above are currently held
The current value of the error-critical word occurred in row statistical system is loaded into real-time task actuator.
Accordingly, according to the operation object of the aggregate function in encapsulated data packet, the functional relation of operation object, to polymerization
The current value of the operation object of function, according to the functional relation of operation object, performs corresponding operation, including：
By encapsulated data packet, obtain the operation object of the aggregate function in encapsulated data packet, the function of operation object and close
System；
Specifically, the encapsulated data packet of loading is decapsulated, gets the behaviour of the aggregate function in the encapsulated data packet
Make object, the functional relation of operation object.
For example, occur in real-time statistics system error-critical word summation when, by the Stream Processing task of loading first
The encapsulated data packet decapsulation of the error-critical word summation occurred in statistical system is performed, gets in the execution statistical system and sends out
The operation object of the error-critical word occurred in the encapsulated data packet of raw error-critical word summation in statistical system, and to system
The statistical function of the operation object of the error-critical word occurred in meter systems.
To the current value of the operation object of aggregate function, according to the functional relation of operation object, aggregate function behaviour is performed
Make.
Specifically, in the function of the above-mentioned operation object and operation object for obtaining the Stream Processing task aggregation function
Relation, to the current value of the operation object of the aggregate function, according to the letter of the Stream Processing task aggregation function operation object
Number relation, performs aggregate function operation.
For example, occur in real-time statistics system error-critical word summation when, the mistake occurred in statistical system is obtained
By mistake the operation object of keyword, at the statistical function of the operation object of error-critical word and streaming that occur in statistical system
Reason task is currently performed after the current value of the error-critical word occurred in statistical system, according to the mistake occurred in the statistical system
The statistical function of the operation object of keyword is missed, to the current current value for performing the error-critical word occurred in statistical system.
Statistical operation is performed, the error-critical word summation occurred in current system is obtained.
It can be seen that, aggregate function is performed to Stream Processing task by accumulative Stream Processing task first and currently performs polymerization letter
The number period, operation object update the data, will update the data and be accumulated to Stream Processing task and perform aggregate function first
Operation object numerically, obtains the current value that Stream Processing task currently performs the operation object of aggregate function, is easy to obtain
The whole period data renewal amount, it is only necessary to calculate accumulative Stream Processing task and perform aggregate function first at streaming
Reason task currently performs the aggregate function period, the operation object amount of calculation for updating the data, reducing the whole period, improves
Real-time streaming handles the computational efficiency of task.
Alternatively, in a kind of embodiment of the real-time task processing method of the embodiment of the present invention, according to aggregate function with
And encapsulated data packet, to the current value of the operation object of aggregate function, according to the functional relation of operation object, perform polymerization letter
Number operation, as shown in Figure 3.Fig. 3 grasps flow for a kind of execution aggregate function of real-time task processing method of the embodiment of the present invention
Figure, including：
S301, by encapsulated data packet, obtains operation object, the letter of operation object of aggregate function in encapsulated data packet
Number relations, and get in database the current value of the operation object of aggregate function；
Specifically, unsealed by the encapsulated data packet that will be loaded into actuator, obtain the polymerization letter in encapsulated data packet
Several operation object, the functional relation of operation object.Stream Processing task is obtained in database and currently performs aggregate function
The current value of operation object.
For example, occur in real-time statistics system error-critical word summation when, Stream Processing task is performed into system first
The encapsulated data packet of the error-critical word summation occurred in meter systems is loaded into real-time task actuator, is performed in real-time task
Perform the encapsulated data packet decapsulation of the error-critical word summation occurred in statistical system in device first to the Stream Processing task,
The operation object of the error-critical word occurred in the encapsulated data packet in statistical system is got, and to occurring in statistical system
Error-critical word operation object statistical function.The error-critical word that occurs in statistical system is got in database
The current value of operation object.
S302, to the current value of the operation object of aggregate function, according to the functional relation of operation object, performs polymerization letter
Number operation.
Specifically, in above-mentioned operation object, the functional relation of operation object for obtaining the aggregate function in encapsulated data packet
And after the current value of the operation object of the aggregate function, according to the functional relation of the operation object, to the behaviour of aggregate function
Make the current value of object, perform aggregate function operation.
For example, occur in real-time statistics system error-critical word summation when, will in statistical system obtained above send out
The operation object of raw error-critical word, and to the statistical function of the operation object of error-critical word occurred in statistical system
And after the current value of the operation object of the error-critical word occurred in statistical system, according to the mistake occurred in the statistical system
The statistical function relation of the operation object of keyword by mistake, to the current of the operation object of error-critical word that occurs in statistical system
Numerical value performs phylogenetic error-critical word summation operation in the period.
It can be seen that, by getting the aggregate function logical relation of encapsulated data packet, current this is got in database and is gathered
The current value of the operation object of function is closed, to the current value of the operation object of aggregate function, according to the function of operation object
Relation, execution aggregate function operation, it is not necessary to which user writes the program for handling the streaming task aggregate function in real time, it is only necessary to
Get and currently update the data in database, call the aggregate function logical relation to perform the aggregate function, improve reality
When calculate operation development efficiency.
In a kind of embodiment of the real-time task processing method of the embodiment of the present invention, with the mistake in scan text data flow
Keyword, and summing by mistake, real-time statistics go out exemplified by the error-critical word summation occurred in some system, to appointing in real time for the present invention
Business processing method is described in detail.Implement step as follows：
Step one, when Stream Processing task performs aggregate function first, to the numerical value of the operation object of aggregate function, press
The aggregate function operation of functional relation, execution Stream Processing task according to operation object.
Correspond to when Stream Processing task performs real-time statistics and goes out the error-critical word summation occurred in some system first,
Specially：
When the Stream Processing task performs the error-critical word summation occurred in statistical system first, external storage is connected
Database Redis, according to the title for the aggregate function for performing the error-critical word summation occurred in the system, such as counter
(" error_num ") is, in operation " error_num " this index, the numerical value of the index to be obtained in database.According to statistics
The statistical function of the operation object of the error-critical word occurred in system, to the error-critical word occurred in the system in database
The numerical value of operation object operated, for example：++ mean the number of the operation object of the error-critical word to occurring in system
Value carries out Jia one operating, -- mean the numerical value operation that subtracts one of the operation object of the error-critical word to occurring in system.Obtain
The result for the error-critical word summation that the Stream Processing task occurs in execution system first.
Step 2, at the end of Stream Processing task performs aggregate function first, by the polymerization of Stream Processing tasks carrying
The operation object of function, the functional relation of operation object, are encapsulated in encapsulated data packet.
Correspond to when Stream Processing task performs real-time statistics and goes out the error-critical word summation occurred in some system first,
Specially：
At the end of the Stream Processing task performs the error-critical word summation occurred in statistical system first, by the streaming
Processing task performs the error-critical word that occurs in the statistical system of the error-critical word summation occurred in statistical system first
The statistical function of the operation object of the error-critical word occurred in operation object and statistical system, encapsulation forms statistical system
The error-critical word summation encapsulated data packet of middle generation.
Step 3, storage enclosure packet.
Correspond to when Stream Processing task performs real-time statistics and goes out the error-critical word summation occurred in some system first,
Specially：
The Stream Processing task is performed to the error-critical word summation occurred in statistical system, the statistical system of formation first
The error-critical word summation encapsulated data packet storage of middle generation.
Step 4, by the numerical value of the operation object of aggregate function, is performed at streaming according to the functional relation of operation object
The aggregate function operation of reason task, obtains the result of the numerical value of operation object, is stored in database.
Correspond to when Stream Processing task performs real-time statistics and goes out the error-critical word summation occurred in some system first,
Specially：
The result of the error-critical word summation occurred in the Stream Processing task first execution system is stored in database
In.
Step 5, when Stream Processing task currently performs aggregate function, loads the Stream Processing tasks carrying stored
Aggregate function encapsulated data packet, wherein, encapsulated data packet includes：The operation object of the aggregate function of Stream Processing task,
The functional relation of operation object.
Correspond to when Stream Processing task currently performs real-time statistics and goes out the error-critical word summation occurred in some system,
Specially：
When the error-critical word summation occurred in the current execution system of Stream Processing task, the statistics stored is loaded
The error-critical word summation encapsulated data packet occurred in system.
Step 6, according to aggregate function and encapsulated data packet, to the current value of the operation object of aggregate function, according to
The functional relation of operation object, execution aggregate function operation.
Correspond to when Stream Processing task currently performs real-time statistics and goes out the error-critical word summation occurred in some system,
Specially：
According to what is occurred in the function and the statistical system of the error-critical word summation occurred in the execution statistical system
Error-critical word summation encapsulated data packet, to the current number of the operation object of error-critical word occurred in the system in database
Value, according to the statistical function of the operation object of the error-critical word occurred in statistical system in the packet, performs statistical system
In currently occur error-critical word summation operation.
To achieve the above object of the invention, the embodiment of the invention also discloses a kind of real-time task processing unit, such as Fig. 4 institutes
Show.Fig. 4 is a kind of real-time task processing unit figure of the embodiment of the present invention, including：
State load module 401, for when Stream Processing task currently performs aggregate function, loading the streaming stored
The encapsulated data packet of the aggregate function of tasks carrying is handled, wherein, encapsulated data packet includes：The aggregate function of Stream Processing task
Operation object, the functional relation of operation object；
Performing module 402, for according to aggregate function and encapsulated data packet, to the current of the operation object of aggregate function
Numerical value, according to the functional relation of operation object, performs aggregate function operation.
The embodiment of the invention discloses a kind of real-time task processing unit, by by the aggregate function of Stream Processing task
The functional relation packing sealed storage of operation object and the operation object, when performing same Stream Processing task in real time
During aggregate function, operation object and operation pair of the aggregate function of the Stream Processing task stored are directly obtained
There is provided the logic of the operation object for same operation function and the functional relation of operation object pass for the functional relation of elephant
The method of system's encapsulation caching.Do not need developer to repeat to write data read and write interface with regard to the function interface of same problem every time, drop
The time of the low real time data processing business for same operation and improve whole development efficiency.The embodiment of the present invention is most
Realize eventually and real time data processing status of processes is managed, and be easily updated operation, improve the exploitation of real-time task
Efficiency.
It should be noted that the device of the embodiment of the present invention is the device using above-mentioned real-time task processing method, then on
All embodiments for stating real-time task processing method are applied to the device, and can reach same or analogous beneficial effect.
Alternatively, in a kind of embodiment of the real-time task processing unit of the embodiment of the present invention, real-time task processing dress
Putting also includes：
First performing module, for when Stream Processing task performs aggregate function first, the operation pair to aggregate function
The numerical value of elephant, is operated according to the aggregate function of the functional relation of operation object, execution Stream Processing task；
Package module, at the end of performing aggregate function first in Stream Processing task, by Stream Processing tasks carrying
Aggregate function operation object, the functional relation of operation object, be encapsulated in encapsulated data packet；
First memory module, for storage enclosure packet.
Alternatively, in a kind of embodiment of the real-time task processing unit of the embodiment of the present invention, real-time task processing dress
Putting also includes：
Second memory module, for by the numerical value of the operation object of aggregate function, according to the functional relation of operation object
The aggregate function operation of Stream Processing task is performed, the result of the numerical value of operation object is obtained, is stored in database.
Alternatively, in a kind of embodiment of the real-time task processing unit of the embodiment of the present invention, real-time task processing dress
Putting also includes：
Accumulation module, for add up Stream Processing task perform first aggregate function currently performed to Stream Processing task it is poly-
Close versus time section, operation object update the data, will update the data and be accumulated to Stream Processing task and perform polymerization letter first
Several operation objects numerically, obtain the current value that Stream Processing task currently performs the operation object of aggregate function；
Packet load-on module, the encapsulated data packet of the aggregate function for loading Stream Processing task, and operation pair
The current value of elephant；
Accordingly, performing module 402, including：
Acquisition submodule, for by encapsulated data packet, obtaining the operation object of the aggregate function in encapsulated data packet, grasping
Make the functional relation of object；
Implementation sub-module, for the current value of the operation object to aggregate function, according to the functional relation of operation object,
Perform aggregate function operation.
Alternatively, in a kind of embodiment of the real-time task processing unit of the embodiment of the present invention, performing module 402, also
For by encapsulated data packet, obtaining operation object, the functional relation of operation object of the aggregate function in encapsulated data packet, with
And the current value of the operation object of aggregate function is got in database；To the current number of the operation object of aggregate function
Value, according to the functional relation of operation object, performs aggregate function operation.
In a kind of embodiment of the real-time task processing unit of the embodiment of the present invention, as shown in Figure 5.Fig. 5 is the present invention
Another real-time task processing unit figure of embodiment, including：
First performing module 501, for when Stream Processing task performs aggregate function first, the operation to aggregate function
The numerical value of object, is operated according to the aggregate function of the functional relation of operation object, execution Stream Processing task.
Package module 502, at the end of performing aggregate function first in Stream Processing task, Stream Processing task is held
The operation object of capable aggregate function, the functional relation of operation object, are encapsulated in encapsulated data packet.
First memory module 503, for storage enclosure packet.
Second memory module 504, for by the numerical value of the operation object of aggregate function, being closed according to the function of operation object
System performs the aggregate function operation of Stream Processing task, obtains the result of the numerical value of operation object, is stored in database.
State load module 505, for when Stream Processing task currently performs aggregate function, loading the streaming stored
The encapsulated data packet of the aggregate function of tasks carrying is handled, wherein, encapsulated data packet includes：The aggregate function of Stream Processing task
Operation object, the functional relation of operation object.
Performing module 506, for according to aggregate function and encapsulated data packet, to the current of the operation object of aggregate function
Numerical value, according to the functional relation of operation object, performs aggregate function operation.
In a kind of embodiment of the real-time task processing unit of the embodiment of the present invention, with the mistake in scan text data flow
Keyword, and summing by mistake, real-time statistics go out exemplified by the error-critical word summation occurred in some system, to appointing in real time for the present invention
Business processing unit is described in detail, as shown in Figure 6.Fig. 6 implements figure for a kind of real-time task processing unit of the embodiment of the present invention.
In Figure 5, the first performing module 501, for when Stream Processing task performs aggregate function first, to polymerization letter
The numerical value of several operation objects, is operated according to the aggregate function of the functional relation of operation object, execution Stream Processing task.
Correspond to when Stream Processing task performs real-time statistics and goes out the error-critical word summation occurred in some system first,
Settable real-time task actuator is performed.Specially：
Real-time task actuator, for performing the error-critical word occurred in statistical system first when the Stream Processing task
During summation, external storage database Redis is connected, according to the aggregate function for performing the error-critical word summation occurred in the system
Title, such as counter (" error_num ") is that in database obtaining this refers in operation " error_num " this index
Target numerical value.According to the statistical function of the operation object of the error-critical word occurred in statistical system, to the system in database
The numerical value of the operation object of the error-critical word of middle generation is operated, for example：++ mean to close the mistake occurred in system
Operation that the numerical value progress of the operation object of key word Jia one, -- mean the operation object of the error-critical word to occurring in system
Operation that numerical value subtracts one.Obtain the result for the error-critical word summation that the Stream Processing task occurs in execution system first.
Package module 502, at the end of performing aggregate function first in Stream Processing task, Stream Processing task is held
The operation object of capable aggregate function, the functional relation of operation object, are encapsulated in encapsulated data packet.
Correspond to when Stream Processing task performs real-time statistics and goes out the error-critical word summation occurred in some system first,
Specially：
Real-time task actuator, for performing the error-critical word occurred in statistical system first when the Stream Processing task
At the end of summation, in the statistical system that the Stream Processing task is performed to the error-critical word summation occurred in statistical system first
The statistics letter of the operation object of the error-critical word occurred in the operation object and statistical system of the error-critical word of generation
Number, encapsulation forms the error-critical word summation encapsulated data packet occurred in statistical system.
First memory module 503, for storage enclosure packet.
Correspond to when Stream Processing task performs real-time statistics and goes out the error-critical word summation occurred in some system first,
Specially：
State metadata information storage database, for the Stream Processing task to be performed into what is occurred in statistical system first
The error-critical word summation encapsulated data packet storage occurred in error-critical word summation, the statistical system of formation.
Second memory module 504, for by the numerical value of the operation object of aggregate function, being closed according to the function of operation object
System performs the aggregate function operation of Stream Processing task, obtains the result of the numerical value of operation object, is stored in database.
Correspond to when Stream Processing task performs real-time statistics and goes out the error-critical word summation occurred in some system first,
Specially：
External storage database, for the error-critical word summation that will occur in the Stream Processing task first execution system
Result be stored in database.
State load module 505, for when Stream Processing task currently performs aggregate function, loading the streaming stored
The encapsulated data packet of the aggregate function of tasks carrying is handled, wherein, encapsulated data packet includes：The aggregate function of Stream Processing task
Operation object, the functional relation of operation object.
Correspond to when Stream Processing task currently performs real-time statistics and goes out the error-critical word summation occurred in some system,
Specially：
State supervisor, for when the error-critical word summation occurred in the Stream Processing task again execution system,
Load the error-critical word summation encapsulated data packet occurred in the statistical system stored.
Performing module 506, for according to aggregate function and encapsulated data packet, to the current of the operation object of aggregate function
Numerical value, according to the functional relation of operation object, performs aggregate function operation.
Correspond to when Stream Processing task currently performs real-time statistics and goes out the error-critical word summation occurred in some system,
Specially：
Task performer, for the function and the system according to the error-critical word summation occurred in the execution statistical system
The error-critical word summation encapsulated data packet occurred in meter systems, to the behaviour of the error-critical word occurred in the system in database
Make the current value of object, according to the statistics letter of the operation object of the error-critical word occurred in statistical system in the packet
The error-critical word summation operation currently occurred in number, execution statistical system.
It should be noted that a kind of real-time task processing unit of the embodiment of the present invention implements figure, it is only that the present invention is implemented
One kind in the disclosed real-time task processing unit of example, it is any that there is the first performing module 501, package module 502, first to store
Module 503, the second memory module 504, state load module 505, the identical function of performing module 506 are implemented with the present invention
The actuator or performing module of the real-time task processing method thought of example, belong to protection scope of the present invention.
It should be noted that herein, such as first and second or the like relational terms are used merely to a reality
Body or operation make a distinction with another entity or operation, and not necessarily require or imply these entities or deposited between operating
In any this actual relation or order.Moreover, term " comprising ", "comprising" or its any other variant are intended to
Nonexcludability is included, so that process, method, article or equipment including a series of key elements not only will including those
Element, but also other key elements including being not expressly set out, or also include being this process, method, article or equipment
Intrinsic key element.In the absence of more restrictions, the key element limited by sentence "including a ...", it is not excluded that
Also there is other identical element in process, method, article or equipment including key element.
Each embodiment in this specification is described by the way of related, identical similar portion between each embodiment
Divide mutually referring to what each embodiment was stressed is the difference with other embodiment.It is real especially for system
Apply for example, because it is substantially similar to embodiment of the method, so description is fairly simple, related part is referring to embodiment of the method
Part explanation.
Presently preferred embodiments of the present invention is these are only, is not intended to limit the scope of the present invention.It is all in this hair
Any modification, equivalent substitution and improvements made within bright spirit and principle etc., are all contained in protection scope of the present invention.
Claims (10)
1. a kind of real-time task processing method, it is characterised in that including：
When Stream Processing task currently performs aggregate function, load the described of the Stream Processing tasks carrying stored and gather
The encapsulated data packet of function is closed, wherein, the encapsulated data packet includes：The behaviour of the aggregate function of the Stream Processing task
Make object, the functional relation of the operation object；
According to the aggregate function and the encapsulated data packet, to the current number of the operation object of the aggregate function
Value, according to the functional relation of the operation object, performs the aggregate function operation.
2. real-time task processing method according to claim 1, it is characterised in that described current in Stream Processing task
When performing aggregate function, before the encapsulated data packet for the aggregate function for loading the Stream Processing tasks carrying, the side
Method also includes：
When the Stream Processing task performs the aggregate function first, to the number of the operation object of the aggregate function
Value, is operated according to the aggregate function of the functional relation of the operation object, the execution Stream Processing task；
At the end of the Stream Processing task performs the aggregate function first, by described in the Stream Processing tasks carrying
The operation object, the functional relation of the operation object of aggregate function, are encapsulated in the encapsulated data packet；
Store the encapsulated data packet.
3. real-time task processing method according to claim 2, it is characterised in that in the behaviour to the aggregate function
Make the numerical value of object, grasped according to the aggregate function of the functional relation of the operation object, the execution Stream Processing task
After work, methods described also includes：
By to the numerical value of the operation object of the aggregate function, the stream is performed according to the functional relation of the operation object
The aggregate function operation of formula processing task, obtains the result of the numerical value of the operation object, is stored in database.
4. real-time task processing method according to claim 1 or 2, it is characterised in that described according to the polymerization letter
Several and described encapsulated data packet, to the current value of the operation object of the aggregate function, according to the operation object
Functional relation, perform aggregate function operation before, methods described also includes：
The accumulative Stream Processing task performs the aggregate function and currently performs described gather to the Stream Processing task first
Updating the data for the close versus time section, operation object, the Stream Processing task head is accumulated to by described update the data
The secondary operation object for performing the aggregate function it is described numerically, obtain the Stream Processing task currently perform it is described
The current value of the operation object of aggregate function；
Load the encapsulated data packet of the aggregate function of the Stream Processing task, and the operation object is described
Current value；
Accordingly, the operation object of the aggregate function according in the encapsulated data packet, the letter of the operation object
Number relation, to the current value of the operation object of the aggregate function, according to the functional relation of the operation object, is performed
Corresponding operation, including：
By the encapsulated data packet, the operation object of the aggregate function in the encapsulated data packet, the operation are obtained
The functional relation of object；
To the current value of the operation object of the aggregate function, according to the functional relation of the operation object, hold
The row aggregate function operation.
5. real-time task processing method according to claim 1, it is characterised in that it is described according to the aggregate function and
The encapsulated data packet, to the current value of the operation object of the aggregate function, according to the function of the operation object
Relation, the execution aggregate function operation, including：
By the encapsulated data packet, the operation object of the aggregate function in the encapsulated data packet, the operation are obtained
The functional relation of object, and get in database the current value of the operation object of the aggregate function；
To the current value of the operation object of the aggregate function, according to the functional relation of the operation object, hold
The row aggregate function operation.
6. a kind of real-time task processing unit, it is characterised in that including：
State load module, for when Stream Processing task currently performs aggregate function, loading at the streaming stored
The encapsulated data packet of the aggregate function of tasks carrying is managed, wherein, the encapsulated data packet includes：The Stream Processing task
The aggregate function operation object, the functional relation of the operation object；
Performing module, for according to the aggregate function and the encapsulated data packet, the operation to the aggregate function
The current value of object, according to the functional relation of the operation object, performs the aggregate function operation.
7. real-time task processing unit according to claim 6, it is characterised in that described device also includes：
First performing module, for when the Stream Processing task performs the aggregate function first, to the aggregate function
The operation object numerical value, according to the functional relation of the operation object, perform the described poly- of the Stream Processing task
Close function operation；
Package module, at the end of performing the aggregate function first in the Stream Processing task, by the Stream Processing
The operation object, the functional relation of the operation object of the aggregate function of tasks carrying, are encapsulated in the encapsulation number
According in bag；
First memory module, for storing the encapsulated data packet.
8. real-time task processing unit according to claim 7, it is characterised in that described device also includes：
Second memory module, for by the numerical value of the operation object of the aggregate function, according to the operation object
Functional relation performs the aggregate function operation of the Stream Processing task, obtains the result of the numerical value of the operation object,
It is stored in database.
9. the real-time task processing unit according to claim 6 or 7, it is characterised in that described device also includes：
Accumulation module, for add up the Stream Processing task perform first the aggregate function to the Stream Processing task work as
It is preceding perform it is described it is the aggregate function period, the operation object update the data, described update the data is accumulated to the stream
Formula processing task perform first the aggregate function the operation object it is described numerically, obtain the Stream Processing task
Currently perform the current value of the operation object of the aggregate function；
Packet load-on module, the encapsulated data packet of the aggregate function for loading the Stream Processing task, with
And the current value of the operation object；
Accordingly, the performing module, including：
Acquisition submodule, for by the encapsulated data packet, obtaining the behaviour of the aggregate function in the encapsulated data packet
Make object, the functional relation of the operation object；
Implementation sub-module, for the current value of the operation object to the aggregate function, according to the operation pair
The functional relation of elephant, the execution aggregate function operation.
10. real-time task processing unit according to claim 6, it is characterised in that the performing module, is additionally operable to pass through
The operation object of the aggregate function in the encapsulated data packet, the acquisition encapsulated data packet, the letter of the operation object
Number relations, and get in database the current value of the operation object of the aggregate function；To the aggregate function
The current value of the operation object, according to the functional relation of the operation object, performs the aggregate function operation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710210712.XA CN107168681B (en) | 2017-03-31 | 2017-03-31 | Real-time task processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710210712.XA CN107168681B (en) | 2017-03-31 | 2017-03-31 | Real-time task processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107168681A true CN107168681A (en) | 2017-09-15 |
| CN107168681B CN107168681B (en) | 2020-11-06 |
Family
ID=59849803
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710210712.XA Active CN107168681B (en) | 2017-03-31 | 2017-03-31 | Real-time task processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107168681B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040205134A1 (en) * | 2003-02-14 | 2004-10-14 | Digate Charles J. | System and method for immediate and delayed real-time communication activities using availability data from and communications through an external instant messaging system |
| US20050055329A1 (en) * | 2000-02-28 | 2005-03-10 | Reuven Bakalash | Database management system having data aggregation module integrated therein |
| CN102799423A (en) * | 2011-05-27 | 2012-11-28 | 深圳市金蝶中间件有限公司 | Method and device for implementing dynamic method in JSF (java service face) |
| CN103152126A (en) * | 2012-04-06 | 2013-06-12 | 中国科学技术大学 | Data packaging method and device based on forward error correction protection codes |
| CN104615777A (en) * | 2015-02-27 | 2015-05-13 | 浪潮集团有限公司 | Method and device for real-time data processing based on stream-oriented calculation engine |
| CN105608219A (en) * | 2016-01-07 | 2016-05-25 | 上海通创信息技术有限公司 | Stream-oriented recommended engine, recommendation system and recommendation method based on clustering |
| CN106506266A (en) * | 2016-11-01 | 2017-03-15 | 中国人民解放军91655部队 | Network flow analysis method based on GPU, Hadoop/Spark mixing Computational frame |
| CN106708487A (en) * | 2015-11-16 | 2017-05-24 | 北京国双科技有限公司 | Data presenting method and device |
-
2017
- 2017-03-31 CN CN201710210712.XA patent/CN107168681B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050055329A1 (en) * | 2000-02-28 | 2005-03-10 | Reuven Bakalash | Database management system having data aggregation module integrated therein |
| US20040205134A1 (en) * | 2003-02-14 | 2004-10-14 | Digate Charles J. | System and method for immediate and delayed real-time communication activities using availability data from and communications through an external instant messaging system |
| CN102799423A (en) * | 2011-05-27 | 2012-11-28 | 深圳市金蝶中间件有限公司 | Method and device for implementing dynamic method in JSF (java service face) |
| CN103152126A (en) * | 2012-04-06 | 2013-06-12 | 中国科学技术大学 | Data packaging method and device based on forward error correction protection codes |
| CN104615777A (en) * | 2015-02-27 | 2015-05-13 | 浪潮集团有限公司 | Method and device for real-time data processing based on stream-oriented calculation engine |
| CN106708487A (en) * | 2015-11-16 | 2017-05-24 | 北京国双科技有限公司 | Data presenting method and device |
| CN105608219A (en) * | 2016-01-07 | 2016-05-25 | 上海通创信息技术有限公司 | Stream-oriented recommended engine, recommendation system and recommendation method based on clustering |
| CN106506266A (en) * | 2016-11-01 | 2017-03-15 | 中国人民解放军91655部队 | Network flow analysis method based on GPU, Hadoop/Spark mixing Computational frame |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107168681B (en) | 2020-11-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Vineet et al. | Fast minimum spanning tree for large graphs on the GPU | |
| Larkin et al. | A back-to-basics empirical study of priority queues | |
| AU2017202640B2 (en) | System architecture for control systems via knowledge layout search | |
| CN104915717B (en) | Data processing method, Analysis of Knowledge Bases Reasoning method and relevant apparatus | |
| CN106897411A (en) | ETL system and its method based on Spark technologies | |
| Lincoln et al. | Deterministic Time-Space Tradeoffs for k-SUM | |
| CN106156047A (en) | A kind of SNAPSHOT INFO processing method and processing device | |
| Yevsieiev et al. | Development of databases interconnection “essences” information model for cyber-physical production systems additive cyber design creation automation | |
| CN106802927A (en) | A kind of date storage method and querying method | |
| CN109117475A (en) | A kind of method and relevant device of text rewriting | |
| US20220114103A1 (en) | Graph memory engine | |
| CN103959652B (en) | Embedded memory and dedicated processor structure within an integrated circuit | |
| CN111624965A (en) | Configuration system for industrial intelligent manufacturing | |
| CN107168681A (en) | A kind of real-time task processing method and processing device | |
| Cao et al. | A knowledge-based prototype system to support product conceptual design | |
| CN100504770C (en) | Method for packing data, and unpacking packed data | |
| CN112445776B (en) | Presto-based dynamic barrel dividing method, system, equipment and readable storage medium | |
| Dai et al. | Modelling industrial cyber-physical systems using iec 61499 and opc ua | |
| CN107391257A (en) | Predictor method, device and the server of memory size needed for business | |
| CN104216729B (en) | The logic module of application system updates implementation method and device | |
| CN108763381A (en) | Divide table method and apparatus based on consistency hash algorithm | |
| US7984045B2 (en) | Scalable data extraction from data stores | |
| CN109033468A (en) | The processing method and processing device of resource change | |
| Esser et al. | Applying an object-oriented petri net language to heterogeneous systems design | |
| CN104932936A (en) | Data processing method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |