A kind of processing method of pipelined data, device and equipment
Technical field
This specification is related to field of computer technology more particularly to a kind of processing method of pipelined data, device and equipment.
Background technique
With the continuous development of network finance and network payment, network finance and the corresponding portfolio of network payment and day are all
Increase, in this way will generate a large amount of pipelined data.And how a large amount of pipelined datas of generation store becoming and need to solve
Major issue certainly.
In general, the related system of the chains such as network finance and network payment road often uses a large amount of pipelined datas of generation
Point library divides the mode of table to land storage, and is related in the relevant operation of pipelined data, and storage can be landed by the way of dividing table.
But when the portfolio of certain business is larger, using table mode is divided, the pipelined data of the business of certain specified dimension
Data volume it is still huge, in this way, often being led when needing to carry out the operation such as relevant business diagnosis processing to pipelined data
It causes big affairs problem to occur, and then easily causes the shake of database, and business scenario higher for timeliness can not expire
Sufficient actual demand, for this reason, it may be necessary to which providing the flowing water grouping scheme that one kind can be real-time, adaptive is grouped place to pipelined data
Reason.
Summary of the invention
The purpose of this specification embodiment is to provide processing method, device and the equipment of a kind of pipelined data, to provide one
Flowing water grouping scheme that kind can be real-time, adaptive is grouped processing to pipelined data.
To realize that above-mentioned technical proposal, this specification embodiment are achieved in that
A kind of processing method for pipelined data that this specification embodiment provides, which comprises
When detecting that target pipelined data generates, the business dimension of the corresponding target service of the target pipelined data is obtained
Spend attribute;
Based on the business dimension attribute, the target pipelined data is stored corresponding extremely to the business dimension attribute
In target data grouping in a few data grouping.
Optionally, described to be based on the business dimension attribute, by target pipelined data storage to the business dimension
In target data grouping at least one corresponding data grouping of attribute, comprising:
If including the business dimension attribute in database, current time and the target pipelined data are obtained
Grouping information;
According to the grouping information of the target pipelined data, the time and the corresponding storage of the target pipelined data are arrived
In the target data grouping.
Optionally, described to be based on the business dimension attribute, by target pipelined data storage to the business dimension
In target data grouping at least one corresponding data grouping of attribute, comprising:
If in database not including the business dimension attribute, the corresponding data point of the business dimension attribute are created
Group includes that target data is grouped in the data grouping of creation;
The current time is obtained, and the time and the corresponding storage of the target pipelined data are arrived to the institute of the database
It states in target data grouping.
Optionally, the method also includes:
Receive the pipelined data processing request of the target service;
According to the business dimension attribute of the target service, obtain and the business dimension attribute phase from the database
At least one corresponding data grouping;
According to the initial time of the pipelined data stored at least one described data grouping and deadline, respectively to institute
The pipelined data stored at least one data grouping is stated to be handled.
Optionally, the flowing water number of predetermined quantity can be stored in the corresponding each data grouping of the business dimension attribute
According to.
Optionally, the grouping information is that the flowing water of the target pipelined data identifies, and the flowing water mark includes flowing water
Number, transaction one or more of odd numbers, alternatively, the grouping information is to be based on business dimension with the target pipelined data
Spend the relevant information of storage order under attribute.
A kind of processing unit for pipelined data that this specification embodiment provides, described device include:
Dimensional attribute obtains module, for obtaining the target pipelined data when detecting that target pipelined data generates
The business dimension attribute of corresponding target service;
Flowing water memory module, for being based on the business dimension attribute, by target pipelined data storage to the industry
It is engaged in the target data grouping at least one corresponding data grouping of dimensional attribute.
Optionally, the flowing water memory module, comprising:
Information acquisition unit, if for including the business dimension attribute in database, obtain current time and
The grouping information of the target pipelined data;
Flowing water storing unit, for the grouping information according to the target pipelined data, by the time and the target
The corresponding storage of pipelined data is into target data grouping.
Optionally, the flowing water memory module, comprising:
It is grouped creating unit, if creating the business dimension for not including the business dimension attribute in database
The corresponding data grouping of attribute is spent, includes that target data is grouped in the data grouping of creation;
Flowing water storing unit is deposited for obtaining the current time, and by the time and target pipelined data correspondence
It stores up in the target data grouping of the database.
Optionally, described device further include:
Request receiving module, the pipelined data for receiving the target service handle request;
Be grouped determining module, for the business dimension attribute according to the target service, obtain from the database with
At least one corresponding data grouping of the business dimension attribute;
Processing module, for the initial time and cut-off according to the pipelined data stored at least one described data grouping
Time is respectively handled the pipelined data stored at least one described data grouping.
Optionally, the flowing water number of predetermined quantity can be stored in the corresponding each data grouping of the business dimension attribute
According to.
Optionally, the grouping information is that the flowing water of the target pipelined data identifies, and the flowing water mark includes flowing water
Number, transaction one or more of odd numbers, alternatively, the grouping information is to be based on business dimension with the target pipelined data
Spend the relevant information of storage order under attribute.
A kind of processing equipment for pipelined data that this specification embodiment provides, the processing equipment packet of the pipelined data
It includes:
Processor;And
It is arranged to the memory of storage computer executable instructions, the executable instruction makes the place when executed
Manage device:
When detecting that target pipelined data generates, the business dimension of the corresponding target service of the target pipelined data is obtained
Spend attribute;
Based on the business dimension attribute, the target pipelined data is stored corresponding extremely to the business dimension attribute
In target data grouping in a few data grouping.
By the technical solution of above this specification embodiment offer as it can be seen that this specification embodiment is by detecting target
When pipelined data generates, the business dimension attribute of the corresponding target service of target pipelined data is obtained, then, is based on target service
Business dimension attribute, by business dimension attribute corresponding at least one data point of target pipelined data storage to target service
In target data grouping in group, in this way, whenever detecting that new pipelined data (i.e. target pipelined data) generates
With according to the business dimension attribute of the corresponding target service of target pipelined data, by the storage of target pipelined data to accordingly at least
In a data grouping in one data grouping, thereby may be ensured that pipelined data processing real-time, meet timeliness compared with
The demand of high business scenario.It, therefore, can be with moreover, because pipelined data is stored in a manner of an at least data grouping
To be subsequent in the pipelined data of processing target business, pipelined data therein can be fished for as unit of data grouping, and by
The limited amount of pipelined data in each data grouping, and quantity will not be excessive, this way it is possible to avoid in issued transaction mistake
Occurs big affairs problem in journey.
Detailed description of the invention
In order to illustrate more clearly of this specification embodiment or technical solution in the prior art, below will to embodiment or
Attached drawing needed to be used in the description of the prior art is briefly described, it should be apparent that, the accompanying drawings in the following description is only
The some embodiments recorded in this specification, for those of ordinary skill in the art, in not making the creative labor property
Under the premise of, it is also possible to obtain other drawings based on these drawings.
Fig. 1 is a kind of processing method embodiment of pipelined data of this specification;
Fig. 2 is a kind of structural schematic diagram of the processing system of pipelined data of this specification;
Fig. 3 is the processing method embodiment of this specification another kind pipelined data;
Fig. 4 is a kind of processing device embodiment of pipelined data of this specification;
Fig. 5 is a kind of processing equipment embodiment of pipelined data of this specification.
Specific embodiment
This specification embodiment provides processing method, device and the equipment of a kind of pipelined data.
In order to make those skilled in the art more fully understand the technical solution in this specification, below in conjunction with this explanation
Attached drawing in book embodiment is clearly and completely described the technical solution in this specification embodiment, it is clear that described
Embodiment be only this specification a part of the embodiment, instead of all the embodiments.The embodiment of base in this manual,
Every other embodiment obtained by those of ordinary skill in the art without making creative efforts, all should belong to
The range of this specification protection.
Embodiment one
As shown in Figure 1, this specification embodiment provides a kind of processing method of pipelined data, the executing subject of this method can
Think terminal device or server, wherein the terminal device can such as personal computer equipment, can also be such as mobile phone, plate
The mobile terminal devices such as computer, the terminal device can be the terminal device that user uses.The server can be independent clothes
Business device, is also possible to the server cluster being made of multiple servers, moreover, the server can be the backstage clothes of a certain business
Business device is also possible to the background server etc. of certain website (such as websites or payment application).This method can be used for
In real time, adaptively processing etc. is grouped to pipelined data, in order to improve the treatment effeciency of pipelined data, in the present embodiment with
Executing subject is to be illustrated for server, the case where for terminal device, can be handled according to following related contents, herein
It repeats no more.This method can specifically include following steps:
In step s 102, when detecting that target pipelined data generates, the corresponding target industry of target pipelined data is obtained
The business dimension attribute of business.
Wherein, target pipelined data can be the pipelined data of any business, such as the transaction flow water number in payment transaction
According in, financial business for pipelined data of reconciliation etc..Different target services can be corresponding with different pipelined datas.Business
Dimensional attribute can be the attribute that a dimensional information for being included based on target service is constituted, for example, target service includes machine
One dimensional information of structure time, then its business dimension attribute can be authority time, alternatively, being also possible to based on target service institute
The attribute that the multiple dimensional informations for including are constituted, for example, target service includes two dimensions such as authority time and accounting authority identification
Information is spent, then its business dimension attribute can be authority time and accounting authority identification, in practical applications, the business dimension category
Property can be and be arranged by way of such as above-mentioned text information, further, it is also possible to corresponding code is arranged in each dimensional information,
The dimensional information can also be arranged by way of above-mentioned code, and specifically such as, authority time 20180917 calculates authority identification
It is 3009, then the business dimension attribute can be 20180917/3009 etc..
In an implementation, with the continuous development of network finance and network payment, network finance and the corresponding industry of network payment
Business amount is growing day by day, in this way will generate a large amount of pipelined data.And how a large amount of pipelined datas of generation to be stored
The major issue solved as needs.
In general, the related system of the chains such as network finance and network payment road often uses a large amount of pipelined datas of generation
Point library divides the mode of table to land storage, and is related in the relevant operation of pipelined data, and storage can be landed by the way of dividing table.
But when the portfolio of certain business is larger, using table mode is divided, the pipelined data of the business of certain specified dimension
Data volume it is still huge, in this way, often being led when needing to carry out the operation such as relevant business diagnosis processing to pipelined data
It causes big affairs problem to occur, and then easily causes the shake of database.Such as reconciliation centring system to corresponding pipelined data into
When row collation process, the single settlement day of specified services, there are a large amount of pipelined datas, after carrying out inner part list processing, based on single
The operations such as the pipelined data verification of tables of data, filing, editing and processing will appear the big affairs problem of database, specifically such as, specify industry
It is engaged under single settlement day, includes 200,000,000 pipelined datas in transaction details, carry out a point list processing, be stored to 100 tables of data
In after, the quantity of the pipelined data for including under individual data table is 2,000,000, in the batch pipelined data core for carrying out individual data table
Pair process in, single affairs submission amount that network finance and network payment side, mechanism side handle pipelined data 4,000,000 with
On, when carrying out batch filing to the pipelined data in individual data table, the affairs submission amount being related to is 8,000,000 or more, due to thing
Submission amount of being engaged in is larger, causes corresponding affairs larger, database can be caused to shake when submitting affairs, influence regular traffic on line.
It to solve the above-mentioned problems, can be by using timed task to the pipelined data generated at " particular point in time "
Mark grouping is carried out, for example, can combine according to the dimension of corresponding service, meets what the dimension combined under each tables of data
Pipelined data is grouped mark processing.When needing using pipelined data, data dimension and grouping convection current water number can be passed through
According to being handled, in this manner it is ensured that the quantity that affairs are submitted when service operation for single time is relatively low, so as to solve major issue
Business problem, still, aforesaid way business scenario higher for timeliness are unable to satisfy actual demand, for this purpose, it is also required to provide
A kind of real-time, adaptive flowing water group technology is grouped processing to pipelined data, can specifically include the following contents:
As shown in Fig. 2, server can be based at scheduled business processing mechanism whenever executing a certain business
Reason, can produce pipelined data (i.e. target pipelined data) during business processing, whenever the new target stream water number of generation
According to when, can send a notification message to the server or request message, to notify the server to there is new pipelined data to generate, this
When, the available target pipelined data of the server.For example, when user is transferred accounts using payment application to certain account, user
The application program that can star payment application finds key of transferring accounts in the page that payment application provides, and user can be with
The key of transferring accounts is clicked, payment application can obtain corresponding transfer page, Yong Huke from the background server of payment application
To input the information of transfer amounts and the account in transfer page, payment application generates corresponding trading card, and after being sent to
Platform server, background server can carry out processing of transferring accounts based on the trading card, in above-mentioned treatment process, whenever payment is applied
When interacting with background server, corresponding pipelined data can produce, such as payment application obtains phase from background server
When the transfer page answered, corresponding pipelined data can produce, trading card is sent to background server, Yi Jihou by payment application
Platform server is based on the trading card and carries out processing of transferring accounts, and can produce corresponding pipelined data.Whenever the new flowing water number of generation
According to when, can send a notification message to the server or request message, at this point, the available target pipelined data of the server.
In this way, server can be the pipelined data for getting generation in real time.
It should be noted that server can be made to perceive above by sending a notification message or the mode of request message
Or detect that new pipelined data generates, in practical applications, however it is not limited to detect target pipelined data through the above way
Generation, can also realize, specifically may be set according to actual conditions otherwise, this specification embodiment does not do this
It limits.
It may include the service identification of the affiliated business of target pipelined data in target pipelined data, server gets target
After pipelined data, service identification therein can be extracted from target pipelined data, target can be determined by the service identification
Target service belonging to pipelined data.It may include one or more dimensional informations and each dimensional information in each business
Corresponding data, the dimensional information for including in the available target service of server, can extract from the dimensional information of acquisition
One or more dimensional informations can construct the business dimension attribute of target service based on the dimensional information of extraction, wherein for
The dimensional information of acquisition includes multiple situation, and multiple dimensional informations that server can will acquire are combined, and constitutes target
Business dimension attribute of business etc..
In step S104, it is based on above-mentioned business dimension attribute, the business dimension attribute is arrived into the storage of target pipelined data
In target data grouping at least one corresponding data grouping.
In an implementation, one or more data groupings can be set for each business dimension attribute, and each data grouping can
To accommodate the pipelined data of certain amount or certain data volume, such as each data grouping can accommodate 1000 pipelined datas,
Alternatively, each data grouping can accommodate the pipelined data etc. of 10MB.Server gets the business dimension attribute of target service
Afterwards, it can be searched whether in the database of storage pipelined data comprising the business dimension attribute, if comprising being somebody's turn to do in database
Business dimension attribute then shows that the pipelined data of target service has been stored before current time, at this point it is possible to by target
Into database in one data grouping of the business dimension attribute, specific storage mode can be with a variety of more for pipelined data storage
Sample, for example, may include the information such as serial number in each pipelined data, different pipelined datas, serial number can be different, can
To determine the data grouping where target pipelined data according to the serial number of target pipelined data, specifically such as, if each data
Grouping can accommodate 1000 pipelined datas, and if the serial number of target pipelined data is 2089, then target pipelined data should the 3rd
In a data grouping, i.e., for accommodate serial number be 2001~3000 data grouping (wherein, the 1st data grouping by
Serial number be 1~1000 pipelined data occupied by, the 2nd data grouping by serial number be 1001~2000 flowing water number
According to occupied).Other than it can realize through the above way, multiple data groupings can also be set simultaneously, then, respectively successively
Store pipelined data etc. into each data grouping, specifically may be set according to actual conditions, this specification embodiment to this not
It limits.
This specification embodiment provides a kind of processing method of pipelined data, by detecting the generation of target pipelined data
When, obtain the business dimension attribute of the corresponding target service of target pipelined data, then, the business dimension category based on target service
Property, by target pipelined data storage to the number of targets at least one corresponding data grouping of business dimension attribute of target service
According in grouping, in this way, whenever detecting that new pipelined data (i.e. target pipelined data) generates, it can according to target stream
Water number stores target pipelined data at least one corresponding data grouping according to the business dimension attribute of corresponding target service
In a data grouping in, thereby may be ensured that pipelined data processing real-time, meet the higher business scenario of timeliness
Demand.Moreover, because pipelined data is stored in a manner of an at least data grouping, therefore, can handled to be subsequent
When the pipelined data of target service, pipelined data therein can be fished for as unit of data grouping, and due to each data point
The limited amount of pipelined data in group, and quantity will not be excessive, this way it is possible to avoid occurring major issue in transaction process
Business problem.
Embodiment two
As shown in figure 3, this specification embodiment provides a kind of processing method of pipelined data, the executing subject of this method can
Think terminal device or server, wherein the terminal device can such as personal computer equipment, can also be such as mobile phone, plate
The mobile terminal devices such as computer, the terminal device can be the terminal device that user uses.The server can be independent clothes
Business device, is also possible to the server cluster being made of multiple servers, moreover, the server can be the backstage clothes of a certain business
Business device is also possible to the background server etc. of certain website (such as websites or payment application).This method can be used for
In real time, adaptively processing etc. is grouped to pipelined data, in order to improve the treatment effeciency of pipelined data, in the present embodiment with
Executing subject is to be illustrated for server, the case where for terminal device, can be handled according to following related contents, herein
It repeats no more.This method can specifically include following steps:
In step s 302, when detecting that target pipelined data generates, the corresponding target industry of target pipelined data is obtained
The business dimension attribute of business.
In an implementation, it when whether detection pipelined data generates, can be accomplished in several ways, such as when new flowing water number
When according to generating, it can be notified by way of notification message, alternatively, can be led to by way of external request message
Know, specifically may be set according to actual conditions, this specification embodiment does not limit this.Whenever detecting new flowing water
When data (i.e. target pipelined data) generate, available target pipelined data, in order to which the target pipelined data is stored,
And searched when facilitating subsequent use, storage can be grouped to pipelined data.In view of different business, the business
Business dimension attribute is often different, therefore, can be using the business dimension attribute of business as the pipelined data of storage different business
Foundation, for this purpose, target service belonging to target pipelined data can be determined, and the one or more of available target service
Attribute dimensions.
It should be noted that the business dimension attribute of target service may include one, also may include it is multiple, if mesh
The business dimension attribute of mark business include it is multiple, then multiple business dimension attributes can be combined, obtain dimension combination, will
After multiple business dimension attributes are combined, it can be indicated by way of the text after multiple business dimension combinations of attributes, example
It then can be time/operator etc. after multiple business dimension combinations of attributes if business dimension attribute includes time and operator, or
Person can also be indicated by the corresponding numerical value of different business dimensional attribute (being properly termed as business dimension attribute value), such as the time
It is 20180918, operator A11 can then pass through the group of each business dimension attribute value after multiple business dimension combinations of attributes
Conjunction value indicates, specific such as 20180918/A11.
In step s 304, if including above-mentioned business dimension attribute in database, current time and target are obtained
The grouping information of pipelined data.
Wherein, grouping information, which can be, is used to indicate the relevant information which data grouping target pipelined data belongs to,
Grouping information may include a variety of, such as grouping information can identify for the flowing water of target stream water number evidence, and flowing water mark can be with
Including one or more of serial number, transaction odd numbers.Trading odd numbers specifically can such as order number, Invoice Number.Data
Library can be the database for storing pipelined data, and the pipelined data stored in database can be carried out with business dimension attribute
It distinguishes.
In an implementation, due to different business, corresponding business dimension attribute is often different, therefore, can pass through industry
Business dimensional attribute determines the pipelined data that certain business dimension attribute whether has been stored in data, is based on this, determines in server
After the business dimension attribute of target service, it can be searched in the database by the business dimension attribute, if in the database
Including the business dimension attribute, then show before current time, the pipelined data of target service stored in the database,
And the corresponding relationship of the existing service attribute and pipelined data in database, at this point, server does not need to re-create this
The corresponding relationship and corresponding data grouping of service attribute and pipelined data, it is only necessary to which target pipelined data is stored in number
According in library.In order to accurately store target pipelined data, can be obtained according to the storage rule of preset pipelined data
The relevant information of target pipelined data, the corresponding storage of the relevant information that can be will acquire, the service attribute and target pipelined data
In the database.A kind of storage mode of pipelined data is provided in the present embodiment, can specifically include the following contents:
In view of pipelined data drop into the time in database for some application scenarios, for user or technical staff
Be it is important, therefore, the available current time, current time can be converted to timestamp, and can be by this when
Between timestamp field value of stamp when falling into database as target pipelined data, such as current time is on September 18th, 2018, then
Timestamp is converted to, can be 20180918, correspondingly, timestamp field value can be 20180918.
It should be noted that current time can also be the time only comprising clock, such as above-mentioned only comprising the date
10:21:38 etc. such as on September 18,10:21:38 etc. 2018, specifically can be with alternatively, may include the time of date and clock
It is set according to actual conditions, this specification embodiment does not limit this.
In addition, each data grouping can store since the corresponding data grouping of business dimension attribute may include multiple
The limited amount of pipelined data therefore can be in order to which target pipelined data to be accurately grouped, in target pipelined data
Including grouping information, for this purpose, including target pipelined data can also be obtained after above-mentioned business dimension attribute in determining database
Grouping information.
It is according to the grouping information of target pipelined data, above-mentioned time and target pipelined data is corresponding in step S306
It stores in target data grouping.
Wherein, target data grouping can be at least one corresponding data grouping of business dimension attribute of target service
A data grouping.The pipelined data of predetermined quantity can be stored in the corresponding each data grouping of business dimension attribute.In advance
Fixed number amount may be set according to actual conditions, such as 1000 or 5000 etc..
In an implementation, the corresponding data grouping of business dimension attribute of business may include multiple, and multiple data groupings
In each data grouping may include one or more pipelined datas, the pipelined data for including in multiple data groupings is fallen into
The timestamp of database may be difference, therefore, fall into database to record the pipelined data of a certain business dimension attribute, can
To use database to be handled from ID mechanism is increased, specifically, in database may include business dimension attribute-timestamp from
Increase ID table, it can be as shown in table 1.
Table 1
ID (from increasing, initial value 0) |
Timestamp |
0 |
20180917 |
1 |
20180918 |
Wherein, the ID in table 1 can be continuously increased with falling into for pipelined data, the numerical value of ID can since 0, according to
Secondary increase, in the case, when ID is 0, then it represents that the pipelined data of record is the initial value of first data grouping, works as ID
Numerical value when being the integer P times of above-mentioned predetermined quantity M, then the pipelined data of current record is located in the P data grouping.In reality
In the application of border, the initial value of ID is not limited to since 0, can be since preset any number, for example, can be from
1000 start to be sequentially increased or reduce.Business dimension attribute-timestamp can be specified services dimensional attribute from ID table is increased
Under timestamp from increasing ID record sheet, i.e., each specified services dimensional attribute have one such as above-mentioned business dimension attribute-when
Between stab from increase ID table.Based on above content, above-mentioned grouping information can be to be based on above-mentioned business dimension with target pipelined data
The relevant information of storage order under attribute, it can for the numerical value of above-mentioned ID or based on ID numerical value determined etc..
In addition, can recorde the business dimension of different business for the ease of checking or the business dimension attribute of inquiry business
Attribute stores the flowing water number into data grouping since any business dimension attribute may include one or more data groupings
According to time can be different, in this way, in different data grouping when can store the initial time and cut-off of pipelined data
Between, it is based on this, can also include business business dimension attribute-grouped record table in database, it can be as shown in table 2.
Table 2
Wherein, the major key in table 2 may be set according to actual conditions, such as the above-mentioned serial number being arranged successively, alternatively, can be with
Pass through the character representation etc. of other forms.
In addition, what is mainly stored in database is pipelined data, it therefore, in database can also include that pipelined data is practical
The detail storage table (being referred to as flowing water business record table) for falling into database, can be as shown in table 3.
Table 3
Wherein, the major key in table 3 may be set according to actual conditions, such as the above-mentioned serial number being arranged successively, alternatively, can be with
Pass through the character representation etc. of other forms.N can be the positive integer more than or equal to 1.
The timestamp that server can determine current time is stored into above-mentioned table 1, when due to increasing one in table 1
Between stab, then the numerical value of corresponding ID can increase automatically, to obtain the corresponding ID numerical value of target pipelined data, can based on
To the numerical value of ID determine data grouping locating for target pipelined data (i.e. target data grouping) and the current institute of target service
The quantity for the data grouping for including (concrete processing procedure may refer to above-mentioned related content, and details are not described herein).Then, may be used
Target pipelined data to be stored in target data grouping, and the number can be determined based on the quantity of obtained data grouping
According to the initial time stamp for the grouping for including in grouping and the deadline stamp of grouping, and obtained above-mentioned timestamp is recorded
It states in table 2.Furthermore it is also possible to which the relevant information of target pipelined data is recorded in table 3.
By taking the business scenario of the detail reconciliation at the reconciliation center in financial business, data filing, flowing water editor carry out as an example,
The business business dimension attribute of the business includes: the mechanism date, operator, currency type, calculates mechanism ID, then the date is for mechanism
For 20180303, if including 20180303/sysOperator/156/3009 business business dimension attribute with include
The business business dimension attribute of 20180303/sysOperator/840/3009, respectively there are two business business dimension attribute-when
Between stab from increasing ID table (i.e. table 1), then the business business dimension category in business business dimension attribute-grouped record table (i.e. table 2)
Property be 20180303/sysOperator/156/3009 and 20180303/sysOperator/840/3009 combination " two
Class " traffic packets record and an actual services record sheet.
It should be noted that after the corresponding timestamp of current time is set in the database, if the ID in above-mentioned table 1
When numerical value is the integral multiple of predetermined quantity, the business dimension attribute is recorded in business dimension attribute-grouped record table (i.e. table 2)
Deadline stamp of (or business dimension combinations of attributes), the initial time stamp of current group and current group etc..Complete target stream
After the grouping of water data storage procedure, actual data grouping can pass through the grouped record table under the business dimension attribute of business
Initial time (or initial time stamp) and deadline (or deadline stamp), corresponding data can be got and be grouped the stream for including
The set of water number evidence.
In step S308, if in database not including above-mentioned business dimension attribute, the business dimension attribute is created
Corresponding data grouping includes that target data is grouped in the data grouping of creation.
In an implementation, if by searching for determining the business dimension attribute for not including target service in database, then showing
There are no the pipelined datas for storing target service in database, at this point it is possible to create the business dimension attribute in the database
Corresponding data grouping.While server creates above-mentioned data grouping, it can also establish as shown in above-mentioned table 1, table 2 and table 3
Message store table, to record the pipelined data storage of target service to the relevant information during database.
In step s310, the current time is obtained, and the time and the corresponding storage of target pipelined data are arrived into database
Target data grouping in.
The concrete processing procedure of above-mentioned steps S310 may refer to the related content of above-mentioned steps S306, no longer superfluous herein
It states.
By above-mentioned processing can by the pipelined data generated in real time store into corresponding data grouping, when need using
It, can be by following step S312~step S316 treatment process, from data grouping or when handling the pipelined data of certain business
Middle taking-up pipelined data is handled, and specifically may refer to following related contents.
In step S312, the pipelined data processing request of target service is received.
Wherein, the service identification of target service can be carried in pipelined data processing request, which can be
The title of target service or coding etc..
In step S314, according to the business dimension attribute of target service, obtained and the business dimension category from database
At least one corresponding data grouping of property.
In an implementation, it after server gets pipelined data processing request, can be mentioned from pipelined data processing request
Service identification is taken, corresponding target service can be determined by the service identification of extraction, it is then possible to obtain the one of target service
A or multiple business dimension attributes, wherein if the business dimension attribute of target service include it is multiple, target industry can be assembled
It is engaged in corresponding business business dimension attribute, so that multiple business business dimension attributes are combined (alternatively, can also be by multiple industry
Business business dimension attribute value is combined).It can the fishing from above-mentioned business business dimension attribute-grouped record table (i.e. table 2)
Take the corresponding grouped record set of business business dimension attribute of target service.
In step S316, according to the initial time and cut-off of the pipelined data stored at least one above-mentioned data grouping
Time is respectively handled the pipelined data stored at least one data grouping.
In an implementation, server fishes for target service from above-mentioned business business dimension attribute-grouped record table (i.e. table 2)
The corresponding grouped record set of business business dimension attribute after, each grouped record can be successively traversed, from above-mentioned such as table 3
According to initial time stamp (i.e. initial time) and deadline stamp (i.e. deadline) in flowing water business record table, fish for current
The corresponding pipelined data of data grouping set.At this point, server can carry out the pipelined data under current data grouping
Business procession.
It should be noted that be pipelined data in data grouping due to fishing for, and the flowing water in a data grouping
The limited amount of data, and quantity will not be excessive, in this way, transaction process can avoid the occurrence of big affairs problem.
Based on above content, this specification embodiment combines database from ID technology is increased, under specified services operation
" the business business dimension attribute of business " has corresponding ID to record respectively, and the time one when in conjunction with pipelined data storage is set to increasing
It is long, that is, the timestamp generated the feature increasing when each pipelined data stores, when the timestamp of specified pipelined data is inserted
What is generated after entering increases ID certainly, when ID value is the multiple of predetermined quantity, then it is assumed that and current pipelined data is the boundary of data grouping, when
Front border is the record of current data grouping, the time in true flowing water business record table to the timestamp in previous boundary
Stamp can corresponding with data grouping timestamp associate, be grouped in real time in pipelined data storing process to reach
Purpose.In the treatment process of pipelined data storage, if finding not setting up currently " the business business dimension attribute of business " corresponding
Timestamp record (i.e. from increasing ID) table, then can automatically create a data grouping, and flow current pipelined data as first
Water number is according to storing into the data grouping, to achieve the purpose that adaptive grouping.
On the basis of this specification embodiment, it can also extend for different business business dimension combinations of attributes, no
Same business business dimension attribute value combination etc., and the threshold value (i.e. predetermined quantity) of different data groupings is configured, to reach
The purpose of more flexible adaptive grouping.
Flowing water under the present invention lands process, the grouping for completing flowing water in real time, adaptive.
This specification embodiment provides a kind of processing method of pipelined data, by detecting the generation of target pipelined data
When, obtain the business dimension attribute of the corresponding target service of target pipelined data, then, the business dimension category based on target service
Property, by target pipelined data storage to the number of targets at least one corresponding data grouping of business dimension attribute of target service
According in grouping, in this way, whenever detecting that new pipelined data (i.e. target pipelined data) generates, it can according to target stream
Water number stores target pipelined data at least one corresponding data grouping according to the business dimension attribute of corresponding target service
In a data grouping in, thereby may be ensured that pipelined data processing real-time, meet the higher business scenario of timeliness
Demand.Moreover, because pipelined data is stored in a manner of an at least data grouping, therefore, can handled to be subsequent
When the pipelined data of target service, pipelined data therein can be fished for as unit of data grouping, and due to each data point
The limited amount of pipelined data in group, and quantity will not be excessive, this way it is possible to avoid occurring major issue in transaction process
Business problem.
Embodiment three
The above are the processing methods for the pipelined data that this specification embodiment provides, and are based on same thinking, this specification
Embodiment also provides a kind of processing unit of pipelined data, as shown in Figure 4.
The processing unit of the pipelined data includes: that business dimension attribute obtains module 401 and flowing water memory module 402,
In:
Business dimension attribute obtains module 401, for obtaining the target stream when detecting that target pipelined data generates
Water number according to corresponding target service business dimension attribute;
Flowing water memory module 402 stores the target pipelined data to described for being based on the business dimension attribute
In target data grouping at least one the corresponding data grouping of business dimension attribute.
In this specification embodiment, the flowing water memory module 402, comprising:
Information acquisition unit, if for including the business dimension attribute in database, obtain current time and
The grouping information of the target pipelined data;
Flowing water storing unit, for the grouping information according to the target pipelined data, by the time and the target
The corresponding storage of pipelined data is into target data grouping.
In this specification embodiment, the flowing water memory module 402, comprising:
It is grouped creating unit, if creating the business dimension for not including the business dimension attribute in database
The corresponding data grouping of attribute is spent, includes that target data is grouped in the data grouping of creation;
Flowing water storing unit is deposited for obtaining the current time, and by the time and target pipelined data correspondence
It stores up in the target data grouping of the database.
In this specification embodiment, described device further include:
Request receiving module, the pipelined data for receiving the target service handle request;
Be grouped determining module, for the business dimension attribute according to the target service, obtain from the database with
At least one corresponding data grouping of the business dimension attribute;
Processing module, for the initial time and cut-off according to the pipelined data stored at least one described data grouping
Time is respectively handled the pipelined data stored at least one described data grouping.
In this specification embodiment, predetermined quantity can be stored in the corresponding each data grouping of the business dimension attribute
Pipelined data.
In this specification embodiment, the grouping information is that the flowing water of the target pipelined data identifies, the flowing water mark
Knowing includes one or more of serial number, transaction odd numbers, alternatively, the grouping information is to be based on the target pipelined data
The relevant information of storage order under the business dimension attribute.
This specification embodiment provides a kind of processing unit of pipelined data, by detecting the generation of target pipelined data
When, obtain the business dimension attribute of the corresponding target service of target pipelined data, then, the business dimension category based on target service
Property, by target pipelined data storage to the number of targets at least one corresponding data grouping of business dimension attribute of target service
According in grouping, in this way, whenever detecting that new pipelined data (i.e. target pipelined data) generates, it can according to target stream
Water number stores target pipelined data at least one corresponding data grouping according to the business dimension attribute of corresponding target service
In a data grouping in, thereby may be ensured that pipelined data processing real-time, meet the higher business scenario of timeliness
Demand.Moreover, because pipelined data is stored in a manner of an at least data grouping, therefore, can handled to be subsequent
When the pipelined data of target service, pipelined data therein can be fished for as unit of data grouping, and due to each data point
The limited amount of pipelined data in group, and quantity will not be excessive, this way it is possible to avoid occurring major issue in transaction process
Business problem.
Example IV
The above are the processing units for the pipelined data that this specification embodiment provides, and are based on same thinking, this specification
Embodiment also provides a kind of processing equipment of pipelined data, as shown in Figure 5.
The processing equipment of the pipelined data can be server provided by the above embodiment or terminal device.
The processing equipment of pipelined data can generate bigger difference because configuration or performance are different, may include one or
More than one processor 501 and memory 502 can store one or more storages in memory 502 using journey
Sequence or data.Wherein, memory 502 can be of short duration storage or persistent storage.The application program for being stored in memory 502 can be with
Including one or more modules (diagram is not shown), each module may include one in the processing equipment to pipelined data
Family computer executable instruction.Further, processor 501 can be set to communicate with memory 502, in pipelined data
Processing equipment on execute memory 502 in series of computation machine executable instruction.The processing equipment of pipelined data can be with
Including one or more power supplys 503, one or more wired or wireless network interfaces 504, one or more
Input/output interface 505, one or more keyboards 506.
Specifically in the present embodiment, the processing equipment of pipelined data includes memory and one or more
Program, perhaps more than one program is stored in memory and one or more than one program may include one for one of them
A or more than one module, and each module may include that series of computation machine in processing equipment to pipelined data is executable
Instruction, and be configured to execute this or more than one program by one or more than one processor to include for carrying out
Following computer executable instructions:
When detecting that target pipelined data generates, the business dimension of the corresponding target service of the target pipelined data is obtained
Spend attribute;
Based on the business dimension attribute, the target pipelined data is stored corresponding extremely to the business dimension attribute
In target data grouping in a few data grouping.
It is described to be based on the business dimension attribute in this specification embodiment, by target pipelined data storage to institute
It states in the target data grouping at least one the corresponding data grouping of business dimension attribute, comprising:
If including the business dimension attribute in database, current time and the target pipelined data are obtained
Grouping information;
According to the grouping information of the target pipelined data, the time and the corresponding storage of the target pipelined data are arrived
In the target data grouping.
It is described to be based on the business dimension attribute in this specification embodiment, by target pipelined data storage to institute
It states in the target data grouping at least one the corresponding data grouping of business dimension attribute, comprising:
If in database not including the business dimension attribute, the corresponding data point of the business dimension attribute are created
Group includes that target data is grouped in the data grouping of creation;
The current time is obtained, and the time and the corresponding storage of the target pipelined data are arrived to the institute of the database
It states in target data grouping.
In this specification embodiment, further includes:
Receive the pipelined data processing request of the target service;
According to the business dimension attribute of the target service, obtain and the business dimension attribute phase from the database
At least one corresponding data grouping;
According to the initial time of the pipelined data stored at least one described data grouping and deadline, respectively to institute
The pipelined data stored at least one data grouping is stated to be handled.
In this specification embodiment, predetermined quantity can be stored in the corresponding each data grouping of the business dimension attribute
Pipelined data.
In this specification embodiment, the grouping information is that the flowing water of the target pipelined data identifies, the flowing water mark
Knowing includes one or more of serial number, transaction odd numbers, alternatively, the grouping information is to be based on the target pipelined data
The relevant information of storage order under the business dimension attribute.
This specification embodiment provides a kind of processing equipment of pipelined data, by detecting the generation of target pipelined data
When, obtain the business dimension attribute of the corresponding target service of target pipelined data, then, the business dimension category based on target service
Property, by target pipelined data storage to the number of targets at least one corresponding data grouping of business dimension attribute of target service
According in grouping, in this way, whenever detecting that new pipelined data (i.e. target pipelined data) generates, it can according to target stream
Water number stores target pipelined data at least one corresponding data grouping according to the business dimension attribute of corresponding target service
In a data grouping in, thereby may be ensured that pipelined data processing real-time, meet the higher business scenario of timeliness
Demand.Moreover, because pipelined data is stored in a manner of an at least data grouping, therefore, can handled to be subsequent
When the pipelined data of target service, pipelined data therein can be fished for as unit of data grouping, and due to each data point
The limited amount of pipelined data in group, and quantity will not be excessive, this way it is possible to avoid occurring major issue in transaction process
Business problem.
It is above-mentioned that this specification specific embodiment is described.Other embodiments are in the scope of the appended claims
It is interior.In some cases, the movement recorded in detail in the claims or step can be come according to the sequence being different from embodiment
It executes and desired result still may be implemented.In addition, process depicted in the drawing not necessarily require show it is specific suitable
Sequence or consecutive order are just able to achieve desired result.In some embodiments, multitasking and parallel processing be also can
With or may be advantageous.
In the 1990s, the improvement of a technology can be distinguished clearly be on hardware improvement (for example,
Improvement to circuit structures such as diode, transistor, switches) or software on improvement (improvement for method flow).So
And with the development of technology, the improvement of current many method flows can be considered as directly improving for hardware circuit.
Designer nearly all obtains corresponding hardware circuit by the way that improved method flow to be programmed into hardware circuit.Cause
This, it cannot be said that the improvement of a method flow cannot be realized with hardware entities module.For example, programmable logic device
(Programmable Logic Device, PLD) (such as field programmable gate array (Field Programmable Gate
Array, FPGA)) it is exactly such a integrated circuit, logic function determines device programming by user.By designer
Voluntarily programming comes a digital display circuit " integrated " on a piece of PLD, designs and makes without asking chip maker
Dedicated IC chip.Moreover, nowadays, substitution manually makes IC chip, this programming is also used instead mostly " is patrolled
Volume compiler (logic compiler) " software realizes that software compiler used is similar when it writes with program development,
And the source code before compiling also write by handy specific programming language, this is referred to as hardware description language
(Hardware Description Language, HDL), and HDL is also not only a kind of, but there are many kind, such as ABEL
(Advanced Boolean Expression Language)、AHDL(Altera Hardware Description
Language)、Confluence、CUPL(Cornell University Programming Language)、HDCal、JHDL
(Java Hardware Description Language)、Lava、Lola、MyHDL、PALASM、RHDL(Ruby
Hardware Description Language) etc., VHDL (Very-High-Speed is most generally used at present
Integrated Circuit Hardware Description Language) and Verilog.Those skilled in the art also answer
This understands, it is only necessary to method flow slightly programming in logic and is programmed into integrated circuit with above-mentioned several hardware description languages,
The hardware circuit for realizing the logical method process can be readily available.
Controller can be implemented in any suitable manner, for example, controller can take such as microprocessor or processing
The computer for the computer readable program code (such as software or firmware) that device and storage can be executed by (micro-) processor can
Read medium, logic gate, switch, specific integrated circuit (Application Specific Integrated Circuit,
ASIC), the form of programmable logic controller (PLC) and insertion microcontroller, the example of controller includes but is not limited to following microcontroller
Device: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320 are deposited
Memory controller is also implemented as a part of the control logic of memory.It is also known in the art that in addition to
Pure computer readable program code mode is realized other than controller, can be made completely by the way that method and step is carried out programming in logic
Controller is obtained to come in fact in the form of logic gate, switch, specific integrated circuit, programmable logic controller (PLC) and insertion microcontroller etc.
Existing identical function.Therefore this controller is considered a kind of hardware component, and to including for realizing various in it
The device of function can also be considered as the structure in hardware component.Or even, it can will be regarded for realizing the device of various functions
For either the software module of implementation method can be the structure in hardware component again.
System, device, module or the unit that above-described embodiment illustrates can specifically realize by computer chip or entity,
Or it is realized by the product with certain function.It is a kind of typically to realize that equipment is computer.Specifically, computer for example may be used
Think personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media play
It is any in device, navigation equipment, electronic mail equipment, game console, tablet computer, wearable device or these equipment
The combination of equipment.
For convenience of description, it is divided into various units when description apparatus above with function to describe respectively.Certainly, implementing this
The function of each unit can be realized in the same or multiple software and or hardware when specification one or more embodiment.
It should be understood by those skilled in the art that, the embodiment of this specification can provide as method, system or computer journey
Sequence product.Therefore, complete hardware embodiment, complete software embodiment or knot can be used in this specification one or more embodiment
The form of embodiment in terms of conjunction software and hardware.Moreover, this specification one or more embodiment can be used at one or more
A wherein includes computer-usable storage medium (the including but not limited to magnetic disk storage, CD- of computer usable program code
ROM, optical memory etc.) on the form of computer program product implemented.
The embodiment of this specification is referring to the method, equipment (system) and computer journey according to this specification embodiment
The flowchart and/or the block diagram of sequence product describes.It should be understood that flow chart and/or box can be realized by computer program instructions
The combination of the process and/or box in each flow and/or block and flowchart and/or the block diagram in figure.It can provide this
A little computer program instructions are to general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices
Processor to generate a machine so that the finger executed by the processor of computer or other programmable data processing devices
It enables and generates to specify in one or more flows of the flowchart and/or one or more blocks of the block diagram
The device of function.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
In a typical configuration, calculating equipment includes one or more processors (CPU), input/output interface, net
Network interface and memory.
Memory may include the non-volatile memory in computer-readable medium, random access memory (RAM) and/or
The forms such as Nonvolatile memory, such as read-only memory (ROM) or flash memory (flash RAM).Memory is computer-readable medium
Example.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method
Or technology come realize information store.Information can be computer readable instructions, data structure, the module of program or other data.
The example of the storage medium of computer includes, but are not limited to phase change memory (PRAM), static random access memory (SRAM), moves
State random access memory (DRAM), other kinds of random access memory (RAM), read-only memory (ROM), electric erasable
Programmable read only memory (EEPROM), flash memory or other memory techniques, read-only disc read only memory (CD-ROM) (CD-ROM),
Digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape magnetic disk storage or other magnetic storage devices
Or any other non-transmission medium, can be used for storage can be accessed by a computing device information.As defined in this article, it calculates
Machine readable medium does not include temporary computer readable media (transitory media), such as the data-signal and carrier wave of modulation.
It should also be noted that, the terms "include", "comprise" or its any other variant are intended to nonexcludability
It include so that the process, method, commodity or the equipment that include a series of elements not only include those elements, but also to wrap
Include other elements that are not explicitly listed, or further include for this process, method, commodity or equipment intrinsic want
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including described want
There is also other identical elements in the process, method of element, commodity or equipment.
It will be understood by those skilled in the art that the embodiment of this specification can provide as the production of method, system or computer program
Product.Therefore, this specification one or more embodiment can be used complete hardware embodiment, complete software embodiment or combine software
With the form of the embodiment of hardware aspect.Moreover, this specification one or more embodiment can be used it is one or more wherein
It include computer-usable storage medium (including but not limited to magnetic disk storage, CD-ROM, the light of computer usable program code
Learn memory etc.) on the form of computer program product implemented.
This specification one or more embodiment can computer executable instructions it is general on
It hereinafter describes, such as program module.Generally, program module includes executing particular task or realization particular abstract data type
Routine, programs, objects, component, data structure etc..Can also practice in a distributed computing environment this specification one or
Multiple embodiments, in these distributed computing environments, by being executed by the connected remote processing devices of communication network
Task.In a distributed computing environment, the local and remote computer that program module can be located at including storage equipment is deposited
In storage media.
All the embodiments in this specification are described in a progressive manner, same and similar portion between each embodiment
Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.Especially for system reality
For applying example, since it is substantially similar to the method embodiment, so being described relatively simple, related place is referring to embodiment of the method
Part explanation.
The foregoing is merely the embodiments of this specification, are not limited to this specification.For art technology
For personnel, this specification can have various modifications and variations.It is all made any within the spirit and principle of this specification
Modification, equivalent replacement, improvement etc., should be included within the scope of the claims of this specification.