CN107886352A - A kind of method and system of advertisement clearing - Google Patents

A kind of method and system of advertisement clearing Download PDF

Info

Publication number
CN107886352A
CN107886352A CN201711023402.3A CN201711023402A CN107886352A CN 107886352 A CN107886352 A CN 107886352A CN 201711023402 A CN201711023402 A CN 201711023402A CN 107886352 A CN107886352 A CN 107886352A
Authority
CN
China
Prior art keywords
current
advertiser
data
polymerization
log data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201711023402.3A
Other languages
Chinese (zh)
Other versions
CN107886352B (en
Inventor
肖培林
李东升
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Weimeng Chuangke Network Technology China Co Ltd
Original Assignee
Weimeng Chuangke Network Technology China Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Weimeng Chuangke Network Technology China Co Ltd filed Critical Weimeng Chuangke Network Technology China Co Ltd
Priority to CN201711023402.3A priority Critical patent/CN107886352B/en
Publication of CN107886352A publication Critical patent/CN107886352A/en
Application granted granted Critical
Publication of CN107886352B publication Critical patent/CN107886352B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G06Q30/0273—Determination of fees for advertising
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G06Q30/0277—Online advertisement
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/55—Push-based network services

Landscapes

  • Business, Economics & Management (AREA)
  • Engineering & Computer Science (AREA)
  • Finance (AREA)
  • Strategic Management (AREA)
  • Accounting & Taxation (AREA)
  • Development Economics (AREA)
  • Game Theory and Decision Science (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Physics & Mathematics (AREA)
  • General Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The embodiments of the invention provide a kind of method and system of advertisement clearing, this method includes：Multiple advertisement log datas are obtained, and determine the identification information of each self-corresponding advertiser of each advertisement log data；Each advertisement log data is respectively sent in the polymerization queue of each self-corresponding advertiser；The advertisement log data in polymerization queue to each advertiser polymerize parallel；For the aggregated data of each advertiser's advertisement log data, following steps are performed respectively：Judge whether the aggregated data of the main advertisement log data of Current ad meets to make a reservation for any pushing condition in multiple pushing conditions, when meeting, the aggregated data of the main advertisement log data of Current ad is sent into buffer queue corresponding to Current ad master；Each aggregated data is sent into statistics association corresponding with current cache queue journey, and in write into Databasce.By the present invention, the write efficiency of database is drastically increased, can efficiently handle mass advertising daily record data in time.

Description

A kind of method and system of advertisement clearing
Technical field
The present invention relates to Internet advertising to settle accounts technical field, more particularly to a kind of method and system of advertisement clearing.
Background technology
Existing online advertisement clearing are typically all to calculate advertisement by counting the method for advertisement exposure or interactive daily record The main amount of money consumed, after the amount of money consumed meets or exceeds the limit of advertiser's setting, notice advertisement is offline.When certain After one advertisement is offline, consumption would not be produced again, also will not occupied bandwidth resource again.However, ad log amount can be with wide Accuse exposure growth and increase, if in the short time daily record amount increase sharply or same advertiser largely launch, reached clearing or The processing upper limit of person's database, may result in can not the unnecessary daily record of timely processing, thus it is offline logical to send advertisement in time Know, advertisement will launch always, occupied bandwidth resource, can not but bring advertising income.
, can be by being parsed to the clearing advertisement received, according to wide in the method for solving the problems, such as the super throwing of advertisement at present Announcement advertising plan of advocating peace treats clearing advertisement and carries out one-level polymerization, and each polymerization ad data after polymerization is pushed to extensively respectively In the buffering queue for accusing main corresponding sequence number, to reduce the daily record quantity of advertising plan, and by each poly- in each buffering queue Close ad data and carry out secondary polymerization, to cut down queue length, improve the speed of clearing.
In process of the present invention is realized, inventor has found that at least there are the following problems in the prior art：Although prior art Solve the problems, such as single advertiser largely super throwing caused by dispensing in a short time, but in place of still having some shortcomings：
1st, distributed pretreatment module and statistical module dilatation are dumb：When daily record amount increases severely, pretreatment module is present Its processing limit may be greatly up to, and because prior art is limited by software architecture, front end daily record shunting underaction, from And cause pretreatment module can not quick dilatation, therefore, once occur problem, will appear from a large amount of daily records can not obtain timely processing Situation so that clearing produce delay；
2nd, aggregating algorithm is not intelligent enough：Prior art is polymerize using fixed velocity factor to buffering queue, therefore Only when queue produces more accumulation, polymerization granularity can be improved, so that speed up processing, still, When digesting less heap volume data, which can not digest heap volume data in time.
The content of the invention
The embodiment of the present invention provides a kind of method and system of advertisement clearing, can efficiently handle mass advertising day in time Will data.
On the one hand, the embodiments of the invention provide a kind of method of advertisement clearing, including：
Multiple advertisement log datas are obtained, and determine the mark letter of each self-corresponding advertiser of each advertisement log data Breath；
According to the identification information of fixed each each self-corresponding advertiser of advertisement log data, by each advertisement Daily record data is respectively sent in the polymerization queue of each self-corresponding advertiser；
The advertisement log data in polymerization queue to each advertiser polymerize parallel, obtains each advertiser's ad log The aggregated data of data；
For the aggregated data of each advertiser's advertisement log data, following steps are performed respectively：According to Current ad The aggregated data and current time of main advertisement log data, judge whether the aggregated data of the main advertisement log data of Current ad is full Foot makes a reservation for any pushing condition in multiple pushing conditions, when meeting any predetermined pushing condition, Current ad master is wide The aggregated data for accusing daily record data is sent into buffer queue corresponding to Current ad master；
In each buffer queue, each aggregated data is sent to statistics association corresponding with current cache queue journey In, and journey is assisted by each aggregated data write into Databasce by the statistics.
On the other hand, the embodiments of the invention provide a kind of system of advertisement clearing, including：
Submodule is accessed, for obtaining multiple advertisement log datas, and determines that each advertisement log data is each self-corresponding The identification information of advertiser；
Submodule is pre-processed, is additionally operable to the mark according to fixed each each self-corresponding advertiser of advertisement log data Information, each advertisement log data is respectively sent in the polymerization queue of each self-corresponding advertiser；
It polymerize submodule, polymerize parallel for the advertisement log data in the polymerization queue to each advertiser, obtains each The aggregated data of individual advertiser's advertisement log data；
Judging submodule, the aggregated data for each advertiser's advertisement log data is additionally operable to, performed respectively as follows Step：According to the aggregated data and current time of the main advertisement log data of Current ad, the main ad log number of Current ad is judged According to aggregated data whether meet to make a reservation for any pushing condition in multiple pushing conditions, when meeting any predetermined pushing condition When, the aggregated data of the main advertisement log data of Current ad is sent into buffer queue corresponding to Current ad master；
Statistic submodule, in each buffer queue, by each aggregated data send to current cache queue In corresponding statistics association journey, and journey is assisted by each aggregated data write into Databasce by the statistics.
Above-mentioned technical proposal has the advantages that：It ensure that the advertisement log data of same advertiser can all be sent It polymerize to same in queue, and the advertisement log data of same advertiser is handled in same polymerization association journey, avoids same The daily record data of advertiser is dispersed to the different advertisement log data efficiency reductions for polymerizeing and same advertiser being handled during queue Situation occurs, and drastically increases the data-handling efficiency of the advertisement log data of each same advertiser；By predefining not With pushing condition, realize and rapidly send the advertisement log data after polymerization to corresponding statistics queue, for can and When processing mass advertising daily record data provide important premise guarantee；Meanwhile avoid during write into Databasce not The situation for needing to lock the advertisement log data of the advertiser occurs, so as to drastically increase the write-in of database effect Rate, further, drastically increase the ability of timely processing mass advertising daily record data.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing There is the required accompanying drawing used in technology description to be briefly described, it should be apparent that, drawings in the following description are only this Some embodiments of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can be with Other accompanying drawings are obtained according to these accompanying drawings.
Fig. 1 is the method flow diagram that advertisement is settled accounts in one embodiment of the invention；
Fig. 2 is the system structure diagram that advertisement is settled accounts in another embodiment of the present invention；
Fig. 3 is the structural representation of capacity of queue in one embodiment of the present invention；
Fig. 4 is polymerizing factor functional image schematic diagram in one embodiment of the present invention；
Fig. 5 is the system architecture diagram that advertisement is settled accounts in one embodiment of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete Site preparation describes, it is clear that described embodiment is only part of the embodiment of the present invention, rather than whole embodiments.It is based on Embodiment in the present invention, those of ordinary skill in the art are obtained every other under the premise of creative work is not made Embodiment, belong to the scope of protection of the invention.
As shown in figure 1, be a kind of method flow diagram of advertisement clearing in the embodiment of the present invention, including：
101st, multiple advertisement log datas are obtained, and determine the mark of each self-corresponding advertiser of each advertisement log data Information；
102nd, will be described each according to the identification information of fixed each each self-corresponding advertiser of advertisement log data Advertisement log data is respectively sent in the polymerization queue of each self-corresponding advertiser；
103rd, the advertisement log data in the polymerization queue to each advertiser polymerize parallel, obtains each advertiser's advertisement The aggregated data of daily record data；
104th, for the aggregated data of each advertiser's advertisement log data, following steps are performed respectively：According to current The aggregated data and current time of advertiser's advertisement log data, judging the aggregated data of the main advertisement log data of Current ad is It is no to meet to make a reservation for any pushing condition in multiple pushing conditions, when meeting any predetermined pushing condition, by Current ad The aggregated data of main advertisement log data is sent into buffer queue corresponding to Current ad master；
105th, in each buffer queue, each aggregated data is sent to statistics corresponding with current cache queue and assisted Cheng Zhong, and journey is assisted by each aggregated data write into Databasce by the statistics.
Alternatively, in addition to：
The pretreatment association number of passes amount pre-seted is obtained, creates the multiple pretreatments for assisting number of passes amount to match with the pretreatment Assist journey；
Obtain the polymerization number of queues that pre-sets, create with it is described polymerize multiple polymerization queues that number of queues matches with And it polymerize the one-to-one multiple polymerization associations journey of queue with the multiple, and determine that each polymerization queue polymerize corresponding to unique Queue sequence number；
Obtain the statistics number of queues that pre-sets, create multiple statistics queues for matching with the statistics number of queues with And journey is assisted with the one-to-one multiple statistics of the multiple statistics queue, and determine the corresponding unique statistics of each statistics queue Queue sequence number；
Wherein, it is described to obtain multiple advertisement log datas, and determine each self-corresponding advertiser of each advertisement log data Identification information, including：
Journey is assisted by the multiple pretreatment, each subregion subscribed to from distributed post in message system Kafka obtains Multiple advertisement log datas, and each advertisement log data being analyzed and acquired by respectively；
According to each advertisement log data parsed, the mark of each self-corresponding advertiser of each advertisement log data is determined Know information；
Wherein, each advertisement log data parsed includes the mark of advertising plan corresponding to each advertisement log data Information, the identification information of advertiser and advertisement consumption figures；
Wherein, each advertisement log data that the basis parses, determine that each advertisement log data is each self-corresponding After the identification information of advertiser, in addition to：
According to the identification information of fixed each each self-corresponding advertiser of advertisement log data, at tentation data Adjustment method, data processing is carried out to the identification information of each advertiser in each subregion in Kafka.
Preferably, the identification information according to fixed each each self-corresponding advertiser of advertisement log data, will Each advertisement log data is respectively sent in the polymerization queue of each self-corresponding advertiser, including：
For each advertisement log data, following operation is performed respectively：
According to the identification information of advertiser corresponding to fixed Current ad daily record data, the mark of the advertiser is calculated Know information divided by the remainder polymerizeing obtained by number of queues, obtain the first remainder values；
It is determined that it polymerize queue sequence number with what first remainder values matched；
Current ad daily record data is sent into polymerization queue corresponding to the polymerization queue sequence number determined.
Preferably, the advertisement log data in the polymerization queue to each advertiser polymerize parallel, obtains each wide The aggregated data of main advertisement log data is accused, including：
According to the identification information of fixed each each self-corresponding advertiser of advertisement log data and the mark of advertising plan Know information, polymerize the advertisement log data of each advertising plan of each advertiser parallel, obtain each of each advertiser The aggregated data of individual advertising plan；Wherein, the aggregated data of all advertising plans of any advertiser forms advertiser's advertisement The aggregated data of daily record data；
Wherein, the advertisement log data of each advertising plan for polymerizeing each advertiser parallel, obtain each wide After the aggregated data for accusing each main advertising plan, in addition to：
According to the aggregated data of each advertising plan of each advertiser, by each advertisement of same advertisement master Each advertisement consumption figures of plan merges, and obtains the polymerization consumption figures of each advertising plan of each advertiser；
Wherein, for each advertiser each advertising plan aggregated data, perform following steps respectively：
Current time is obtained, according to the polymerization consumption figures and current time of the Current ad plan of Current ad master, is judged Whether satisfaction makes a reservation for any pushing condition in multiple pushing conditions；
When meeting any predetermined pushing condition, according to the identification information of Current ad master, calculate the advertiser's Remainder obtained by identification information divided by the statistics number of queues, obtains the second remainder values；
It is determined that the buffer queue sequence number of the current aggregated data to match with second remainder values；
The aggregated data of the Current ad plan of the Current ad master is sent corresponding to the buffer queue sequence number Buffer queue in.
Preferably, whether the aggregated data for judging the main advertisement log data of Current ad meets to make a reservation for multiple push bars Any pushing condition in part, including：
Gather corresponding to the aggregated data for obtaining the polymerization bar number threshold value pre-seted and the Current ad plan of Current ad master Syzygy number, according to the polymerization bar number threshold value and the polymerizing factor, the first push threshold value is determined, and judge Current ad master's Whether the polymerization consumption figures of Current ad plan reaches the first push threshold value, if reaching, meets the first predetermined push Condition；
The polymerization threshold value pre-seted is obtained, according to the polymerization threshold value and the polymerizing factor, determines second Threshold value is pushed, and judges whether the polymerization consumption figures of the Current ad plan of Current ad master reaches the second push threshold value, If reaching, meet the second predetermined pushing condition；
The first difference threshold pre-seted is obtained, it is determined that away from the current wide of the last push Current ad master of current time The first push time of the advertisement log data of plan is accused, current time is calculated and the very first time of the described first push time is poor Value, if the first time difference is more than first difference threshold, meets the 3rd predetermined pushing condition；
The second difference threshold for pre-seting is obtained, it is determined that away from current time the last time by the Current ad of Current ad master The advertisement log data of plan pushes to the second push time of buffer queue, calculates current time and the described second push time The second time difference, if second time difference is more than second difference threshold, meet predetermined 4th push article Part.
Preferably, polymerizing factor corresponding to the aggregated data of the Current ad plan for obtaining Current ad master, including：
According to the aggregated data of the Current ad plan of Current ad master, the Current ad meter of the Current ad master is determined The buffer queue sequence number for the aggregated data drawn, and determine that the element in buffer queue corresponding to the buffer queue sequence number accounts for The capacity ratio of the buffer queue capacity；
Based on predetermined computation rule, according to the capacity ratio, the Current ad plan of the Current ad master is determined Aggregated data polymerizing factor.
As shown in Fig. 2 be a kind of system structure diagram of advertisement clearing in the embodiment of the present invention, including：
Submodule 21 is accessed, for obtaining multiple advertisement log datas, and determines that each advertisement log data each corresponds to Advertiser identification information；
Submodule 22 is pre-processed, is additionally operable to the mark according to fixed each each self-corresponding advertiser of advertisement log data Know information, each advertisement log data is respectively sent in the polymerization queue of each self-corresponding advertiser；
It polymerize submodule 23, polymerize parallel for the advertisement log data in the polymerization queue to each advertiser, obtain The aggregated data of each advertiser's advertisement log data；
Judging submodule 24, the aggregated data for each advertiser's advertisement log data is additionally operable to, is performed respectively such as Lower step：According to the aggregated data and current time of the main advertisement log data of Current ad, the main ad log of Current ad is judged Whether the aggregated data of data meets to make a reservation for any pushing condition in multiple pushing conditions, when meeting any predetermined push bar During part, the aggregated data of the main advertisement log data of Current ad is sent into buffer queue corresponding to Current ad master；
Statistic submodule 25, in each buffer queue, by each aggregated data send to current cache team In statistics association journey corresponding to row, and journey is assisted by each aggregated data write into Databasce by the statistics.
Alternatively, in addition to：
First obtains and creates submodule, assists number of passes amount for obtaining the pretreatment pre-seted, creates and the pretreatment Multiple pretreatments association journey that association's number of passes amount matches；
Second obtains and creates submodule, for obtaining the polymerization number of queues pre-seted, creates and polymerize queue with described The multiple polymerization queues and polymerize the one-to-one multiple polymerization associations journey of queue with the multiple that quantity matches, and determine every Polymerization queue sequence number corresponding to one polymerization queue is unique；
3rd obtains and creates submodule, for obtaining the statistics number of queues pre-seted, creates and the statistics queue The multiple statistics queues and assist journey with the one-to-one multiple statistics of the multiple statistics queue that quantity matches, and determine every The corresponding unique statistics queue sequence number of one statistics queue；
Wherein, the access submodule, including：
Acquisition and resolution unit, for assisting journey by the multiple pretreatment, message system is subscribed to from distributed post Each subregion in Kafka obtains multiple advertisement log datas, and each advertisement log data being analyzed and acquired by respectively；
First determining unit, for according to each advertisement log data parsed, determining that each advertisement log data is each The identification information of self-corresponding advertiser；
Wherein, each advertisement log data parsed includes the mark of advertising plan corresponding to each advertisement log data Information, the identification information of advertiser and advertisement consumption figures；
Wherein, first determining unit, is additionally operable to
According to the identification information of fixed each each self-corresponding advertiser of advertisement log data, at tentation data Adjustment method, data processing is carried out to the identification information of each advertiser in each subregion in Kafka.
Preferably, the pretreatment submodule, is specifically used for
For each advertisement log data, following operation is performed respectively：
According to the identification information of advertiser corresponding to fixed Current ad daily record data, the mark of the advertiser is calculated Know information divided by the remainder polymerizeing obtained by number of queues, obtain the first remainder values；
It is determined that it polymerize queue sequence number with what first remainder values matched；
Current ad daily record data is sent into polymerization queue corresponding to the polymerization queue sequence number determined.
Preferably, the polymerization submodule, specifically for each self-corresponding according to fixed each advertisement log data The identification information of advertiser and the identification information of advertising plan, polymerize the advertisement of each advertising plan of each advertiser parallel Daily record data, obtain the aggregated data of each advertising plan of each advertiser；Wherein, all advertisement meters of any advertiser The aggregated data drawn forms the aggregated data of advertiser's advertisement log data；
Wherein, the polymerization submodule, is additionally operable to
According to the aggregated data of each advertising plan of each advertiser, by each advertisement of same advertisement master Each advertisement consumption figures of plan merges, and obtains the polymerization consumption figures of each advertising plan of each advertiser；
Wherein, the judging submodule, specifically for the aggregate number of each advertising plan for each advertiser According to performing following steps respectively：
Current time is obtained, according to the polymerization consumption figures and current time of the Current ad plan of Current ad master, is judged Whether satisfaction makes a reservation for any pushing condition in multiple pushing conditions；
When meeting any predetermined pushing condition, according to the identification information of Current ad master, calculate the advertiser's Remainder obtained by identification information divided by the statistics number of queues, obtains the second remainder values；
It is determined that the buffer queue sequence number of the current aggregated data to match with second remainder values；
The aggregated data of the Current ad plan of the Current ad master is sent corresponding to the buffer queue sequence number Buffer queue in.
Preferably, the judging submodule, including：
First judging unit, the polymerization bar number threshold value pre-seted for acquisition and the Current ad plan of Current ad master Polymerizing factor corresponding to aggregated data, according to the polymerization bar number threshold value and the polymerizing factor, the first push threshold value is determined, and Judge whether the polymerization consumption figures of the Current ad plan of Current ad master reaches the first push threshold value, it is full if reaching The first predetermined pushing condition of foot；
Second judging unit, for obtaining the polymerization threshold value that pre-sets, according to the polymerization threshold value with it is described Polymerizing factor, the second push threshold value is determined, and judge whether the polymerization consumption figures of the Current ad plan of Current ad master reaches The second push threshold value, if reaching, meets the second predetermined pushing condition；
3rd judging unit, for obtaining the first difference threshold pre-seted, it is determined that away from the last push of current time The first push time of the advertisement log data of the Current ad plan of Current ad master, calculate current time and pushed away with described first The first time difference of time is sent, if the first time difference is more than first difference threshold, meets predetermined the 3rd Pushing condition；
4th judging unit, for obtaining the second difference threshold pre-seted, it is determined that ought away from current time the last time The advertisement log data of the Current ad plan of preceding advertiser pushes to the second push time of buffer queue, calculates current time It is full if second time difference is more than second difference threshold with the second time difference of the described second push time The 4th predetermined pushing condition of foot.
Preferably, the judging submodule, in addition to：
Second determining unit, for the aggregated data of the Current ad plan according to Current ad master, determine described current The buffer queue sequence number of the aggregated data of the Current ad plan of advertiser, and determine corresponding to the buffer queue sequence number Element in buffer queue accounts for the capacity ratio of the buffer queue capacity；
Computing unit, for based on predetermined computation rule, according to the capacity ratio, determining the Current ad master's The polymerizing factor of the aggregated data of Current ad plan.
Above-mentioned technical proposal of the embodiment of the present invention has the advantages that：It ensure that the ad log number of same advertiser According to can all be sent in same polymerization queue, and the ad log number of same advertiser is handled in same polymerization association journey According to the daily record data for avoiding same advertiser is dispersed to the ad log number that same advertiser is handled during different polymerization queues Situation about being reduced according to efficiency occurs, and drastically increases the data-handling efficiency of the advertisement log data of each same advertiser； By predefined different pushing condition, realize and rapidly send the advertisement log data after polymerization to corresponding statistics team Row, for can timely processing mass advertising daily record data provide important premise guarantee；Meanwhile avoid in write into Databasce During situation about the advertisement log data of the advertiser need not be locked occur, so as to drastically increase data The write efficiency in storehouse, further, drastically increase the ability of timely processing mass advertising daily record data.
Above-mentioned technical proposal of the embodiment of the present invention is described in detail below in conjunction with application example：
Application example of the present invention is intended to efficiently handle mass advertising daily record data in time.
As shown in figure 1, for example, in online advertisement settlement system A, ad log file is stored in distributed post subscription In message system Kafka, online advertisement settlement system A pulling datas from kafka each partition, and determine each wide The identification information of each self-corresponding advertiser of daily record data is accused, e.g., the identification information of advertiser is corresponding to advertisement log data 1 1001, the identification information of advertiser corresponding to advertisement log data 2 is 1002, and then, advertisement log data 1 is sent to advertisement In main 1001 corresponding polymerization queue, such as it polymerize queue 1, advertisement log data 2 is sent to polymerization corresponding to advertiser 1002 In queue, such as it polymerize queue 2；For advertiser 1001 and advertiser 1002, to polymerization queue 1 and the advertisement polymerizeing in queue 2 The advertisement log data of master 1001 and advertiser 1002 polymerize parallel, obtain the aggregate number of the advertisement log data of advertiser 1001 According to the aggregated data of the advertisement log data with advertiser 1002；For the aggregated data of the advertisement log data of advertiser 1001, Following steps are performed respectively：Current time is obtained, according to the aggregated data and current time of the advertisement log data of advertiser 1001, Judge whether to meet to make a reservation for any pushing condition in multiple pushing conditions, will be wide when meeting any predetermined pushing condition The aggregated data for accusing main 1001 advertisement log datas is sent to corresponding buffer queue, such as buffer queue 1；For advertiser 1002 The aggregated data of advertisement log data, performs following steps respectively：Current time is obtained, according to the ad log number of advertiser 1002 According to aggregated data and current time, judge whether meet make a reservation for multiple pushing conditions in any pushing condition, when meet appoint During one predetermined pushing condition, the aggregated data of the advertisement log data of advertiser 1002 is sent to corresponding buffer queue, such as Buffer queue 2；The aggregated data of the advertisement log data of advertiser 1001 is sent to right with buffer queue 1 in buffer queue 1 In the statistics association journey answered, such as statistics association journey 1, and by counting association's journey 1 by the aggregated data of the advertisement log data of advertiser 1001 In write into Databasce, in buffer queue 2 by the aggregated data of the advertisement log data of advertiser 1002 send to buffer queue 2 In corresponding statistics association journey, such as statistics association journey 2, and by counting association's journey 2 by the aggregate number of the advertisement log data of advertiser 1002 According in write into Databasce.
It should be noted that those skilled in the art are it can be appreciated that Kafka (Apache Kafka) is that a kind of height is handled up The distributed post of amount subscribes to message system, and it can handle the everything flow data in the website of consumer's scale.Webpage Browse, search and the key factor that the behavior of other users is many social functions on modern network, these data It is often as the requirement of handling capacity and is solved by handling daily record and log aggregation.For as Hadoop (one by The distributed system architecture of Apache funds club exploitation) the same daily record data and off-line analysis system, but require The limitation handled in real time, Kafka solution can be used.Kafka purpose be by Hadoop loaded in parallel mechanism come On unified line and offline Message Processing, also for providing real-time consumption by cluster.Assist journey, also known as micro-line journey, English Literary fame is Coroutine.Journey is assisted as subroutine, and a kind of program assembly, for subroutine, association Cheng Gengwei is general With it is flexible, but in practice using no subroutine it is extensive like that.Association's journey is derived from Simula and Modula-2, but also has it He supports language.
In a preferred embodiment, this method also includes：Obtain pre-set pretreatment association number of passes amount, create with it is described pre- Multiple pretreatments association journey that processing association number of passes amount matches；The polymerization number of queues pre-seted is obtained, creates and polymerize team with described The multiple polymerization queues and polymerize the one-to-one multiple polymerization associations journey of queue with the multiple that number of columns matches, and determine Polymerization queue sequence number corresponding to each polymerization queue is unique；The statistics number of queues pre-seted is obtained, is created and the system The multiple statistics queues and assist journey with the one-to-one multiple statistics of the multiple statistics queue that meter number of queues matches, and Determine the corresponding unique statistics queue sequence number of each statistics queue.
For example, in online advertisement settlement system A, obtain the pretreatment pre-seted and assist number of passes amount, such as 2, create and pre-process 2 pretreatment association journeys that association's number of passes amount matches；The polymerization number of queues pre-seted, such as 3 are obtained, is created with polymerizeing number of queues 3 to match polymerize queues and polymerize the one-to-one 3 polymerizations association journey of queue with this 3, wherein, Hash letter can be passed through Number realizes polymerization queue and the one-to-one relationship polymerizeing between association's journey, and determines that each polymerization queue polymerize corresponding to unique Queue sequence number, such as 1,2 and 3；The statistics number of queues pre-seted, such as 4 are obtained, creates 4 to match with statistics number of queues It is individual to count queue and assist journey with one-to-one 4 statistics of this 4 statistics queues, wherein, hash function can be passed through and realize system The one-to-one relationship between queue and statistics association journey is counted, and determines the corresponding unique statistics queue sequence of each statistics queue Number, such as 1,2,3 and 4.
By the present embodiment, because each module is all made up of association's journey, realizing can adjust according to portfolio freedom and flexibility The quantity of journey is assisted, greatly ensure that the treatment effeciency of data, further, when advertisement log data amount increases severely, being can Timely processing mass advertising daily record data provides necessary premise guarantee.
Wherein, it is described to obtain multiple advertisement log datas, and determine each self-corresponding advertiser of each advertisement log data Identification information, including：Journey is assisted by the multiple pretreatment, subscribed to from distributed post each in message system Kafka Subregion obtains multiple advertisement log datas, and each advertisement log data being analyzed and acquired by respectively；It is each according to what is parsed Advertisement log data, determine the identification information of each self-corresponding advertiser of each advertisement log data.
Wherein, each advertisement log data parsed includes the mark of advertising plan corresponding to each advertisement log data Information, the identification information of advertiser and advertisement consumption figures.
For example, in online advertisement settlement system A, by the 2 pretreatment association journeys created, subscribe to and disappear from distributed post Each subregion in breath system Kafka obtains multiple advertisement log datas, and each ad log number being analyzed and acquired by respectively According to；According to each advertisement log data parsed, determine that the mark of each self-corresponding advertiser of each advertisement log data is believed Breath, the identification information of advertiser as corresponding to advertisement log data 1 are 1001, the mark of advertiser corresponding to advertisement log data 2 It is 1002 to know information.
Wherein, each advertisement log data that the basis parses, determine that each advertisement log data is each self-corresponding After the identification information of advertiser, in addition to：According to the mark of each self-corresponding advertiser of each advertisement log data parsed Know information, by tentation data Processing Algorithm, the identification information of each advertiser is entered in each subregion in Kafka Row data processing.
For example, in online advertisement settlement system A, according to each self-corresponding advertiser of each advertisement log data parsed Identification information, by tentation data Processing Algorithm, such as hash algorithm, to storage in each subregion partition of Kafka Advertisement log data carries out Hash processing by the identification information of advertiser, to cause each follow-up pretreatment association's journey from least one Each advertisement log data of each advertiser of each partitioned storage storage is obtained in individual subregion.
By the present embodiment, by carrying out Hash processing to the advertisement log data of different advertisers, ensure that same The daily record of advertiser is only possible to be present in same Kafka partition, avoids each pretreatment association journey reprocessing phase The situation of same advertisement log data, so as to ensure that same Kafka partition advertisement log data can only be by only Individual pretreatment association journey processing one by one；Because the Kafka partition that different pretreatments association journey is read are different, it is achieved thereby that The data separating of advertiser's dimension, to be provided subsequently when database manipulation is carried out with regard to locking need not be carried out to advertiser Necessary precondition.
In a preferred embodiment, it is each self-corresponding wide according to fixed each advertisement log data described in step 102 Main identification information is accused, each advertisement log data is respectively sent in the polymerization queue of each self-corresponding advertiser, Including：For each advertisement log data, following operation is performed respectively：It is corresponding according to fixed Current ad daily record data Advertiser identification information, calculate the advertiser identification information divided by it is described polymerization number of queues obtained by remainder, obtain To the first remainder values；It is determined that it polymerize queue sequence number with what first remainder values matched；Current ad daily record data is sent out Deliver in polymerization queue corresponding to the polymerization queue sequence number determined.
For example, in online advertisement settlement system A, for advertisement log data 1, following operation is performed respectively：According to true The identification information 1001 of advertiser corresponding to fixed Current ad daily record data 1, calculate advertiser identification information 1001 divided by It polymerize the remainder of the gained of number of queues 3, the first remainder values 2 is obtained, it is determined that polymerizeing queue with what first remainder values matched Serial No. 2, the Current ad daily record data 1 parsed is sent into polymerization queue 2；For advertisement log data 2, Following operation is performed respectively：According to the identification information 1002 of advertiser corresponding to fixed Current ad daily record data 2, calculate The remainder of the identification information 1002 of advertiser divided by the gained of polymerization number of queues 3, obtains the first remainder values 0, it is determined that with described the The polymerization queue Serial No. 3 that one remainder values match, the Current ad daily record data 2 parsed is sent to what is determined poly- Close in polymerization queue 3 corresponding to queue sequence number 3.
By the present embodiment, ensure that the advertisement log data of same advertiser can all be sent to same polymerization queue In, and the advertisement log data of same advertiser is handled in same polymerization association journey, avoid the daily record number of same advertiser The situation that the advertisement log data efficiency that same advertiser is handled during different polymerization queues according to being dispersed to reduces occurs, greatly Improve the data-handling efficiency of the advertisement log data of each same advertiser, further for can timely processing it is largely wide Accuse daily record data and provide important premise guarantee.
In a preferred embodiment, the advertisement log data in the polymerization queue in step 103 to each advertiser is parallel Polymerization, obtains the aggregated data of each advertiser's advertisement log data, including：It is each according to fixed each advertisement log data The identification information of self-corresponding advertiser and the identification information of advertising plan, it polymerize each advertisement meter of each advertiser parallel The advertisement log data drawn, obtains the aggregated data of each advertising plan of each advertiser；Wherein, the institute of any advertiser The aggregated data of advertiser's advertisement log data is made up of the aggregated data of advertising plan.
Wherein, the advertisement log data of each advertising plan for polymerizeing each advertiser parallel, obtain each wide After the aggregated data for accusing each main advertising plan, in addition to：According to each advertising plan of each advertiser Aggregated data, each advertisement consumption figures of each advertising plan of same advertisement master is merged, obtains each advertiser Each advertising plan polymerization consumption figures.
Wherein, the aggregated data of each advertising plan for each advertiser, performs following steps respectively： Current time is obtained, according to the polymerization consumption figures and current time of the Current ad plan of Current ad master, judges whether to meet Make a reservation for any pushing condition in multiple pushing conditions；When meeting any predetermined pushing condition, according to Current ad master's Identification information, the identification information of the advertiser divided by the remainder counted obtained by number of queues are calculated, obtains the second remainder Value；It is determined that the buffer queue sequence number of the current aggregated data to match with second remainder values；Will be described current wide The aggregated data for accusing main Current ad plan is sent into buffer queue corresponding to the buffer queue sequence number.
For example, in online advertisement settlement system A, the advertisement log data got includes advertisement log data 1 and advertisement Daily record data 2, wherein, advertisement log data 1 includes the advertising plan a that advertiser's identification information is 1001 daily record data and wide Plan b daily record data is accused, advertisement log data 2 includes the daily record data for the advertising plan c that advertiser's identification information is 1002； For the advertising plan a of the advertiser of identification information 1001 aggregated data, following steps are performed respectively：Obtain current time, According to the Current ad plan a of Current ad master 1001 polymerization consumption figures and current time, judge whether that satisfaction is predetermined multiple Any pushing condition in pushing condition；When meeting any predetermined pushing condition, according to the identification information of Current ad master 1001, the identification information 1001 of the advertiser divided by the remainder of the gained of statistics number of queues 4 are calculated, obtains the second remainder values 1； It is determined that the buffer queue Serial No. 1 of the current aggregated data to match with the second remainder values；By Current ad master 1001 Current ad plan a aggregated data send into buffer queue 1.For the advertising plan that advertiser's identification information is 1001 B daily record data and execution step of advertiser's identification information for 1002 advertising plan c daily record data and processing advertiser The step of identification information is 1001 advertising plan a daily record data is similar, will not be repeated here.
By the present embodiment, realize and further disperseed the advertisement log data of different advertisers, wherein, point Scattered limiting condition is the advertisement log data that each statistics association journey only handles an advertiser；When a certain statistics assists journey to data When the advertisement log data of advertiser is write in storehouse, it ensure that the advertisement log data of the advertiser is only possible to assist journey by the statistics Write-in, therefore, need not be locked during write into Databasce to the advertisement log data of the advertiser, so as to greatly Ground improves the write efficiency of database.
In a preferred embodiment, the aggregated data of the main advertisement log data of Current ad is judged described in step 104 is It is no to meet to make a reservation for any pushing condition in multiple pushing conditions, including：
Gather corresponding to the aggregated data for obtaining the polymerization bar number threshold value pre-seted and the Current ad plan of Current ad master Syzygy number, according to the polymerization bar number threshold value and the polymerizing factor, the first push threshold value is determined, and judge Current ad master's Whether the polymerization consumption figures of Current ad plan reaches the first push threshold value, if reaching, meets the first predetermined push Condition；Wherein, calculate and determine the first push threshold value, such as calculate polymerization bar number threshold value and the product of polymerizing factor, determine the One push threshold value.
The polymerization threshold value pre-seted is obtained, according to the polymerization threshold value and the polymerizing factor, determines second Threshold value is pushed, and judges whether the polymerization consumption figures of the Current ad plan of Current ad master reaches the second push threshold value, If reaching, meet the second predetermined pushing condition；Wherein, calculate and determine the second push threshold value, such as calculate polymerization consumption threshold Value and the product of polymerizing factor, determine the first push threshold value.
The first difference threshold pre-seted is obtained, it is determined that away from the current wide of the last push Current ad master of current time The first push time of the advertisement log data of plan is accused, current time is calculated and the very first time of the described first push time is poor Value, if the first time difference is more than first difference threshold, meets the 3rd predetermined pushing condition；
The second difference threshold for pre-seting is obtained, it is determined that away from current time the last time by the Current ad of Current ad master The advertisement log data of plan pushes to the second push time of buffer queue, calculates current time and the described second push time The second time difference, if second time difference is more than second difference threshold, meet predetermined 4th push article Part.
In a preferred embodiment, it polymerize corresponding to the aggregated data of the Current ad plan for obtaining Current ad master Coefficient, including：According to the aggregated data of the Current ad plan of Current ad master, the Current ad of the Current ad master is determined The buffer queue sequence number of the aggregated data of plan, and determine the element in buffer queue corresponding to the buffer queue sequence number Account for the capacity ratio of the buffer queue capacity；Based on predetermined computation rule, according to the capacity ratio, determine described current The polymerizing factor of the aggregated data of the Current ad plan of advertiser.
For example, in online advertisement settlement system A, it polymerize corresponding to the aggregated data of the Current ad plan of Current ad master Coefficient is calculated according to corresponding statistics queue " water level ", for adjusting polymerization threshold value, to increase polymerization granularity.It is so-called " water level " refers in queue that existing element accounts for the capacity ratio of capacity of queue, span for (0,1], with reference to figure 3.Specifically, According to the aggregated data of the Current ad plan of Current ad master, the polymerization of the Current ad plan of the Current ad master is determined The buffer queue sequence number of data, and determine that the element in buffer queue corresponding to the buffer queue sequence number accounts for the caching The capacity ratio of capacity of queue, such as it is 0.2；Based on predetermined computation rule, formula is calculated as below：
Wherein level is queue water level, and max is sampling factor (configurable), is defaulted as 200.Then, according to the capacity Ratio 0.5, it may be determined that the polymerizing factor of the aggregated data of the Current ad plan of the Current ad master is 5.
Wherein, polymerizing factor functional image is given tacit consent to as shown in figure 4, when " water level " of corresponding statistics queue is less than threshold value When (being less than 0.2), polymerizing factor 1, polymerization threshold value is exactly the default value in configuration.When " the water of corresponding statistics queue (it is more than 0.2) when position " is gradually increased, polymerizing factor also sharp increase, polymerization threshold value can also be doubled and redoubled with polymerizing factor, more The same advertising plan of same advertiser advertisement log data be aggregated into a data be sent to corresponding to statistics queue.Such as " water level " of statistics queue continues to increase corresponding to fruit, then this high polymerization state will maintain always, but the increasing of polymerizing factor Long rate can progressively decline, to prevent from causing clearing delay excessive because polymerizeing excessively.With the data being sent in queue progressively Reduce, " water level " of corresponding statistics queue can progressively decline, and polymerizing factor also can progressively reduce, after the speed of reduction is first slow It hurry up, to prevent the fluctuation back and forth of polymerizing factor.
By the present embodiment, according to queue " water level " dynamic calculation polymerizing factor, because computational methods can be carried out neatly Configuration so that time delay during processing mass advertising daily record data is controllable, further, when advertisement log data amount increases severely, is Can timely processing mass advertising daily record data provide important premise guarantee.
In a concrete application scene, online advertisement settlement system B system architecture diagram is as shown in figure 5, online advertisement knot Calculation system B includes settlement module, wherein, settlement module is pre-processed submodule, polymerize submodule, judged by access submodule Submodule, statistic submodule are formed：
1st, submodule and pretreatment submodule are accessed：Access submodule and pretreatment submodule includes M pretreatment jointly Journey is assisted, M pretreatment association journey is responsible for pulling each advertisement log data from Kafka, and each ad log is parsed, with Obtain the data such as the identification information of advertising plan, the identification information of advertiser, advertisement consumption figures；Access submodule and pretreatment Module assists Cheng Zucheng by multiple pretreatments, and the journey concurrent working of each pretreatment association, independently of each other, quantity can match somebody with somebody, wherein, pretreatment Quantity is generally less than or equal to Kafka partition quantity；Each advertisement log data after parsing can be calculated according to Hash The identification information of advertiser after method processing, it is sent in the polymerization queue in different polymerization submodules.
2nd, it polymerize submodule and judging submodule：Polymerize submodule and judging submodule includes O polymerization queue and O jointly Individual polymerization association journey, wherein, polymerization queue corresponds with polymerizeing association's journey, and O polymerization queue is responsible for receiving and caches pretreatment The advertisement log data and be responsible for merging the advertisement log data of same advertiser that module is sent, to reduce total data Amount, so as to mitigate the pressure to statistic submodule；O polymerization association journey concurrent working, independently of each other, quantity can match somebody with somebody, wherein, polymerization The quantity of association's journey is typically greater than or equal to access submodule and pre-processes the quantity of the pretreatment association journey included jointly in submodule.
3rd, statistic submodule：Including P statistics queue and P statistics association journey, wherein, statistics queue and statistics association Cheng Yiyi Corresponding, P statistics queue, which is responsible for receiving and cache, polymerize advertisement log data that submodule sends and responsible by each advertiser Advertisement consumption Value Data carry out statistics plus and and by the advertisement log data after statistics by advertiser's dimension, advertising plan dimension Send to corresponding statistics association journey, assist journey that the advertisement log data renewal after statistics is arrived into database by P statistics；Statistics Module assists journey to form by multiple statistics, and each to assist journey concurrent working, independently of each other, quantity can match somebody with somebody, wherein, the quantity one of statistics association journey As be more than or equal to polymerization submodule polymerization association journey quantity.
The embodiments of the invention provide a kind of system of advertisement clearing, it is possible to achieve the embodiment of the method for above-mentioned offer, tool Body function realizes the explanation referred in embodiment of the method, will not be repeated here.
It should be understood that the particular order or level of the step of during disclosed are the examples of illustrative methods.Based on setting Count preference, it should be appreciated that during the step of particular order or level can be in the feelings for the protection domain for not departing from the disclosure Rearranged under condition.Appended claim to a method gives the key element of various steps with exemplary order, and not It is to be limited to described particular order or level.
In above-mentioned detailed description, various features combine in single embodiment together, to simplify the disclosure.No This open method should be construed to reflect such intention, i.e. the embodiment of theme claimed needs to compare The more features of feature clearly stated in each claim.On the contrary, as appended claims is reflected Like that, the present invention is in the state fewer than whole features of disclosed single embodiment.Therefore, appended claims It is hereby expressly incorporated into detailed description, wherein each claim is alone as the single preferred embodiment of the present invention.
To enable any technical staff in the art to realize or using the present invention, disclosed embodiment being entered above Description is gone.To those skilled in the art；The various modification modes of these embodiments will be apparent from, and this The General Principle of text definition can also be applied to other embodiments on the basis of the spirit and scope of the disclosure is not departed from. Therefore, the disclosure is not limited to embodiments set forth herein, but most wide with principle disclosed in the present application and novel features Scope is consistent.
Described above includes the citing of one or more embodiments.Certainly, in order to above-described embodiment is described and description portion The all possible combination of part or method is impossible, but it will be appreciated by one of ordinary skill in the art that each implementation Example can do further combinations and permutations.Therefore, embodiment described herein is intended to fall into appended claims Protection domain in all such changes, modifications and variations.In addition, with regard to the term used in specification or claims "comprising", the mode that covers of the word are similar to term " comprising ", just as " including " solved in the claims as link word As releasing.In addition, the use of any one term "or" in the specification of claims is to represent " non-exclusionism Or ".
Those skilled in the art will also be appreciated that the various illustrative components, blocks that the embodiment of the present invention is listed (illustrativelogical block), unit, and step can pass through the combination of electronic hardware, computer software, or both Realized.To clearly show that the replaceability of hardware and software (interchangeability), above-mentioned is various illustrative Part (illustrativecomponents), unit and step universally describe their function.Such function It is that specific application and the design requirement of whole system are depended on to realize by hardware or software.Those skilled in the art can For every kind of specific application, various methods can be used to realize described function, but this realization is understood not to surpass Go out the scope of protection of the embodiment of the present invention.
Various illustrative logical blocks described in the embodiment of the present invention, or unit can by general processor, Digital signal processor, application specific integrated circuit (ASIC), field programmable gate array or other programmable logic devices, discrete gate Or the design of transistor logic, discrete hardware components, or any of the above described combination is come the function described by realizing or operate.General place It can be microprocessor to manage device, and alternatively, the general processor can also be any traditional processor, controller, microcontroller Device or state machine.Processor can also be realized by the combination of computing device, such as digital signal processor and microprocessor, Multi-microprocessor, one or more microprocessors combine a Digital Signal Processor Core, or any other like configuration To realize.
The step of method or algorithm described in the embodiment of the present invention can be directly embedded into hardware, computing device it is soft Part module or the combination of both.Software module can be stored in RAM memory, flash memory, ROM memory, EPROM storages Other any form of storaging mediums in device, eeprom memory, register, hard disk, moveable magnetic disc, CD-ROM or this area In.Exemplarily, storaging medium can be connected with processor, to allow processor to read information from storaging medium, and Write information can be deposited to storaging medium.Alternatively, storaging medium can also be integrated into processor.Processor and storaging medium can To be arranged in ASIC, ASIC can be arranged in user terminal.Alternatively, processor and storaging medium can also be arranged at use In different parts in the terminal of family.
In one or more exemplary designs, above-mentioned function described by the embodiment of the present invention can be in hardware, soft Part, firmware or any combination of this three are realized.If realized in software, these functions can store and computer-readable On medium, or with one or more instruction or code form be transmitted on the medium of computer-readable.Computer readable medium includes electricity Brain storaging medium and it is easy to so that allowing computer program to be transferred to other local telecommunication medias from a place.Storaging medium can be with It is that any general or special computer can be with the useable medium of access.For example, such computer readable media can include but It is not limited to RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or other What can be used for carrying or store with instruct or data structure and it is other can be by general or special computer or general or specially treated The medium of the program code of device reading form.In addition, any connection can be properly termed computer readable medium, example Such as, if software is to pass through a coaxial cable, fiber optic cables, double from a web-site, server or other remote resources Twisted wire, Digital Subscriber Line (DSL) or with defined in being also contained in of the wireless way for transmitting such as infrared, wireless and microwave In computer readable medium.Described disk (disk) and disk (disc) include Zip disk, radium-shine disk, CD, DVD, floppy disk And Blu-ray Disc, disk is generally with magnetic duplication data, and disk generally carries out optical reproduction data with laser.Combinations of the above It can also be included in computer readable medium.
Above-described embodiment, the purpose of the present invention, technical scheme and beneficial effect are carried out further Describe in detail, should be understood that the embodiment that the foregoing is only the present invention, be not intended to limit the present invention Protection domain, within the spirit and principles of the invention, any modification, equivalent substitution and improvements done etc., all should include Within protection scope of the present invention.

Claims (12)

  1. A kind of 1. method of advertisement clearing, it is characterised in that including：
    Multiple advertisement log datas are obtained, and determine the identification information of each self-corresponding advertiser of each advertisement log data；
    According to the identification information of fixed each each self-corresponding advertiser of advertisement log data, by each ad log Data are respectively sent in the polymerization queue of each self-corresponding advertiser；
    The advertisement log data in polymerization queue to each advertiser polymerize parallel, obtains each advertiser's advertisement log data Aggregated data；
    For the aggregated data of each advertiser's advertisement log data, following steps are performed respectively：According to Current ad, master is wide Accuse daily record data aggregated data and current time, judge the main advertisement log data of Current ad aggregated data whether meet it is pre- Any pushing condition in fixed multiple pushing conditions, when meeting any predetermined pushing condition, by Current ad main advertisement day The aggregated data of will data is sent into buffer queue corresponding to Current ad master；
    In each buffer queue, each aggregated data is sent into statistics association corresponding with current cache queue journey, and Journey is assisted by each aggregated data write into Databasce by the statistics.
  2. 2. according to the method for claim 1, it is characterised in that also include：
    The pretreatment association number of passes amount pre-seted is obtained, the multiple pretreatments for assisting number of passes amount to match with the pretreatment is created and assists Journey；
    Obtain the polymerization number of queues that pre-sets, create with it is described polymerize multiple polymerization queues that number of queues matches and with The multiple polymerization queue multiple polymerization associations journey, and determine the unique corresponding polymerization queue of each polymerization queue correspondingly Sequence number；
    Obtain the statistics number of queues that pre-sets, create multiple statistics queues for matching with the statistics number of queues and with The multiple one-to-one multiple statistics associations journey of statistics queue, and determine the corresponding unique statistics queue of each statistics queue Sequence number；
    Wherein, it is described to obtain multiple advertisement log datas, and determine the mark of each self-corresponding advertiser of each advertisement log data Know information, including：
    Journey is assisted by the multiple pretreatment, each subregion acquisition subscribed to from distributed post in message system Kafka is multiple Advertisement log data, and each advertisement log data being analyzed and acquired by respectively；
    According to each advertisement log data parsed, determine that the mark of each self-corresponding advertiser of each advertisement log data is believed Breath；
    Wherein, the mark that each advertisement log data parsed includes advertising plan corresponding to each advertisement log data is believed Breath, the identification information of advertiser and advertisement consumption figures；
    Wherein, each advertisement log data that the basis parses, each self-corresponding advertisement of each advertisement log data is determined After main identification information, in addition to：
    According to the identification information of fixed each each self-corresponding advertiser of advertisement log data, handled and calculated by tentation data Method, data processing is carried out to the identification information of each advertiser in each subregion in Kafka.
  3. 3. according to the method for claim 2, it is characterised in that it is described according to fixed each advertisement log data each The identification information of corresponding advertiser, each advertisement log data is respectively sent to the polymerization of each self-corresponding advertiser In queue, including：
    For each advertisement log data, following operation is performed respectively：
    According to the identification information of advertiser corresponding to fixed Current ad daily record data, the mark for calculating the advertiser is believed Remainder obtained by breath divided by the polymerization number of queues, obtains the first remainder values；
    It is determined that it polymerize queue sequence number with what first remainder values matched；
    Current ad daily record data is sent into polymerization queue corresponding to the polymerization queue sequence number determined.
  4. 4. according to the method for claim 2, it is characterised in that the advertisement day in the polymerization queue to each advertiser Will data parallel polymerize, and obtains the aggregated data of each advertiser's advertisement log data, including：
    Believed according to the mark of the identification information of fixed each each self-corresponding advertiser of advertisement log data and advertising plan Breath, it polymerize the advertisement log data of each advertising plan of each advertiser parallel, each for obtaining each advertiser is wide Accuse the aggregated data of plan；Wherein, the aggregated data of all advertising plans of any advertiser forms advertiser's ad log The aggregated data of data；
    Wherein, the advertisement log data of each advertising plan for polymerizeing each advertiser parallel, obtains each advertiser Each advertising plan aggregated data after, in addition to：
    According to the aggregated data of each advertising plan of each advertiser, by each advertising plan of same advertisement master Each advertisement consumption figures merge, obtain the polymerization consumption figures of each advertising plan of each advertiser；
    Wherein, for each advertiser each advertising plan aggregated data, perform following steps respectively：
    Current time is obtained, according to the polymerization consumption figures and current time of the Current ad plan of Current ad master, is judged whether Meet to make a reservation for any pushing condition in multiple pushing conditions；
    When meeting any predetermined pushing condition, according to the identification information of Current ad master, the mark of the advertiser is calculated Remainder obtained by information divided by the statistics number of queues, obtains the second remainder values；
    It is determined that the buffer queue sequence number of the current aggregated data to match with second remainder values；
    The aggregated data of the Current ad plan of the Current ad master is sent to corresponding to the buffer queue sequence number and delayed Deposit in queue.
  5. 5. according to the method for claim 4, it is characterised in that the polymerization for judging the main advertisement log data of Current ad Whether data meet any pushing condition in predetermined multiple pushing conditions, including：
    Obtain polymerization system corresponding to the aggregated data of the polymerization bar number threshold value pre-seted and the Current ad plan of Current ad master Number, according to the polymerization bar number threshold value and the polymerizing factor, the first push threshold value is determined, and judge that Current ad master's is current Whether the polymerization consumption figures of advertising plan reaches the first push threshold value, if reaching, meets the first predetermined pushing condition；
    The polymerization threshold value pre-seted is obtained, according to the polymerization threshold value and the polymerizing factor, determines the second push Threshold value, and judge whether the polymerization consumption figures of the Current ad plan of Current ad master reaches the second push threshold value, if reaching Arrive, then meet the second predetermined pushing condition；
    The first difference threshold pre-seted is obtained, it is determined that the Current ad meter away from the last push Current ad master of current time The first push time of the advertisement log data drawn, current time and the first time difference of the described first push time are calculated, If the first time difference is more than first difference threshold, meet the 3rd predetermined pushing condition；
    The second difference threshold for pre-seting is obtained, it is determined that away from current time the last time by the Current ad plan of Current ad master Advertisement log data push to the second push time of buffer queue, calculate the of current time and the described second push time Two time differences, if second time difference is more than second difference threshold, meet the 4th predetermined pushing condition.
  6. 6. according to the method for claim 5, it is characterised in that the Current ad plan for obtaining Current ad master gathers Polymerizing factor corresponding to data is closed, including：
    According to the aggregated data of the Current ad plan of Current ad master, the Current ad plan of the Current ad master is determined The buffer queue sequence number of aggregated data, and it is described to determine that the element in buffer queue corresponding to the buffer queue sequence number accounts for The capacity ratio of buffer queue capacity；
    Based on predetermined computation rule, according to the capacity ratio, determine the Current ad master Current ad plan it is poly- Close the polymerizing factor of data.
  7. A kind of 7. system of advertisement clearing, it is characterised in that including：
    Submodule is accessed, for obtaining multiple advertisement log datas, and determines each self-corresponding advertisement of each advertisement log data Main identification information；
    Submodule is pre-processed, for the identification information according to fixed each each self-corresponding advertiser of advertisement log data, Each advertisement log data is respectively sent in the polymerization queue of each self-corresponding advertiser；
    It polymerize submodule, polymerize parallel for the advertisement log data in the polymerization queue to each advertiser, obtains each wide Accuse the aggregated data of main advertisement log data；
    Judging submodule, for the aggregated data for each advertiser's advertisement log data, following steps are performed respectively：Root According to the aggregated data and current time of the main advertisement log data of Current ad, the polymerization of the main advertisement log data of Current ad is judged Whether data meet to make a reservation for any pushing condition in multiple pushing conditions, ought when meeting any predetermined pushing condition The aggregated data of preceding advertiser's advertisement log data is sent into buffer queue corresponding to Current ad master；
    Statistic submodule, in each buffer queue, each aggregated data to be sent to corresponding with current cache queue Statistics association journey in, and by the statistics assist journey by each aggregated data write into Databasce.
  8. 8. system according to claim 7, it is characterised in that also include：
    First obtains and creates submodule, assists number of passes amount for obtaining the pretreatment pre-seted, creates and assist journey with the pretreatment Multiple pretreatments association journey that quantity matches；
    Second obtains and creates submodule, for obtaining the polymerization number of queues pre-seted, creates and polymerize number of queues with described Multiple polymerization queues for matching and it polymerize the one-to-one multiple polymerization associations journey of queue with the multiple, and determines each It polymerize the unique corresponding polymerization queue sequence number of queue；
    3rd obtains and creates submodule, for obtaining the statistics number of queues pre-seted, creates and the statistics number of queues Multiple statistics queues for matching and with the one-to-one multiple statistics associations journey of the multiple statistics queue, and determine each Count the corresponding unique statistics queue sequence number of queue；
    Wherein, the access submodule, including：
    Acquisition and resolution unit, for assisting journey by the multiple pretreatment, subscribed to from distributed post in message system Kafka Each subregion obtain multiple advertisement log datas, and each advertisement log data being analyzed and acquired by respectively；
    First determining unit, for according to each advertisement log data parsed, determining that each advertisement log data is each right The identification information of the advertiser answered；
    Wherein, the mark that each advertisement log data parsed includes advertising plan corresponding to each advertisement log data is believed Breath, the identification information of advertiser and advertisement consumption figures；
    Wherein, first determining unit, it is additionally operable to according to fixed each each self-corresponding advertiser of advertisement log data Identification information, by tentation data Processing Algorithm, the mark of each advertiser is believed in each subregion in Kafka Breath carries out data processing.
  9. 9. system according to claim 8, it is characterised in that the pretreatment submodule, be specifically used for
    For each advertisement log data, following operation is performed respectively：
    According to the identification information of advertiser corresponding to fixed Current ad daily record data, the mark for calculating the advertiser is believed Remainder obtained by breath divided by the polymerization number of queues, obtains the first remainder values；
    It is determined that it polymerize queue sequence number with what first remainder values matched；
    Current ad daily record data is sent into polymerization queue corresponding to the polymerization queue sequence number determined.
  10. 10. system according to claim 8, it is characterised in that the polymerization submodule, specifically for according to fixed The identification information of each each self-corresponding advertiser of advertisement log data and the identification information of advertising plan, it polymerize parallel each wide The advertisement log data of each main advertising plan is accused, obtains the aggregated data of each advertising plan of each advertiser； Wherein, the aggregated data of all advertising plans of any advertiser forms the aggregated data of advertiser's advertisement log data；
    The polymerization submodule, the aggregated data of each advertising plan according to each advertiser is additionally operable to, will be identical Each advertisement consumption figures of each advertising plan of advertiser merges, and obtains each advertising plan of each advertiser It polymerize consumption figures；
    The judging submodule, specifically for the aggregated data of each advertising plan for each advertiser, hold respectively Row following steps：
    Current time is obtained, according to the polymerization consumption figures and current time of the Current ad plan of Current ad master, is judged whether Meet to make a reservation for any pushing condition in multiple pushing conditions；
    When meeting any predetermined pushing condition, according to the identification information of Current ad master, the mark of the advertiser is calculated Remainder obtained by information divided by the statistics number of queues, obtains the second remainder values；
    It is determined that the buffer queue sequence number of the current aggregated data to match with second remainder values；
    The aggregated data of the Current ad plan of the Current ad master is sent to corresponding to the buffer queue sequence number and delayed Deposit in queue.
  11. 11. system according to claim 10, it is characterised in that the judging submodule, including：
    First judging unit, the polymerization of the Current ad plan for obtaining the polymerization bar number threshold value pre-seted and Current ad master Polymerizing factor corresponding to data, according to the polymerization bar number threshold value and the polymerizing factor, the first push threshold value is determined, and judge Whether the polymerization consumption figures of the Current ad plan of Current ad master reaches the first push threshold value, if reaching, meets pre- The first fixed pushing condition；
    Second judging unit, for obtaining the polymerization threshold value pre-seted, it is polymerize according to the polymerization threshold value with described Coefficient, the second push threshold value is determined, and judge whether the polymerization consumption figures of the Current ad plan of Current ad master reaches described Second push threshold value, if reaching, meets the second predetermined pushing condition；
    3rd judging unit, for obtaining the first difference threshold pre-seted, it is determined that current away from the last push of current time The first push time of the advertisement log data of the Current ad plan of advertiser, when calculating current time with the described first push Between first time difference, if the first time difference is more than first difference threshold, meet predetermined 3rd push Condition；
    4th judging unit, for obtaining the second difference threshold pre-seted, it is determined that will be currently wide away from current time the last time The advertisement log data for accusing main Current ad plan pushes to the second push time of buffer queue, calculates current time and institute The second time difference of the second push time is stated, if second time difference is more than second difference threshold, is met pre- The 4th fixed pushing condition.
  12. 12. system according to claim 11, it is characterised in that the judging submodule, in addition to：
    Second determining unit, for the aggregated data of the Current ad plan according to Current ad master, determine the Current ad The buffer queue sequence number of the aggregated data of main Current ad plan, and determine to cache corresponding to the buffer queue sequence number Element in queue accounts for the capacity ratio of the buffer queue capacity；
    Computing unit, for based on predetermined computation rule, according to the capacity ratio, determining that the Current ad master's is current The polymerizing factor of the aggregated data of advertising plan.
CN201711023402.3A 2017-10-27 2017-10-27 Advertisement settlement method and system Active CN107886352B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711023402.3A CN107886352B (en) 2017-10-27 2017-10-27 Advertisement settlement method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711023402.3A CN107886352B (en) 2017-10-27 2017-10-27 Advertisement settlement method and system

Publications (2)

Publication Number Publication Date
CN107886352A true CN107886352A (en) 2018-04-06
CN107886352B CN107886352B (en) 2021-11-19

Family

ID=61782718

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711023402.3A Active CN107886352B (en) 2017-10-27 2017-10-27 Advertisement settlement method and system

Country Status (1)

Country Link
CN (1) CN107886352B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109919674A (en) * 2019-03-04 2019-06-21 厦门美图之家科技有限公司 Advertisement settlement method, device and equipment
CN112217893A (en) * 2020-10-12 2021-01-12 广州欢网科技有限责任公司 Frequency and quantity control method, device and equipment for advertisement delivery and advertisement delivery system
CN112241899A (en) * 2019-07-19 2021-01-19 上海哔哩哔哩科技有限公司 Advertisement charging method, device and system and readable storage medium
CN112433886A (en) * 2020-11-24 2021-03-02 厦门美图之家科技有限公司 Data processing method and device, server and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110040811A1 (en) * 2009-08-17 2011-02-17 International Business Machines Corporation Distributed file system logging
CN105978968A (en) * 2016-05-11 2016-09-28 山东合天智汇信息技术有限公司 Real-time transmission processing method, server and system of mass data
US9626328B1 (en) * 2013-06-19 2017-04-18 EMC IP Holding Company LLC Method and system for on-demand aggregated logging for distributed systems
CN107133818A (en) * 2017-04-25 2017-09-05 微梦创科网络科技（中国）有限公司 The settlement method and settlement system of online advertisement in a kind of internet

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110040811A1 (en) * 2009-08-17 2011-02-17 International Business Machines Corporation Distributed file system logging
US9626328B1 (en) * 2013-06-19 2017-04-18 EMC IP Holding Company LLC Method and system for on-demand aggregated logging for distributed systems
CN105978968A (en) * 2016-05-11 2016-09-28 山东合天智汇信息技术有限公司 Real-time transmission processing method, server and system of mass data
CN107133818A (en) * 2017-04-25 2017-09-05 微梦创科网络科技（中国）有限公司 The settlement method and settlement system of online advertisement in a kind of internet

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109919674A (en) * 2019-03-04 2019-06-21 厦门美图之家科技有限公司 Advertisement settlement method, device and equipment
CN109919674B (en) * 2019-03-04 2021-06-01 厦门美图之家科技有限公司 Advertisement settlement method, device and equipment
CN112241899A (en) * 2019-07-19 2021-01-19 上海哔哩哔哩科技有限公司 Advertisement charging method, device and system and readable storage medium
CN112217893A (en) * 2020-10-12 2021-01-12 广州欢网科技有限责任公司 Frequency and quantity control method, device and equipment for advertisement delivery and advertisement delivery system
CN112433886A (en) * 2020-11-24 2021-03-02 厦门美图之家科技有限公司 Data processing method and device, server and storage medium

Also Published As

Publication number Publication date
CN107886352B (en) 2021-11-19

Similar Documents

Publication Publication Date Title
CN107886352A (en) A kind of method and system of advertisement clearing
CN110390596B (en) Financial futures trading system
CN106020715B (en) Storage pool capacity management
JP6045505B2 (en) Method and apparatus for managing orders in a financial market
CN105264509B (en) Adaptive interruption joint in converged network
CN106030562A (en) Packet shaping in a network processor
CN106649670A (en) Streaming computing-based data monitoring method and apparatus
CN1795463A (en) Customer revenue prediction method and system
CN108681964A (en) A kind of business settlement system and settlement of transactions control method
CN107979477A (en) A kind of method and system of business monitoring
CN105577808A (en) Supervision reporting method and system
CN110490392A (en) Site bank note distribution method and device
CN102902775A (en) Internet real-time computing method and internet real-time computing system
CN107133818A (en) The settlement method and settlement system of online advertisement in a kind of internet
WO2019196254A1 (en) Electronic resource packet processing method and apparatus, terminal device and medium
CN109729023A (en) Transmission of transaction data method and device thereof
CN107247763A (en) Business datum statistical method, device, system, storage medium and electronic equipment
CN107809480A (en) One kind transaction commutation system
CN107292601A (en) A kind of method and system of withdrawing deposit based on bank's API server
KR20160077021A (en) Deposit service method based on the balance of a bankbook
CN111629216B (en) VOD service cache replacement method based on random forest algorithm under edge network environment
CN105554069A (en) Big data processing distributed cache system and method thereof
CN110213239B (en) Suspicious transaction message generation method and device and server
CN104283934A (en) WEB service pushing method and device based on reliability prediction and server
CN106709029A (en) File hierarchical processing method and processing system based on Hadoop and MySQL

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant