NL2031800B1 - Method for managing an advertisement campaign - Google Patents

Abstract

A computer-implemented method is disclosed for managing an advertisement campaign for a job vacancy. The method comprises receiving advertisement campaign information. The 5 advertisement campaign information indicates one or more properties of the job vacancy as well as one or more properties of the advertisement campaign. The method also comprises, based on the advertisement campaign information, automatically selecting a first set of one or more computer systems out of a plurality of computer systems. The method also comprises automatically transmitting, to each computer system in the first set, an instruction to cause display of advertisements relating to 10 the job vacancy on one or more user devices. Herein, the instruction comprising information required by the computer system in question for causing display of advertisements relating to the job vacancy on one or more user devices.

Description

NL33444 —id/av

Method for managing an advertisement campaign

FIELD OF THE INVENTION

This disclosure relates to a computer-implemented method is disclosed for managing an advertisement campaign for a job vacancy, in particular to such method wherein a set of computer systems is automatically selected based on advertisement campaign information. This disclosure further relates to a data processing system, a computer program and a computer-readable storage medium for performing such method.

BACKGROUND it is not trivial to distribute advertisements for a job vacancy efficiently, i.e. in such manner that the advertisements are shown to the appropriate audience while at the same time minimize the costs per interaction with such advertisement such as the “costs per click”. Complicating factor is the vast number of channels that can be used to cause display of advertisements for a job vacancy, each channel having their own characteristics in terms of costs and/or bidding mechanism and/or audience, et cetera. In this complex environment, distributing advertisements for job vacancies may very well result in useless exchange of data, herewith consuming the limited bandwidth of the physical communication channel that is used. To illustrate, advertisements for job vacancies may be distributed via a channel that is not suitable for the job vacancy in question, because the channel is for example too expensive, or because users turn out not to interact with the advertisement if presented via the channel. This may result in cancellation of further advertisements via the selected channel and try another channel that is better suited to the constraints of the advertisement campaign. Such trial-and- error approach of finding the appropriate channel mix and associated budget allocation for an advertisement campaign may thus lead to additional messages that are exchanged back and forth.

Herewith, bandwidth is unnecessarily consumed.

Hence, there is a need in the art for methods for managing advertisement campaigns for job vacancies that enable to effectively distribute advertisements while keeping the messaging to a minimum.

SUMMARY

To that end, a computer-implemented method is disclosed for managing an advertisement campaign for a job vacancy. The method comprises receiving advertisement campaign information.

The advertisement campaign information indicates one or more properties of the job vacancy as well as one or more properties of the advertisement campaign. The method also comprises, based on the advertisement campaign information, automatically selecting a first set of one or more computer systems out of a plurality of computer systems. In an example each computer system is associated with a channel via which advertisements can be distributed online. In any case, each computer system out of the plurality of computer systems is configured to cause display of advertisements on one or more user devices. A computer system may be configured to cause display of advertisements on one or more user device in that it is a server from which client computers download the advertisements if the client computers are downloading a website, for example. The advertisement may then be shown onthe user device together with, e.g. on, the website. The method also comprises automatically transmitting, to each computer system in the first set, an instruction to cause display of advertisements relating to the job vacancy on one or more user devices. Herein, the instruction comprising information required by the computer system in question for causing display of advertisements relating to the job vacancy on one or more user devices. The information required for causing display of the advertisement may comprises a job titie and/or a job description, for example.

The automatic selection of computer systems enable to efficiently manage the advertisement campaign. The automatic selection allows to select the appropriate set of computer systems very fast and accurately, i.e. in a way that complies with a set of given rules, such as budget constraints.

In an embodiment, the one or more properties of the job vacancy comprises -a geographical location or area where the job is to be performed, and/or -a job description, and/or -a job classification, and/or -a salary indication, and/or -a name of the employer.

This embodiment even further improves the selection of computer systems to be used for distributing the advertisements.

In an embodiment, the one or more properties of the advertisement campaign comprises -a total available budget for the advertisement campaign, and/or -a duration of the advertisement campaign.

The one or more properties of the advertisement campaign may be understood to define a set of constraints that have to be met. Typically, an advertisement campaign has a certain budget that should not be exceeded. However, a duration of the advertisement campaign is also defined.

Preferably, the first set of computer systems is selected such that the total available budget of the advertisement campaign is spent gradually, such that advertisements are shown throughout the defined duration of the advertisement campaign.

In an embodiment, the method further comprises receiving, from each computer system in the first set, user interaction information comprising an indication of a number of user interactions with advertisements of which the display was caused by the computer system in question.

Preferably, the user interaction information also comprises an indication of how many advertisements the computer system in question has shown. As referred to herein, a user interaction with an advertisement may for example be a user clicking on an advertisement and/or a user filling in one or more text fields of the advertisement.

This embodiment allows to manage the advertisement campaign based on a number of user interactions.

In an embodiment, the user interaction information indicates an effectiveness of advertisements the display of which is caused by the computer system in question. Preferably, the effectiveness is based on a ratio between a number of user interactions with an advertisement and a number of displayed advertisements. The latter ratio may also be referred to as the conversion ratio of the advertisement.

This embodiment, enables to manage the advertisement campaign effectively in that it allows to only select the channels with high conversion ratios.

In an embodiment, the method comprises receiving, from each computer system in the first set, budget spent information comprising an indication of an amount of budget that was spent as a result by the computer system in question causing display of advertisements on one or more user devices.

Typically, some amount of money has to be paid for each advertisement that is shown. Hence, this embodiment allows to effectively manage the advertisement campaign on budget spent. This embodiment for example allows to ensure that the budget is spent gradually over the duration of the advertisement campaign. Too expensive channels, for example, are preferably taken out of the channel mix.

In an embodiment, the method comprises, based on the received user interaction information and/or based on the received budget spent information, automatically selecting a second set of one or more computer systems out of the plurality of computer systems, the second set being different from the first set. This embodiment also comprises automatically transmitting, to each computer system in the second set, an instruction to cause display of advertisements relating to the job vacancy on one or more user devices.

This embodiment thus allows to adapt the selected channel mix based on the metrics of the advertisement campaign.

In an embodiment, the advertisement campaign information indicates a total budget for the advertisement campaign. This embodiment also comprises, based on the total budget, determining, for each computer system in the first and/or second set, a budget and/or a bid height specifically for the computer system in question. The instruction transmitted to the computer system in question comprises a budget indication indicating the determined budget and/or bid height.

The budget for a computer system may be understood to be the budget that can be spent on advertisements the display of which is caused by the computer system in question. Additionally or alternatively, the budget may indicate a budget that can be spent per unit of time, such as per day or per week.

Further, some computer systems, e.g. some channels, require that bids are placed.

Advertisement spots are then granted to the advertisement campaign with the highest bid.

In an embodiment, the advertisement campaign information comprises a job description. In such embodiment, the method comprises -based on the job description, determining a job classification, and

-based on the job classification, determining the first set of computer system and/or determining, for each computer system in the first and/or second set, a budget and/or a bid height specifically for the computer system in question.

The job classification may be determined using text analysis algorithms known in the art.

In an embodiment, the method comprises, based the received user interaction information and/or based on the received budget spent information, automatically transmitting an updated instruction to at least one computer system in the first and/or second set, the updated instruction comprising an updated budget indication indicating a new budget and/or a new bid height.

This embodiment allows to adapt the amount of budget that is spent on the respective channels.

In an embodiment, selecting the first set of computer systems comprises performing a machine learning algorithm. The machine learning algorithm comprises -constructing a model based on training data, wherein the training data associates, to each advertisement campaign out of a plurality of advertisement campaigns for respective job vacancies, an effectiveness score, wherein the effectiveness score indicates an effectiveness of the advertisement campaign in question, and wherein the training data indicates, for each advertisement campaign out of the plurality of advertisement campaigns, the one or more properties of the job vacancy to which the advertisement campaign in question relates and/or the one or more properties of the advertisement campaign in question and/or the set of computer systems to which respective instructions were sent for the advertisement campaign in question for displaying advertisements on one or more user devices, and -using the constructed model to select the first set of one or more computer systems.

This embodiment provides a convenient method for selecting an appropriate set of channels for any given advertisement campaign. Herein, an appropriate set may be understood as a set that results in a high effectiveness of the advertisement campaign.

Herein, effectiveness of an advertisement campaign may for example refer to the degree to which one or more constraints of the advertisement campaign are met.

In an embodiment, the training data indicates, for each computer system as indicated by the training data, a budget and/or bid height included in the instruction sent to the computer system in question, and using the constructed model, determining, for each computer system out of the selected first set of one or more computer system, the budget and/or bid height specifically for the computer system in question, wherein the instruction sent to each computer system out of the first set of computer systems comprises a budget indication indicating the budget and/or bid height determined for the computer system in question.

This embodiment provides for a convenient method for determining appropriate budgets and/or bid heights for channels, e.g. budgets and/or bid heights that lead to highly effective advertisement campaigns.

One aspect of this disclosure relates to a data processing system comprising a processor that is configured to perform any of the methods described herein.

One aspect of this disclosure relates to a computer program comprising instructions which, when executed by a computer, cause the computer to perform any of the methods described herein. 5 One aspect of this disclosure relates to a non-transitory computer-readable storage medium having stored thereon any of the computer programs described herein.

One aspect of this disclosure relates to a computer comprising a a computer readable storage medium having computer readable program code embodied therewith, and a processor, preferably a microprocessor, coupled to the computer readable storage medium, wherein responsive to executing the computer readable program code, the processor is configured to perform any of the methods described herein.

One aspect of this disclosure relates to computer program or suite of computer programs comprising at least one software code portion or a computer program product storing at least one software code portion, the software code portion, when run on a computer system, being configured for executing any of the methods described herein. One aspect of this disclosure relates to non- transitory computer-readable storage medium storing at least one software code portion, the software code portion, when executed or processed by a computer, is configured to perform any of the methods described herein.

As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, a method or a computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” "module" or "system."

Functions described in this disclosure may be implemented as an algorithm executed by a processor/microprocessor of a computer. Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied, e.g., stored, thereon.

Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a computer readable storage medium may include, but are not limited to, the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of the present invention, a computer readable storage medium may be any tangible medium that can contain, or store, a program for use by or in connection with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber, cable, RF, etc, or any suitable combination of the foregoing. Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java(TM), Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).

Aspects of the present invention are described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor, in particular a microprocessor or a central processing unit (CPU), of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer, other programmable data processing apparatus, or other devices create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Moreover, a computer program for carrying out the methods described herein, as well as a non- transitory computer readable storage-medium storing the computer program are provided. A computer program may, for example, be downloaded (updated) to the existing data processing systems or be stored upon manufacturing of these systems.

Elements and aspects discussed for or in relation with a particular embodiment may be suitably combined with elements and aspects of other embodiments, unless explicitly stated otherwise.

Embodiments of the present invention will be further illustrated with reference to the attached drawings, which schematically will show embodiments according to the invention. It will be understood that the present invention is not in any way restricted to these specific embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

Aspects of the invention will be explained in greater detail by reference to exemplary embodiments shown in the drawings, in which:

FIG. 1 illustrates a system according to an embodiment;

FIG. 2 illustrates exchange of information among the computer systems that are shown in figure 1, according to an embodiment;

FIG. 3 illustrates different modules within the advertisement campaign management system, according to an embodiment;

FIG. 4 is a flow chart illustrating a method according to an embodiment;

FIG. 5 is a flow chart illustrating a method according to an embodiment;

FIG. 6 depicts a block diagram illustrating a data processing system according to an embodiment.

DETAILED DESCRIPTION OF THE DRAWINGS

In the figures, identical reference numbers indicate identical or similar elements.

Figure 1 illustrates a system according to an embodiment. Herein, a computer system 2, which may be referred to as a web shop computer system, is connected via a network, such as a packet switched network, such as the internet, to a computer system 4 that is configured to manage an advertisement campaign. The clouds in figure 1 indicate such networks. Computer system 4 may also be referred to as the advertisement campaign management system 4, or performance steering agent (PSA). In turn, the advertisement campaign management system 4 is connected to a plurality of computer systems 6 each of which is configured to cause display of advertisements on one or more user devices 8. Each of the computer systems 6 may comprise one or more servers having stored thereon information for rendering an advertisements on one or more displays of user devices 8. The user devices may be any type of user devices, such as desktop computer, laptops, smart phones, tablet computers, et cetera.

As referred to herein, one computer system being connected to another computer system may be understood as that the computer systems can send messages to each other so that they can communicate with each other.

Further, each of the web shop computer system 2, the advertisement campaign management system, computer systems 6 and user devices 8 may be understood to be a data processing system 100. Figure 6 illustrates a more detailed of a data processing system 100.

Figure 2 illustrates exchange of information among the computer systems that are shown in figure 1, according to an embodiment.

In an example, the web shop computer system 2 is a server system that is hosting a web shop via which parties can order an advertisement campaign. The web shop may for example comprise a user interface and/or a user form, via which a user can input advertisement campaign information, such as one or more properties of the job vacancy and/or advertisement campaign. Preferably, at least a job description is provided, a budget that the ordering party is willing to spend on the advertisement campaign and a duration of the advertisement campaign. The advertisement campaign information may indicate any property of the job vacancy, such as the geographical location(s) where the job is to be performed, job classification, salary indication, required educations, employer name, experience requirements of a candidate, industry category, occupational category, working hours, et cetera. Also, the advertisement campaign information may indicate any property of the advertisement campaign itself such as a total available budget for the advertisement campaign and a duration of the advertisement campaign and a goal of the campaign. A goal of the campaign is for example to achieve a total number of user interactions, optionally within a certain period of time. A budget referred to in this disclosure may be understood to indicate an amount of money that should be spent on the advertisement campaign and/or a maximum amount of money available for the advertisement campaign.

Then, the web shop computer system 2 may transmit one or more message 10 to the advertisement campaign management system 4 that comprise the advertisement campaign information that was input by the user via the interface and/or form on the website.

Subsequently, based on the advertisement campaign information, computer system 4 automatically selects in step 12 computer system 6a and computer system 6d. As referred to herein, a step that is performed automatically may be understood as that the step is performed without a human being involved in this step. Step 12 may also be referred to as the advertisement campaign management system determining the channel mix for the advertisement campaign. The selection of computer systems 6a and 6d may be performed using machine learning algorithms as explained in more detail below. The channel mix may not only define the computer systems 6 to which an instruction is to be sent, but also a budget for each channel.

In one example, the campaign management system 4 is configured to perform text analysis algorithms on the advertisement campaign information, such as on the job description provided, in order to determine a job classification. The system 4 may for example assign an International

Standard Classification of Occupations (ISCO) code to a job description. The classification model may for example be a recurrent neural network operating on customly trained deep neural network word representations that have been extracted from anonymized vacancies. This model may be trained on historical vacancies with ISCO codes available. Validation of the model may be performed by periodical manual inspection of sufficiently large samples.

Once assigned, the computer system 4 can determine the appropriate channels (computer systems 6) and/or the appropriate budget and/or bid height for each channel (computer system 6) based on the job classification.

The advertisement campaign management system 4 may be configured to perform a machine learning algorithm. Such machine learning algorithm revolves around constructing a model, which may also referred as training the computer system 4, to couple a set of input parameters to an appropriate channel mix. To this end, in the training phase, the computer system is provided training data relating to a plurality of actual advertisement campaigns. These are typically campaign that have been actually executed. The training data may then couple, for each advertisement campaign in the training data, a set of parameters to the channel mix that was selected to the advertisement campaign in question and also to an effectiveness of the campaign. The effectiveness is for example a total conversion ratio for the entire campaign and/or a set of conversion ratios, one for each channel. In any case, the training data allows to construct a model that, given a set of input parameters, can determine a channel mix, e.g. can determine a set of computer system and optionally a budget and/or bid height per computer system, that yields an effective campaign. The parameters present in the training data, and the input parameters that are input in the constructed model, may indicate for example the one or more properties of the job vacancy to which the advertisement campaign in question relates and/or the one or more properties of the advertisement campaign in question and/or the budget and/or bid height that was used for each channel.

Suitable machine learning algorithms for this are for example based on reinforcement learning, such as a multi-armed bandit algorithm.

The selection of an appropriate set of channels may have to meet certain requirements that are independent from the advertisement campaign information. Example of such requirements is that a channel can never user more than 50% of the total budget.

After selection of the appropriate channel mix for the advertisement campaign in step 12, computer system 4 transmits one or more message 14 to computer system 6a and one or more messages 16 to computer system 6d. These messages contain instructions to cause display of advertisements relating to the job vacancy on one or more user devices 8. It should be appreciated that each computer system 6 may require such instructions in a different format, e.g. may use a different communication protocol for receiving instructions. Hence, preferably, the campaign management system 4 has stored information enabling it to send instructions to any computer system.

It should be appreciated that the instructions 14 and 16 also comprise information, in the appropriate format, required for ultimately rendering the advertisements on the user devices 8.

As shown in figure 2, computer 6a sends, in response to the instruction received with the one or more messages 14, one or more message 18 to user device 8a and sends one or more message 22 to user device 8b. Similarly computer 6d sends, in response to the instruction received with the one or more message 16, one or more messages 26 to user device 8c and one or more message 30 to user device 8d. The message 19, 22, 26, 30 may comprise information required for displaying the advertisements of the campaign.

In response to these messages user device 8a, 8b, 8c, 8d, displays an advertisement related to the job vacancy in step 20, 24, 28, 32 respectively. Although figure 2 shows only two user devices for each of computer systems 6a and 6d, typically, many user devices can be reached via such computer system 6, thousands even.

In the embodiment of figure 2, the user device 8a transmits feedback information 34 back to computer system 6a, user device 8b transmits feedback information 35 back to computer system 6a, user device 8c transmits feedback information 36 back to computer system 6d and user device 8d transmits feedback information 37 back to computer system 6d. Such feedback information may for example comprise an indication whether a user of the user device in question has interacted with the advertisement. Such user interaction is for example a user clicking on the advertisement. Preferably, each user device transmits such information back to the computer system 6 in question.

Then, based on the received feedback information, computer system 6a can send user interaction information 38 to the advertisement campaign management system 4 and computer system 6b can send, based on the feedback information it received, user interaction information 40 to the advertisement campaign management system 4 as well.

Such user interaction information is very important for assessing the effectiveness of the different channels. In an example, the user interaction information, as sent by a computer system 6, comprises an indication of a number of user interactions with advertisements of which the display was caused by the computer system 6 in question. This allows the advertisement campaign management system 4 to determine, for each channel separately, the number of user interactions caused with the channel in question.

Preferably, the user interaction information 38, 40 indicates an effectiveness of advertisements the display of which is caused by the computer system 6a, 6b, respectively. Preferably, the effectiveness is based on, e.g. is, a ratio between a number of user interactions with an advertisement and a number of displayed advertisements. Such ration may also be referred to as a conversion ratio.

Preferably, budget spent information, which comprises an indication an amount of budget that was spent as a result of the computer system 6a or 6b causing display of advertisements, is sent to the campaign management system together with the user interaction information 38, 40. This is or course highly relevant as it allows not only to assess whether the campaign is effective, but also against which costs.

In the embodiment of figure 2, the campaign management system 4 automatically selects, in a step 42, based on the received user interaction information 38, 40 and/or based on budget spent information, only computer system 6d. Thus, apparently, the campaign management system 4 determines that the channel of computer system 6a is not effective, e.g. too few clicks per advertisements shown, and/or that too much budget is spent per unit of time on channel 6a. In order to correct for this, the computer system 4 selects a second set of one or more computer systems in the sense that it only selects computer system 8d for distributing advertisements relating to the job vacancy in question.

After computer system 4 has selected computer system 8b, it may transmit an instruction to computer 6d to (continue to) cause display of advertisements relating to the job vacancy. Optionally, if the instructions that are sent to computer system comprises a budget indication indicating a budget for the channel in question and/or a bid height that is to be used for purchasing advertisement opportunities via the channel in question, the instruction 44 may comprise an updated budget for the channel in question and/or an updated bid height for the channel in question. It may also be (not shown) that the campaign management system 4 transmits an instruction to computer system Ga to cancel the advertisement campaign, i.e. to stop causing display of advertisement on user devices 8.

It should be appreciated that throughout the advertisement campaign the campaign management system 4 may continuously receive user interaction information and/or budget spent information from the respective computer systems 6, which enables computer system 4 to continuously optimize the channel mix, e.g. to continuously optimize budget allocation and bids for external advertising channels to optimize towards a certain goal. As explained, in this process the real- time performance of individual campaigns, but also prediction based on historical data is used.

Figure 3 illustrates different modules within the advertisement campaign management system 4, also referred to as the Performance Steering Agent (PSA). The Campaign module receives advertisement from an external system. This can be web shop, as already explained above, but the external system can also be a Job Manager module, which may be a module that a recruiter uses to directly send advertisement campaign information to the Campaign module.

Based on the advertisement campaign information, the Campaign module selects a set of computer systems, each being associated with some channel, as depicted on the right hand side.

Instructions are sent to these computer systems to cause display of advertisements on user devices.

In an example, one of the channels is a social network website. Then, the computer system of this social network site can cause display of advertisements on user devices that are visiting the social network website.

A channel may be of the aggregator type, see module “Aggr gateway”, which typically means that a party has to place bids in order to receive an opportunity for displaying an advertisement. To this end, the bidding engine module determines, based on the advertisement campaign information an appropriate bidding strategy. This bidding strategy may define a start bid. Additionally such bidding strategy may define how the height of the bid should evolve over time. The bidding engine module may then provide the bidding strategy to the aggregator channel in question.

As shown, the computer systems associated with the respective channels provide feedback information to the campaign management system 4. The aggregator module is for example shown to provide feedback information about the bidding strategy, e.g. the costs per advertisement, to a bid/config tracking module. This module for example calculates how much of the budget is spent on the aggregator channel through the bids.

Campaigns with the aggregator channel included in the channel configuration are typically using programmatic bidding to optimize the performance of individual job campaigns. The campaign management system 4 is preferably configured for performing these two important steps: setting starting bids and re-evaluating these bids over time.

At the start of a campaign the bidding engine module may provide a starting bid for each of aggregator channel based on a predefined set of starting bids, and preferably also based on the advertisement campaign information.

After the starting bids have been generated the campaign management system 4 may start the optimization process by frequently re-evaluating the bids for each channel and changing them when necessary. The system may have multiple ways of optimizing, depending on the goal of the campaign.

One approach is based on two data points: budget and time. Each job campaign with this campaign may have a budget and an end date, which enables optimization towards spending the budget before the end date is reached. The system may try to distribute the budget equally across the duration of the campaign. If progress on budget spent is lacking behind, the bids may be increased, if progress is ahead of schedule the bids may be decreased.

As explained above, a more complex proposition based on machine learning methods is also envisioned. Such machine learning methods may be based on three components: a bid landscape forecaster; an impression value estimator; and a reinforcement learning agent combining the previous two components with other available information into an actionable model. In brief, the bid landscape forecaster can predict how high other parties that participate in the auction will bid based on previous auctions. The impression value estimator predicts the probability and value of a conversion based on available client characteristics. The reinforcement learning agent decides how much to bid based on past results, job characteristics, available budget, time, and the estimates from the landscape forecaster and value predictor.

The system 4 may for example be configured to get the required data to aggregators using XML and/or API. A lot of aggregators are using XML-feeds to get job information and bids. Preferably, system 4 is able to create and frequently update the XML-feeds.

Figure 3 shows that the non-aggregator channels may provide feedback information to the budget tracking module. The feedback information for example comprises budget spent information as explained above.

The budget tracking module provide the budget spent and for example conversion ratios to the performance tracker, which monitors the effectiveness of the campaign. The performance may for example serve as input for a machine learning algorithm that is executed by the prediction engine.

Using a machine learning algorithm, the feedback may be used to improve the model for determining an appropriate channel mix for an advertisement campaign. In any case, the prediction engine may forecast, for a give channel mix, the effectiveness... Based on these analyses, the budget allocation engine may adapt the budget and/or may adapt the selected channel. As a side note, if the budget allocation engine determines a budget of zero for a channel, this is equivalent to de-selecting the channel in question for the advertisement campaign.

Figure 4 is a flow chart illustrating a method according to an embodiment. In this embodiment, the campaign is “boosted”, which may be understood as that a significant increase in the available budget for the advertisement campaign has occurred. The boost of the campaign triggers that appropriate channels are selected and an appropriate bidding strategy is determined based on the new budget.

Figure 5 is a flow chart illustrating a method according to an embodiment.

After the creation of the campaign the campaign management system 4 may continuously analyze the performance of the campaign and change the strategy and/or bidding if needed. The system 4 may register spending and performance metrics to determine the status of active campaigns and use this data to increase or decrease bidding, or change the allocated budget for a channel in order to get performance for the lowest cost in such manner that not more is spent than defined in the advertisement campaign information.

After a certain goal is reached (whether it's budget or performance) the system 4 may automatically disable the campaign, for example by sending appropriate instructions to the computer systems 6 of the respective channels.

Fig. 6 depicts a block diagram illustrating a data processing system according to an embodiment.

As shown in Fig. 8, the data processing system 100 may include at least one processor 102 coupled to memory elements 104 through a system bus 106. As such, the data processing system may store program code within memory elements 104. Further, the processor 102 may execute the program code accessed from the memory elements 104 via a system bus 106. In one aspect, the data processing system may be implemented as a computer that is suitable for storing and/or executing program code. It should be appreciated, however, that the data processing system 100 may be implemented in the form of any system including a processor and a memory that is capable of performing the functions described within this specification.

The memory elements 104 may include one or more physical memory devices such as, for example, local memory 108 and one or more bulk storage devices 110. The local memory may refer to random access memory or other non-persistent memory device(s) generally used during actual execution of the program code. A bulk storage device may be implemented as a hard drive or other persistent data storage device. The processing system 100 may also include one or more cache memories (not shown) that provide temporary storage of at least some program code in order to reduce the number of times program code must be retrieved from the bulk storage device 110 during execution. input/output (1/0) devices depicted as an input device 112 and an output device 114 optionally can be coupled to the data processing system. Examples of input devices may include, but are not limited to, a keyboard, a pointing device such as a mouse, a touch-sensitive display, or the like.

Examples of output devices may include, but are not limited to, a monitor or a display, speakers, or the like. Input and/or output devices may be coupled to the data processing system either directly or through intervening I/O controllers.

In an embodiment, the input and the output devices may be implemented as a combined input/output device (illustrated in Fig. 6 with a dashed line surrounding the input device 112 and the output device 114). An example of such a combined device is a touch sensitive display, also sometimes referred to as a “touch screen display” or simply “touch screen”. In such an embodiment, input to the device may be provided by a movement of a physical object, such as e.g. a stylus or a finger of a user, on or near the touch screen display.

A network adapter 116 may also be coupled to the data processing system to enable it to become coupled to other systems, computer systems, remote network devices, and/or remote storage devices through intervening private or public networks. The network adapter may comprise a data receiver for receiving data that is transmitted by said systems, devices and/or networks to the data processing system 100, and a data transmitter for transmitting data from the data processing system 100 to said systems, devices and/or networks. Modems, cable modems, and Ethernet cards are examples of different types of network adapter that may be used with the data processing system 100.

As pictured in Fig. 6, the memory elements 104 may store an application 118. In various embodiments, the application 118 may be stored in the local memory 108, the one or more bulk storage devices 110, or apart from the local memory and the bulk storage devices. It should be appreciated that the data processing system 100 may further execute an operating system (not shown in Fig. 6) that can facilitate execution of the application 118. The application 118, being implemented in the form of executable program code, can be executed by the data processing system 100, e.g., by the processor 102. Responsive to executing the application, the data processing system 100 may be configured to perform one or more operations or method steps described herein.

In one aspect of the present invention, the data processing system 100 may represent a computer system 2, 4, 6, 8 described herein.

In another aspect, the data processing system 100 may represent a client data processing system. In that case, the application 118 may represent a client application that, when executed, configures the data processing system 100 to perform the various functions described herein with reference to a "client". Examples of a client can include, but are not limited to, a personal computer, a portable computer, a mobile phone, or the like.

In yet another aspect, the data processing system 100 may represent a server. For example, the data processing system may represent an (HTTP) server, in which case the application 118, when executed, may configure the data processing system to perform (HTTP) server operations.

Various embodiments of the invention may be implemented as a program product for use with a computer system, where the program(s) of the program product define functions of the embodiments (including the methods described herein). In one embodiment, the program(s) can be contained on a variety of non-transitory computer-readable storage media, where, as used herein, the expression “non-transitory computer readable storage media” comprises all computer-readable media, with the sole exception being a transitory, propagating signal. In another embodiment, the program(s) can be contained on a variety of transitory computer-readable storage media. llfustrative computer-readable storage media include, but are not limited to: (i) non-writable storage media (e.g., read-only memory devices within a computer such as CD-ROM disks readable by a CD-ROM drive, ROM chips or any type of solid-state non-volatile semiconductor memory} on which information is permanently stored; and (ii} writable storage media (e.g., flash memory, floppy disks within a diskette drive or hard-disk drive or any type of solid-state random-access semiconductor memory) on which alterable information is stored. The computer program may be run on the processor 102 described herein.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of embodiments of the present invention has been presented for purposes of illustration, but is not intended to be exhaustive or limited to the implementations in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the present invention. The embodiments were chosen and described in order to best explain the principles and some practical applications of the present invention, and to enable others of ordinary skill in the art to understand the present invention for various embodiments with various modifications as are suited to the particular use contemplated.

Claims (15)

CONCLUSIONS 1. A computer-implemented method for managing an advertising campaign for a vacancy, the method comprising receiving advertising campaign information, wherein the advertising campaign information indicates one or more properties of the vacancy and the advertising campaign information indicates one or more properties of the advertising campaign, and automatically selecting, on the basis of the advertising campaign information, a first collection of one or more computer systems from a plurality of computer systems, wherein each computer system from the plurality of computer systems is arranged to cause advertisements to be displayed on one or more user devices, and automatically sending, to each computer system in the first set, an instruction to cause display of advertisements related to the job posting on one or more user devices. The method according to claim 1, wherein the one or more properties of the vacancy include - a geographical location or area where the job is performed, and/or - a job description, and/or - a job classification, and/or - a salary indication , and/or -a name of the employer. The method according to claim 1 or 2, wherein the one or more properties of the advertising campaign include - a total available budget for the advertising campaign, and/or - a duration of the advertising campaign. The method of any one of the preceding claims, the method further comprising receiving, from each computer system in the first set, user interaction information including an indication of a number of user interactions with advertisements whose display was caused by the computer system in question. 5. The method according to the preceding claim, wherein the user interaction information indicates an effectiveness of advertisements whose display is caused by the computer system in question, wherein the effectiveness is preferably based on a ratio between a number of user interactions with an advertisement and a number of displayed advertisements. The method according to any of the preceding claims, further comprising receiving, from each computer system in the first set, spent budget information including an indication of an amount of budget spent as a result of the computer system in question causing advertisements to be displayed on one or more user devices. 7. The method according to any of claims 4-6, further comprising automatically selecting a second set of one or more computer systems from the plurality of computer systems based on the received user interaction information and/or based on the received spent budget information , wherein the second set is different from the first set, and automatically sending, to each computer system in the second set, an instruction to cause display of advertisements related to the job opening on one or more user devices. The method according to any of the preceding claims, wherein the advertising campaign information indicates a total budget for the advertising campaign, the method comprising determining, based on the total budget, for each computer system in the first and/or second set, of a budget and/or a bid amount specific to the computer system in question, wherein the instruction sent to the computer system in question includes a budget indication indicating the certain budget and/or the certain bid amount. 9. The method according to any of the preceding claims, wherein the advertising campaign information comprises a job description, the method comprising - determining a job classification on the basis of the job description, and - determining, on the basis of the job classification, the first collection of computer systems and/or determine, for each computer system in the first and/or second collection, a budget and/or bid amount specific to the computer system in question. The method according to any of claims 4-8 and according to claim 8 or 9, further comprising, based on the received user interaction information and/or based on the received spent budget information, automatically sending an updated instruction to at least one computer system in the first and/or second collection, wherein the updated instruction includes an updated budget indication indicating a new budget and/or new bid amount. The method of any of the preceding claims, wherein selecting the first set of computer systems includes executing a machine learning algorithm, the machine learning algorithm comprising -constructing a model based on training data, wherein the training data associates an effectiveness score with each advertising campaign from a plurality of advertising campaigns for respective vacancies, wherein the effectiveness score indicates an effectiveness of the advertising campaign in question, and wherein the training data for each advertising campaign from the multitude of advertising campaigns, indicate the one or more properties of the vacancy to which the advertising campaign in question is related and/or indicate the one or more properties of the advertising campaign in question and/or indicate the set of computer systems to which instructions have been sent for the advertising campaign in question to display advertisements on one or more user devices, and - using the constructed model to select the initial set from one or more computer systems. The method according to the preceding claim and according to claim 9, wherein the training data for each computer system as indicated by the training data indicates a budget and/or a bid amount that was included in the instruction sent to the computer system in question, and it, under using the constructed model, determine for each computer system from the selected first collection of one or more computer systems, the budget and/or the bid amount specific to the computer system in question, whereby the budget assigned to each computer system from the selected first collection of one or more computer systems includes a budget indication that indicates the budget and/or the bid amount determined for the computer system in question. 13. A data processing system comprising a processor adapted to carry out the method according to any of the preceding claims 1-12. A computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any of the preceding claims 1-12. A non-volatile, computer-readable storage medium on which the computer program of claim 14 is stored.