JP2017084403A - Data mining to determine online user responses to broadcast messages - Google Patents

Abstract

PROBLEM TO BE SOLVED: To increase advertising effectiveness and limit advertising so that it goes to interested consumers.SOLUTION: In a networked system, a method and apparatus for measuring advertising effectiveness involves: broadcasting an advertisement at a broadcast time; monitoring at least one social network or other online forum following the broadcast time; filtering social network traffic to identify messages that appear to be responsive to the broadcast advertisement, filtering search engine queries to identify such queries, or filtering other such traffic entries; determining a time of initiation for at least a plurality of entries that appear to be responsive to the broadcast advertisement; and determining at least one demographic for at least the plurality of the entries that appear to be responsive to the broadcast advertisement.SELECTED DRAWING: Figure 2

Description

[Cross-reference with related applications] This application is a benefit of priority of US Provisional Patent Application No. 61 / 414,384, filed on November 16, 2010, under 35 USC 119 (e). Which is incorporated herein by reference in its entirety for all purposes.

  The present invention relates generally to data mining network activity to infer statistics regarding user response and behavior, and in particular, using time-constrained data sources, unrelated media and random events. Related to identifying time-based responses to advertising messages in social activities.

  The effectiveness of TV advertising is measured to some extent by broadcasting a “call now” message (sometimes loudly) so that interested people can call immediately. can do. However, this response is limited to people who are immediately interested in what is being sold and may have been influenced by the ad, despite being unable to make a decision about the proposal or being able to call immediately. There is a possibility that people with sex will be excluded.

  With the advent of Internet-based advertising (in this document, the term “Internet” is used as an example of a network that allows many people to participate in many roles) Other techniques for measuring effectiveness have been developed in which the URL required when a viewer clicks on an ad first calls the ad tracking server and then this URL is , Inherited by another URL that provides additional information on the screen. If the system that provides the advertisement is the same as the system that measures the response, this measurement can be simple. However, it is more complicated when there is no direct connection.

  A well-known adage attributed to John Wanamaker (1832-1922), a department store manager, “I know that half of the advertising costs are wasted, but the problem is that I do n’t know which half.” There is something called. This adage urged advertisers to focus on limiting advertising to reach interested consumers rather than uninterested consumers, in addition to increasing advertising effectiveness.

  Some ad effectiveness initiatives rely on collecting large amounts of data about individual consumers and targeting ads to each of these consumers. This effort is often disliked by consumers who do not want their personal details to be preserved with these efforts and may still be looking at unrelated advertisements. This approach may also require excessive effort on the part of the advertiser in handling individual messages for individual potential consumers.

  In a networked system, a method and apparatus for measuring advertising effectiveness broadcasts an advertisement at a broadcast time and monitors at least one social network, search engine, or other online forum after the broadcast time. Filter social network traffic to identify messages that appear to be a response to broadcast ads, filter search engine queries to identify such queries, or filter other such traffic inputs Identifying a start time of at least a plurality of inputs that are likely to be a response to the broadcast advertisement, and at least a response that is likely to be a response to the broadcast advertisement. And a step of evaluating the advertising effect by identifying at least one demographic of the number of inputs.

  An advertisement management method for delivering a message to be tracked by a broadcast method is provided. After this broadcast, the aggregate response (ie, the aggregation of individual or group responses to the broadcast message) is monitored. Based on this aggregate response, a selection is made regarding the next action, which may include subsequent advertisements, often near real-time media, product, or presentation changes. Aggregate responses (and even individual or group responses) can be broadcast. Aggregate responses can be aggregated based on the demographic slice of the entire viewer.

  The nature and advantages of the teachings of this application may be better understood with reference to the following detailed description in conjunction with the accompanying drawings.

  For a more complete understanding of the teachings of the present application, reference is made to the accompanying drawings. With the understanding that these drawings should not be considered as limiting the scope of the subject matter of the present application, additional details regarding the presently described embodiments and the best mode currently contemplated will be used, with reference to the accompanying drawings. It explains together.

FIG. 2 illustrates an example data mining and feedback system according to one embodiment. It is a flowchart of the method of identifying the reaction with respect to a message by performing data mining. FIG. 2 is a block diagram of a computer system in which some of the elements can operate. FIG. 6 is a block diagram illustrating the operation of various processing components that process a message stream.

  This document describes an improved data mining and feedback system. Such a system can be implemented using hardware and / or software in a networked, distributed and / or local server system.

  In an example data mining and feedback system, a message is broadcast in a broadcast manner to known or unknown recipient groups or potential recipient groups. The receiver's aggregate response (or their approximations, samples, or proxies) is usually monitored in real time or near real time, after which the data mining and feedback system determines what to broadcast next Take some action. These responses may or may not be broadcast. Examples of actions to take may include changing the message based on feedback of the reaction, changing the content, and / or providing additional messages. Depending on the case, the behavior that is presumed to be a response to the initial message may be used to infer an uncertain aggregate response and the messaging system may be monitored for messages that are presumed to be a response to the initial message, or Search queries or other behavior can be monitored.

  In some cases, the estimated reaction itself is inferred from other activities. For example, if an ad server or content server provides content based on other aggregate behavior, such as user queries or messages, the activity of the ad server or content server could not have been observed directly It is possible to infer what the user activity is. As a specific example, assume that the initial message was a broadcast advertisement for a movie entitled “Speedy Sam Invades Lower Manhattan”. After this broadcast, if the search query “Will Speed sam in showdown in Smalltown?” Increases or shows a surge, then the advertiser is advertising on this movie in Smalltown ( Or increase the number of hotel reservations in Lower Manhattan at approximately the same time, but ignore the initial message as irrelevant Can do. Of course, in many cases, the correspondence between the initial message and its response may not be as significant as in this example, and may not be as significant for the automatic data mining process.

  The details of the various components of the data mining and feedback system are described below.

  FIG. 1 shows an example of a data mining and feedback system. It should be noted that the construction of a general-purpose server that can broadcast and / or operate messages over a network is well known, so in order to understand the teachings of this application, details of a server or client device, or a broadcast transmitter and the Internet Operations such as communication need not be described in detail herein.

  FIG. 1 illustrates a system 100 that includes a message server 101 that communicates the initial message 102 to a number of recipient electronic systems 103 where the recipient can view, listen to, and experience the initial message 102. In response, the recipient is not necessarily a clear response to the initial message, but electronically, such as sending a message to a friend, message sender, etc., performing a search, or taking some other action. It is thought that it will take some action that can be recognized by people. Such actions are believed to be performed via the Internet 104, sending a message via the user message server / service 105, initiating a search using the search engine system 106, or requesting content By doing so, it is considered to include the step of causing the advertisement server 107 to provide another advertisement. The data mining and feedback system 108 makes an application programming interface (“API”) call 110 to one or more of the user message server / service 105, the search engine system 106, or the advertisement server 107, and the aggregate response Is configured to query such a system, such as by looking back or obtaining a data record 111 that can be used to identify it. The data mining and feedback system 108 can then process the information to determine the appropriate action and send an electronic instruction 115 to this effect.

  Unlike a pure point-to-point system where the message server sends a message to a known set of individual nodes and obtains individual responses in return, the data mining and feedback system 108 In addition, aggregate responses can be identified among messages initiated from non-recipients or messages initiated from recipients that are not responses to receipt of the initial message. The data mining and feedback system 108 can use publicly accessible information or privately accessible information.

Monitoring the validity of transmitted messages In some embodiments, a message is transmitted to an unknown number of recipients via a broadcast medium (television, radio, etc.). In order to identify the impact of the message on these recipients (and possibly the size of the set of recipients), the data mining and feedback system 108 “traits traffic streams for various online social networks or other sources. Data mining "can be performed, messages to these sources can be scanned using computer processing, and these messages can be flagged according to the time of message creation and the relationship with a particular advertisement. The low latency feedback from these sources can be used to tailor the programming of the advertisement on a time scale that can range from hours to days.

  In a specific example, assume that Company C runs an advertising campaign for product P and schedules an advertisement for product P in the national spot frame at 6:00 pm Pacific Standard Time. Public advertisements can be monitored by computer processing when advertisements are flowing. For example, it is possible to monitor a data stream of a message posted by a user of Twitter service provided by Twitter or monitor a message in another online forum by computer processing. In other cases, the scan data is not published, but can be provided to or utilized by computer processing. An example may be a search engine service having a search stream that has been entered and a data stream that includes all of the times that they were entered.

  Computer processing can filter unrelated messages from messages that react to advertisements. For example, it is assumed that the product P is an automobile and has a model of “Speedster”. The ad was aired on TV, and immediately after that on the social network (Would you like to see a Speedster car commercial on the wall of the building? ("Did you see the commercial for the car, the Speeding the climbing the climbing the crime." message "or" Saw an for a car that I want 2 buy "). As is apparent from these two examples, identifying messages sent in response to advertisements can be easy or difficult. Naturally, ("I want to buy one car after seeing an advertisement for a car that goes up the building, where can I buy it?" I can you buy for the car that builds building and want to buy one. Where can I?) Or “I just called you because I was dissatisfied with your ad (I just had to call to say with your advertisement)” and the message was from the user and directed to the advertiser If it is related to advertising, it is easy to identify the message. If the message is not necessarily returned to the advertiser but is a public message intended for others, or is inferred from non-messaging behavior, such as a search using keywords, message identification is It becomes more difficult.

  In some cases, non-messaging behavior may be speculative. For example, an operator of a messaging server can access an ad server (or aggregated data from this server) that delivers advertisements based on what the participant group is doing or requesting, but these participants If the data about what the person is doing or is requesting is not accessible, the advertising service pattern from the advertising server can be used as a proxy.

If the response of message traffic, such as an aggregate indication of message density for a particular ad, can be measured at least approximately, the details of this campaign (content, timing, repetition, etc.), even while the ad campaign is running Can be adjusted.
In some cases where demographics can be inferred or used, this can be used to more precisely refine traffic analysis and feedback. For example, the product may be deeply discussed about the product, perhaps with messages from social network users, such as those who admit cat lovers, or who are 20 to 35 years of age, and such demographics are interesting and unexpected for advertisers And / or can be useful.

Several methods can be provided for evaluating sparse vector representation message activity. In one embodiment, each input (message, search, action, etc.) is represented as a sparse vector of length O [1000-10000], where each input is of importance to a particular “word” or unit action. Contains a floating point number representing the weight. This representation allows conventional or novel subspace techniques to be used for further processing.

  In a specific example, a stream of thousands of messages or search queries (“input”) is scanned and parsed (which can be a string of letters and / or numbers and / or punctuation) individually. To create a word list by frequency. From this, the top N words in the selected set and the words that occur less frequently than the top M words in the selected set are excluded. In a specific example, let N = 300 and M = 10,000, consider 10,000 most frequently used words, except first discard the top 300 words. A word can be “stuck” before a tense expression (eg, “stop” and “stopping” are treated as the same single word).

  A set of sparse vectors can then be formed with the respective weights of the remaining 9,700 words. In some embodiments, the weight is directly related to the frequency of occurrence, while in other embodiments, the weight also considers an estimate of the descriptive power of the “word” in which it appears. For example, “horspower” can obtain a higher descriptive weight than “the”.

Filtering automatic input from manual input In some cases, it may be desirable to use a sub-filter that weights automatic input lower than manually created input and removes automatic input as part of the filtering process There is a case. Of course, this order can be reversed if the particular application works better with automatic input.

  Separating machine input or partial machine input from input written alone can further enhance the characteristics of what people are doing rather than what computer processing is doing. Examples of machine input or partially machine-created input include a large number of “retweets” in the Twitter system, reposted blog items, or “blog this” or “tweet this” on articles. An input created using the (Tweet to this) button.

  This separation is very important for further statistical analysis because these inputs are much more consistent than inputs written alone. In some cases, if the method that created the input can be identified, different weights can be applied to the input depending on this. For example, it is considered that there are too many inputs that are easy to create compared to inputs that are difficult to create. As an example of weights, a value of “1” is applied to pure manual input, “0” is applied to generate fully automatic input, and other values are considered between these extremes. it can.

  One way to measure various weights from machine input to manual input is to generate identical or nearly identical input species and then use these species as nuclei to “close” to these species. It is to accumulate input. The definition of “near” may be in terms of Hamming distance, normalized distance in the subspace representation, and / or other definitions.

  Another way to measure the weight is to define the input as a set of weights for the subspace basis vectors. The filtering or module can then search for peaks in the one-dimensional or multi-dimensional histogram of weights representing machine inputs or partial machine inputs or partially created inputs.

  In this way, the basis vectors can be optimized to maximize to load singularities. For example, the varimax scale can be maximized in the input direction to generate a basis function representing the reference tweet.

  In some embodiments, peak detection includes two histograms that are offset by a half cell interval, and peak detection is performed by comparing cell values to two overlapping cells in the other histogram. This captures peaks that may have been lost because the peak data points are split across the two bins.

  In certain implementations, the multidimensional space is populated with consistent inputs (such as tweets) that can form clusters that can produce peaks. In this case, the system takes the magnitude of the load to form a histogram and phase shifts the histogram so that the peaks are not flat where the histogram bins are split.

Hierarchical grouping of inputs Start with a set of inputs and divide them into sets of similar input hierarchical groups. One use of such partitioning is to display to the user the statistics of the set of inputs and the time evolution each identified by a “typical” input. These time series can also be used to identify responses to advertising events.

In one method, the analysis system builds an optimized set of basis vectors of size about 5-50 in the method described above, for example, by maximizing the varimax measure.
The analysis system uses the loads on these basis vectors to classify inputs that have significantly higher loads on one of these vectors into a class defined by the vector with the largest load amplitude. The analysis system then recursively splits these classes and the remaining classes that do not have a large load on any of these vectors.

  Specifically, this splitting can include splitting inconsistent tweets, examining details and taking subspace modes. This coordinate system then becomes the reference tweet. Rather than conventional clustering with a complexity of N squares, this method provides an operation similar to using a load vector and then drawing a line that cuts the space into the space.

  Alternatively, the analysis system builds a hierarchical class by using traditional population exchange clustering techniques, and instead of using Hamming distances, distances are defined by normalized vector distances in the subspace. Is done.

Monitoring System A monitoring system can be provided for quickly identifying new topics of input by continuously mapping new inputs to the basis functions defined above. In this case, the remaining inputs are examined to determine if there are new consistent input classes, and the inputs associated with these new consistent classes are identified as new discussion topics and potential interests. . Modify the subspace for these inputs and update for future monitoring. In fact, in contrast to searching for new hot topics by word search, this monitoring system is used to transmit new memes (ie, from person to person in a manner similar to the biological propagation of genes). Cultural or social ideas or concepts).

Basis Set Calculation Basis set over almost the entire subspace defined by the first O [3-50] singular value decomposition (“SVD”) or principal component analysis (“PCA”) mode of a large sparse data matrix ( The method of calculating the truncation set) uses a power iterative approach to calculate an approximate first vector, and the convergence criterion is not that this first vector is a true mode, but the truncation set Based on being included only in the defined subspace. After computing the first vector, the original data matrix does not “shrink” because this “shrinkage” results in a low-sparseness data matrix. Instead, power schemes are used to construct subsequent basis vectors, in which case the previous vectors are used to contract the iterative vector in a potentially “delayed” manner.

  The process is simplified by relaxing the requirement to accurately determine the eigenvalues. Sparse SVD is difficult to analyze because the original data matrix is sparse. One method uses truncation instead of accurately determining the eigenvalues.

One module of the author classification system can classify the input set authors into a reference class by grouping the input set authors with authors with similar word usage patterns. These classes can be used to divide authors into “equivalence classes” that can be used to infer demographic or interest group information. These classes can also be used to normalize the metric of preference since the definition tag for mean likability factor or likability varies from author group to author group. The classification is whether the responder (ie author) is a mechanical author (computer responder, bot, etc.) or a non-mechanic author such as a human who responds to the fact that he actually received the initial message. Instructions can also be included.

  For example, other demographics can be obtained by performing the same analysis with equivalence classes on the word distribution used by authors across multiple tweets and knowing some demographics. Thereafter, the favorableness by demographics can be evaluated and identified relative to the baseline.

  As an example of the word distribution, when a response from a specific recipient has a high appearance rate of “Mom”, “Dad”, “Shopping Mall”, and “(School school)” Can be inferred that the message is from a demographic “suburban teenager”, whereas “burden of proof”, “evidence hearing” cite a rule that if the rate of occurrence of “healing” and “cross-examination” is high, the message can be inferred from a demographic “trial lawyer” Can do.

Other General Aspects As highlighted in detail above, a data mining and feedback system can also be implemented over a network, where messages flow between nodes and a user uses computer equipment at multiple nodes. And at least initiating the message, some of these users are recipients of the initial message broadcast from the message server or source. For one, broadcast an advertisement at broadcast time, monitor at least one aggregate activity stream after the broadcast time, and filter the aggregate activity stream to identify messages that appear to be a response to the broadcast advertisement. By identifying when these messages were sent, filtering out messages with a high probability of being unrelated, and identifying at least one demographic for at least these multiple messages that appear to be a response to a broadcast advertisement , Advertising effectiveness for at least one demographic can be measured.

  The activity stream can represent one or more messages sent by one or more users, and the aggregate activity stream forms a statistically significant group for which advertising effectiveness should be measured. An activity stream from a collection of users can be represented. The aggregate activity stream can be a message that is social network traffic, a message that is a search query, a request for one or more web assets, or other messages or traffic. The request for one or more web assets may be a request for a web asset identified by a URL stored in a monitored URL list, at least one of the monitored URLs being a movie advertisement, etc. In connection with at least one previously broadcast advertisement, the monitored URL list includes URLs of web pages containing trailers for informational material, promotional material, or movies to be advertised.

  The real-time advertisement management system extracts or mines a user activity determined to be a response to a previously broadcast advertisement from a server for monitoring user messages in real time and a user message to be monitored, and extracts the extracted user And a data mining module for selecting one of a plurality of broadcast advertisement candidates based on the identification result of the activity. As used herein, “real time” refers to the time that a user, viewer, and others can perceive a response as a real time response. For example, if a 30-minute TV program is broadcast and a user message sent 10 minutes after the program starts, the viewer is interested in a particular advertisement, a scenario change, or other reaction, Changing the scenario at the 20-minute mark is considered to be sufficiently real-time, but it cannot be considered real-time if a reaction occurs several times later in the broadcast. Depending on the specific content, some real-time responses can be within a second of the user message being broadcast, some within a few minutes and some within a longer time Can do. In other situations, perhaps even a few minutes will not be considered a real-time interval.

  Previously broadcast advertisements can be advertisements delivered via one or more of television, radio, bulletin board, and / or internet advertising media. A television, such that a selected one of a plurality of broadcast advertisement candidates can be communicated using a different medium between a previously broadcast advertisement medium and a selected one of the plurality of broadcast advertisement candidates; It may be an advertisement to be communicated via one or more of radio, bulletin board, and / or internet advertising media.

  The real-time ad management system includes a message type analyzer to analyze messages and at least estimate or identify which messages are from mechanical sources and which messages are from non-mechanical sources Can do. A non-mechanical source may be a source that includes a human response or response that responds to or reacts to the actual viewing of a previously broadcasted advertisement and later to a previously broadcasted advertisement. it can.

  Real-time ad management systems can also include logic to take action based on a statistical measure of aggregated messages, where actions taken on messages from mechanical sources are taken from messages from non-mechanical sources. It is different from the action taken against.

  The real-time advertisement management system may also include a message demographic analyzer that analyzes the message and at least estimates or identifies demographics about the message based on the content of the message.

  One response of the real-time advertisement management system is to extract or mine user activity that is determined to be a response to a previously broadcast advertisement from the monitored user message, and based on the extracted user activity identification results, One or more of the direction of the broadcast content, the program production policy, and / or the camera angle may be changed.

  While many examples are possible, a set of specific examples will be described with the understanding that the scope of this application is not limited to the following examples. One implementation is to measure the marketing effect of a special movie “Famous Pirate Movie Part 2” and a broadcast message about its star actor “Standley Big-Talent”. Suppose there is.

  First, a filter check can be applied to the search engine stream to identify words that match search terms such as “Pirates”, “Stanley”, “Big-Talent”. The demographic tagger then tags messages with words that are “two word sets” or “phrases”, but this tagging can also be applied to a single word. A demographic tagger can be used to identify the age group of the message source. For example, “Stanley is the groovy”, “Stanley is the bomb”, and “Stanley is sick” is the one that Stanley looks like. Are different, but specific to a particular age group (eg, in one teenager or preteen language, “sick” is a synonym for “great”). Several dialects, idioms, or language details can perform demographic tagging by geographic region.

  Once the message is tagged, it can be further characterized by a message type such as “mechanical” versus “non-mechanical”. This characterization can be done by identifying the level of similarity between messages, and mechanical messages have very high scores for similarity, so they can be deleted or weighted down.

  Next, emotion analysis can be performed. Many messages may have mixed emotions, but nevertheless they can be grouped into "generally positive", "generally negative", and "mixed". Depending on the case, it is possible to separate the mixing of messages. For example, "I can't believe that I'll make the movie" famous pirates ", but Stanley Big-Talent is wonderful (I cant believe the hey making another pirate move movie. Stanley Big-Talent. Message). The message shows a negative rating for one topic (movie) and a positive rating for the other (starring actor). This is useful because it can be used effectively for marketing.

Pulse and continuous triggers As described above, broadcast advertisements can be issued to measure responses. This is an example of how the system described herein can be used to generate a “pulse” and measure a “pulse response” shortly thereafter. For example, for a movie, two advertisements can be run, each focusing on a different character in the movie. When the first pulse is sent (eg, an advertisement focused on the first character is broadcast) and the response is measured, the pulse after focusing on the second character (or the same Or different direction or market pulses at different times) and the response is measured. This difference in pulse response can be used to adjust the movie or further marketing. In some cases, there may be two (or more) simultaneous pulses in the same market where pulse response differentiation is possible.

  In some embodiments, a pulse is transmitted, the response is simply recorded, and unrelated or irrelevant responses are discarded. The analysis system can generate a sparse vector set to be used as a sparse SVD from the remaining responses and perform hierarchical clustering. The message type (mechanical or non-mechanical) may or may not be considered. Message formats (SMS text using services such as Twitter, email, social network messages, short messages, etc.) can also be considered. From such a response analysis, it is possible to make a guess about the effect of the pulse and take action according to these guesses. Some actions may include changing future advertising, product or service proposals, or other actions. In the case of a movie, this guess is used to make edits or other changes to the movie to increase the importance of this character in the movie in advance if the pulse for that particular character produces a higher response than expected. You can make your audience happy.

  In some embodiments, the pulse and response are not limited to discrete events, form a continuous monitor feedback loop in which a pulse stream exists, or the response to the pulse is an action, which is itself for further response analysis. A discrete feedback loop that is a pulse of

  FIG. 2 is a flow diagram of a method for identifying a response to a message by performing data mining. As shown, the method begins with the generation of a pulse event (step S1). This pulse event may be a message, communication or other event that should cause a response. At time tb, a message such as a broadcast advertisement that reflects this pulse event is generated (step S2). After time tb, the message monitor monitors network traffic (S3). This traffic can take the form of social network messages (such as posting Twitter messages, activities on Facebook® pages, etc.), search queries, URL requests, and the like.

  The traffic to be monitored is filtered to exclude, ignore and / or reduce the weight of messages that are deemed not originating from pulse events and that occurred after time tb (S4). Among the remaining messages, messages considered to be responses, results, and / or responses to pulse events are collectively analyzed to identify message characteristics (S5). In some cases, the words used are considered, demographics are estimated, and / or discussion topics are extracted. Details of such operation have been described above.

  From this analysis result, the analyzer system makes a selection from a plurality of predetermined options that can include selection of follow-up advertisements, selection of changes to products, services or proposals, or selection of other actions (S6). The selected action is then taken or a command is issued that can take the form of an automatic electronic message to perform this action.

  Further actions can be taken if a feedback loop exists (S7), for example, this selected action can be followed by sending a broadcast message (S8), sending a message (S9), and the resulting network Loopback to traffic monitoring.

  FIG. 3 is a block diagram of a computer system in which some of the elements of the method and system of the present application can operate. This figure illustrates a general computer system 300 according to various embodiments described in this application. In various embodiments, the computer system 300 typically includes a monitor 310, a computer 320, a keyboard 330, a user input device 340, a network interface 350, and the like.

  In the present embodiment, the user input device 340 is typically embodied as a computer mouse, a trackball, a trackpad, a wireless remote controller, or the like. Typically, the user input device 340 allows the user to enter action selections and insert rules for the various rule sets used by the components of the data mining system.

  Typically, network interface 350 embodiments include Ethernet cards, modems (telephone, satellite, cable, ISDN), (asynchronous) digital subscriber line (DSL) units, and the like. Typically, network interface 350 is coupled to a computer network as shown. In other embodiments, the network interface 350 may be physically integrated on the motherboard of the computer 320, or may be a software program such as soft DSL.

  Computer 320 typically includes well-known computer components such as processor 360, memory storage devices such as random access memory (RAM) 370, disk drive 380, and system bus 390 interconnecting these components. In one embodiment, computer 320 is a PC compatible computer having a plurality of microprocessors, such as a Xeon ™ microprocessor commercially available from Intel.

  RAM 370 and disk drive 380 are examples of tangible media for storing rules, program code, content that can be used for broadcast message and pulse details, and for storing analysis results and message aggregation data. The storage device may also include other computer programs, operating system code, embodiments described in the present application, program code configured to instruct the computer system 300 to perform the functions described above, and the like. Other types of tangible media include floppy (registered trademark) disks, removable hard disks, CD-ROMs, DVDs, Blu-Ray disks and other optical storage media, flash memory, read-only memory (ROM), battery-type volatile memory Semiconductor memory, and network-connected storage devices.

  In the present embodiment, the computer system 300 may include software that enables communication via a network, such as an HTTP protocol, a TCP / IP protocol, and an RTP / RTSP protocol. In other embodiments, other communication software and transfer protocols such as, for example, IPX or UDP may be used.

  FIG. 3 is a representative example of a computer system that can embody aspects of the teachings of the present application. It will be readily apparent to those skilled in the art that many other hardware and software configurations are suitable for use with the embodiments described herein. For example, the use of other microprocessors such as Core ™ or Itanium ™ microprocessors and Opteron ™ or Phenom ™ microprocessors commercially available from Advanced Micro Devices, is contemplated. Further, Windows Vista (registered trademark) or Windows NT (registered trademark) commercially available from Microsoft Corporation, LINUX (registered trademark) commercially available from Sun Microsystems, Apple Inc., UNIX (registered trademark), etc. Trademark) and other types of operating systems such as MAC OS are contemplated.

  FIG. 4 is a block diagram illustrating the operation of various processing components that process message streams. As shown here, messages from various sources (as described elsewhere herein) are provided to pipeline 402. Within the pipeline 402, a filter check process 404, a demographic tagger 406, a message type determiner 408, and an aggregator 410 act on the message. The output of the aggregator 410 is provided to the analyzer / selector 412 which outputs instructions 414 regarding the action to be taken. An example of an instruction may be an electronic alert that shows a media director with statistics of a response to a pulse event.

  The filter check process 404 can be program code that compares message / search / URL elements (text, content, metadata, timestamps, etc.) with a filter rule set 420 stored in electronic form. Rule 422 (1) to filter out messages that are time stamped before the time stamp of the pulse (because the causal relationship indicates that it is not a response to the pulse), and filter out messages that contain the word "movie" Some example rules 422 are shown, such as rule 422 (2). The rules can be applied in various orders, or can be independent of the order. Rules can also be combined in an “and”, “or”, and / or “xor” manner. Weights can be assigned by rules, where the weights of the combined rules form a score for certainty as to whether the message is a response to a pulse.

  Although one message is shown as an example only, more messages may exist. This message has the text “I like this movie” and has a time stamp of 1:38 pm. This message may or may not be related to a movie advertisement made at 1:30 pm. Of course, this movie ad was about Disney / Pixar's “Cars 2” that was broadcast at 1:30 pm in the San Francisco Bay Area, and the message was from a social network user who admits that he lives in San Francisco, California. And the text of the message is "Oh! I just watched the trailer of Cars2 on TV, but I want to watch this movie soon (Wow! I just saw the Cars 2 trailer on TV and cant wait to see the movie)" If there is a filter check process 404, (the broadcast area and the user demographic area overlap, the time stamp is close in time, “saw” and “t” are included in the message. find a match with some "filter extraction" rules (such as "railer") and there is a match of "Cars 2" between the topic of the ad and the text in the message, This message will be heavily weighted in the direction that the message is a response to this advertisement.

  Demographic tagger 406 compares message / search / URL elements (text, content, metadata, timestamps, etc.) with demographic rule set 424 to generate a demographic indication or estimate for a given message. These instructions or estimates can then be attached or associated with the respective message to provide program code that allows downstream processing to consider this information.

  The message type determiner 408 determines from the available information an estimate whether the message is human or mechanically generated (eg by a computer), and downstream processing follows the message accordingly. Can be weighted.

  In this way, the message stream is intensively analyzed to identify possible responses to responses to pulse events such as advertisements.

  One skilled in the art can deduce further embodiments after reading this disclosure. In other embodiments, combinations or partial combinations can be advantageously created. It should be understood that example arrangements of components are given by way of example, and that other embodiments are intended to be combined, added, rearranged, and the like. Thus, while this application describes exemplary embodiments, those skilled in the art will recognize that many modifications are possible.

For example, the processes described herein can be implemented using hardware components, software components, and / or any combination thereof. The specification and drawings are, accordingly, to be regarded in an illustrative sense rather than a restrictive sense. However, various modifications and changes may be made to the invention without departing from the broader spirit and scope of the invention as set forth in the claims, and the invention is intended to be within the scope of the following claims. It will be apparent that it is intended to cover modifications and equivalents of

Claims (24)

A method for measuring advertising effectiveness in a system in which a user at a plurality of nodes uses a computer device to initiate a message flowing between at least one of the plurality of nodes, the method comprising:
Monitoring at least one aggregate activity stream representing messages sent by the collection of users forming a statistical group for which the advertising effectiveness is measured after a broadcast time of a broadcast message;
Filtering the at least one aggregate activity stream to identify messages that are likely to be responses to the broadcast message, the filtering of the at least one aggregate activity stream comprising: Including identification of which messages are from non-mechanical sources, the mechanical sources are computer-automated, and the non-mechanical sources are human for broadcast messages A source comprising a response or reaction of
Classifying the identified messages into a plurality of groups and classifying the identified messages into groups based on at least a portion of a word pattern or phrase associated with the author;
Identifying a demographic for each of the plurality of groups and for each demographic;
Identifying a preference based on the identified message in one or more groups corresponding to the demographic;
A method comprising the steps of: The steps to monitor are
Monitoring at least one social network after the broadcast time, or monitoring at least one search engine query stream after the broadcast time, wherein the at least one aggregate activity stream is the at least one social Including messages that are monitored in the network and identified in social net traffic, or messages that are the identified search queries by monitoring the at least one search engine query stream;
The method according to claim 1.   The at least one aggregate activity stream includes a message that is a search query, and monitoring includes monitoring at least one search engine query stream after the broadcast time. The method described. Filtering the at least one aggregate activity stream to identify a message that is likely to be a response to the broadcast message;
Identifying a start time for each of the messages;
Filtering each of the messages based on identifying that the start time of the message occurs before the broadcast time;
The method of claim 1, comprising:   Further comprising assigning a first weight value to the identified message from a mechanical source and assigning a second weight value to the identified message from a non-mechanical source, The method of claim 1, wherein a weight value of is less than the second weight value. The first weight value is different from the second weight value, and the demographic is based at least in part on the content of the identified message generated non-mechanically,
The method of claim 5.   The method of claim 5, wherein the preference is further specified based on a weight value assigned to the identified message in the one or more groups. Identifying which messages are from mechanical sources and which messages are from non-mechanical sources
Inferring whether the identified message is from a mechanical source or a non-mechanical source;
Assigning a weight value to the identified message, depending on the estimated source of the identified message;
The method of claim 1 comprising: The message from the mechanical source includes a Short Message System (SMS) message retransmitted after the broadcast time, or a social network message retransmitted after the broadcast time.
The method of claim 1. The message from the mechanical source is a mechanically generated message that was previously transmitted at a previous time and retransmitted after the broadcast time;
The message from the non-mechanical source includes a message that is transmitted after the broadcast time and is based on a human response from observation of the broadcast message.
The method of claim 1. Filtering the identified message includes parsing the identified message into words and breaking it into a plurality of words, weighting each of the specific words based at least in part on the frequency of occurrence of the specific words. Including assigning values,
The method of claim 1. Identifying which messages are from mechanical sources includes identifying which messages have similar phrases,
The method of claim 1. The identified messages are further classified into groups based on a word distribution of a set of words associated with an author corresponding to the group.
The method of claim 1. Generating multiple seed terms,
Further comprising calculating a calculated value indicating the proximity of the identified message to the plurality of seed terms by comparing the identified message with the plurality of seed terms;
A weight value is assigned to each identified message based on the calculated value for each identified message.
The method of claim 1.   The method of claim 1, wherein the broadcast message has a first format type that is different from a second format type of each of the messages that represents at least one of the aggregate activity streams. An ad management system,
A user comprising a server computer for monitoring at least one aggregate activity stream after a broadcast time of a broadcast message, wherein the aggregate activity stream is transmitted by a collection of users forming a statistical group for measuring advertising effectiveness Represents the message,
A message type analyzer that filters at least one aggregate activity stream to analyze the user messages to be monitored and identify user messages that are likely to be responses to the broadcast messages, the at least one Aggregate activity stream filtering includes identifying which user messages are from mechanical sources and which user messages are from non-mechanical sources, which are automated by the computer. The non-mechanical source is a source that includes a human response or response to a broadcast message, and the message type analyzer classifies the identified user messages into a plurality of groups and identifies the identified user messages. At least a portion of di, the word pattern or phrase associated with the author, and classified into groups based,
Including a message demographic analyzer for identifying demographics for each of the plurality of groups and the respective demographics, and identifying a preference based on user messages identified in one or more groups corresponding to the demographics ,
An advertisement management system characterized by that.   The advertisement management system according to claim 16, wherein the broadcast message is an advertisement transmitted through one or more of a television advertisement, a radio advertisement, a bulletin board, and an Internet advertisement medium.   The advertisement management system of claim 16, wherein the user message includes a social media message, a search query, a request for one or more web assets, or a combination thereof.   The request for one or more web assets includes a request for one or more web assets identified by URLs stored in a monitored URL list, and at least one of the monitored URLs The advertisement management system of claim 18, wherein one relates to at least one of previously broadcast messages.   The previously broadcast message is an advertisement about a movie, and the URL list to be monitored includes a URL of a web page containing information material, promotional material, or a trailer associated with the movie advertisement, The advertisement management system according to claim 19. The message demographic analyzer is:
17. The advertisement management system of claim 16, further configured to at least estimate or identify a demographic for each of the plurality of groups based on message content of each identified user message.   The message type analyzer is further configured to at least estimate or identify which user message is from which of a plurality of source classes, whether the content of the user message or the header of the user message The advertisement management system according to claim 16, wherein a source class of a user message is specified from either or both.   Means for assigning a first weight value to the identified message from a mechanical source and assigning a second weight value to the identified message from a non-mechanical source; The advertisement management system according to claim 16, wherein the first weight value is smaller than the second weight value.   The advertisement management system of claim 23, wherein the preference is further specified based on a weight value assigned to the identified message in one or more groups.

Images