EP3345146A1 - System and method for analyzing user device information - Google Patents
System and method for analyzing user device informationInfo
- Publication number
- EP3345146A1 EP3345146A1 EP16766741.9A EP16766741A EP3345146A1 EP 3345146 A1 EP3345146 A1 EP 3345146A1 EP 16766741 A EP16766741 A EP 16766741A EP 3345146 A1 EP3345146 A1 EP 3345146A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- data
- sighting
- devices
- score
- sighting data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0241—Advertisements
- G06Q30/0251—Targeted advertisements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
Definitions
- different users may use a single computing device, and methods for discerning between different users may be useful. For instance, there may be, within a single household, a number of different users that use the same computing devices. It would be useful to discern between such individuals for the purpose of delivering targeted advertising to these users. For example, one user may be interested in football (e.g., is a fan of the Buffalo Bills), yet another user may not be interested in football at all, and an advertising message targeted to a football fan may not be appropriate or effective in reaching this other user.
- football e.g., is a fan of the Buffalo Bills
- an advertising message targeted to a football fan may not be appropriate or effective in reaching this other user.
- behavioral information e.g., website "signings” of a user, application “sightings,” etc.
- contexts may be clustered together to determine similar contexts. For instance, certain behaviors may be grouped together for a particular device or node or group of nodes within the distributed network as represented by a device graph.
- the device graph includes a representation of devices in the network with connections being formed between the devices, these connections indicating their level of how closely they are related.
- contexts may be propagated to a model of other devices that are determined to be neighbors of the device.
- a correlation matrix e.g., a Pearson correlation matrix
- Such propagated contexts may be used to "train" a scoring engine.
- the system determines relative correlations between systems. It is also appreciated that some highly correlated apps/sites may be fraudulent, activity that is common among platforms and can be attributed to bots, or is at an interest level that is high and therefore is commonly correlated among devices as a result of being popular (and is therefore not a good indication of distinctiveness or similarity). For instance, the site “abc.com” is a commonly accessed site that could be removed as not being distinctive among multiple devices.
- a reduced matrix may be derived.
- a system may use this reduced matrix to multiply feature vectors (e.g., by a dot product) derived from behavior observed at a device to determine a raw score for an edge within the device graph. These raw scores may be mapped to an actual score using bins. The score may be recorded for a particular model of a device graph, indicating that for the particular platform pair, the platforms are related (or not).
- Such information may be used to indicate device use to a user (e.g., for monitoring or other function), may be used to perform some action or series of actions (e.g., determine display(s) of advertising on one or more devices), determine whether certain users are similar or dissimilar to other users, or to perform some other evaluation or indication.
- sighting data from website visits, application use, or other behavioral data is used as a basis to determine similarity between users across multiple devices.
- sighting data is used in association with a device graph to determine similarity.
- sighting data is propagated to other nodes within the device graph to which other nodes are connected.
- Sighting data is compared and propagated to different nodes of the device graph. Such sighting behavior or context among the nodes is grouped together in clusters. Contexts are propagated and joint probabilities are calculated for particular contexts. Attribution matrices are determined for the propagated contexts within the graph. In a scoring procedure, the attribution matrix and determined scope mappings may be used to classify and score future behavioral sighting data. In particular, the system may determine a feature vector based on the behavioral data and multiply that vector by the determined attribution matrix to determine a score that indicates similarity. Such similarity scores may be used to determine how similar or dissimilar particular users are from one another or to identify users that operate across connected devices.
- a marketer may want to show a paid advertisement to someone who is in their target audience.
- Home Depot may want to show an advertisement promoting garden tools that are on sale to someone who actively maintains a garden.
- Home Depot may be forced to show an advertisement only to devices where they can discern the user is a gardener (which may be a subset of that users actual devices) or show an ad to everyone in the household of a gardener (which would potentially target users who have no interest in garden tool).
- a system is provided that has the ability to more precisely target a single user on all of their devices.
- a system that can correctly target the gardener user on a particular device within a household.
- a user may see an ad for the Home Depot promotion on their phone, but ultimately buys the gardening tools on their desktop. Without a way of relating activity between devices by the user, the phone ad is not given any credit for influencing the purchasing decision.
- the phone ad is not given any credit for influencing the purchasing decision.
- a system is provided that provides a true view of the consumer level effectiveness of Home Depot's marketing spend.
- a system comprising a database adapted to store sighting data relating to one or more devices connected in a device graph, and a scoring engine adapted to score one or more edges of the device graph connecting the one or more devices wherein the score of at least one of the one or more edges is determined based upon the sighting data.
- the sighting data includes at least one portion of data identifying a user access to a website.
- the sighting data includes at least one portion of data identifying a user access to an application program.
- the system further comprises a component adapted to train a component of the scoring engine using at least one portion of the sighting data.
- the component adapted to train a component of the scoring engine using the at least one portion of the sighting data is adapted to propagate sighting data between connected nodes within the device graph.
- the system further comprises a component adapted to determine a correlation between connected nodes within the device graph based on the at least one portion of the sighting data.
- the component adapted to determine a correlation is adapted to determine a Pearson correlation matrix based on the propagated sighting data between connected nodes within the device graph.
- the connected nodes within the device graph are different computer platforms.
- the system further comprises a component that is adapted to reduce a volume of sighting data by grouping similar sighting data.
- the system further comprises a component that is adapted to eliminate fraudulent data within the sighting data.
- the scoring engine is adapted to score one or more edges of the device graph connecting the one or more devices responsive to the training of the component of the scoring engine using the at least one portion of the sighting data.
- the system further comprises an interface that indicates the score.
- the system further comprises a component that is adapted to perform an action using a computer system responsive to the determined score.
- a method comprising acts of maintaining, in a memory of a computer system, a device graph including a representation of a plurality of devices connected by one or more edges, storing sighting data relating to the one or more devices connected in the device graph, and scoring, by a scoring engine, one or more edges of the device graph connecting the one or more devices wherein the act of scoring at least one of the one or more edges is determined based upon the sighting data.
- the sighting data includes at least one portion of data identifying a user access to a website.
- the sighting data includes at least one portion of data identifying a user access to an application program.
- the system further comprises an act of training, by a component of the scoring engine, using at least one portion of the sighting data.
- the system further comprises an act of propagating sighting data between connected nodes within the device graph.
- the system further comprises an act of determining, by a correlation component, a correlation between connected nodes within the device graph based on the at least one portion of the sighting data.
- the system further comprises an act of determining, by the correlation component, a Pearson correlation matrix based on the propagated sighting data between connected nodes within the device graph.
- the connected nodes within the device graph are different computer platforms.
- the system further comprises an act of reducing a volume of sighting data responsive to an act of grouping similar sighting data.
- the system further comprises an act of eliminating fraudulent data within the sighting data.
- the system further comprises an act of scoring, but the scoring engine, one or more edges of the device graph connecting the one or more devices responsive to the training of the component of the scoring engine using the at least one portion of the sighting data.
- the system further comprises an act of indicating, by a computer-based interface, the score of the one or more edges of the device graph.
- the system further comprises an act of performing, by a computer system, responsive to the determined score.
- Figure 1 is a block diagram showing a system for analyzing user device information according to various aspects of the present invention
- Figure 2 shows an example process for analyzing user device information according to one embodiment of the present invention
- Figure 3 shows another example process for analyzing user device information according to one embodiment of the present invention
- Figure 4 shows another example process for analyzing user device information according to one embodiment of the present invention
- Figure 5 shows another example process for analyzing user device information according to one embodiment of the present invention
- Figure 6 shows an example device graph that may be used according to one embodiment of the present invention
- Figure 7 shows an example process for merging device identifiers according to one embodiment of the present invention
- Figure 8 shows an example device graph according to one embodiment of the present invention
- Figure 9 shows an example device graph where behavior can be propagated within a device graph according to one embodiment
- Figure 10 shows an example of website and application usage that may be analyzed according to one embodiment of the present invention
- Figure 11 shows a chart that may be created that counts a number of co-occurrences of various website and application accesses according to one embodiment of the present invention
- Figure 12 shows a Pearson matrix for the example according to one embodiment of the present invention
- Figure 13 shows a chart of highly correlated entries that may be removed from the matrix according to one embodiment of the present invention
- Figure 14 shows a chart showing remaining sites of the matrix according to one embodiment of the present invention.
- Figure 15 shows a Pearson matrix for the example with bot sites removed according to one embodiment of the present invention
- Figure 16 shows an example "M" matrix according to one embodiment of the present invention
- Figure 17 shows an example binary feature matrix according to one embodiment of the present invention
- Figure 18 shows an example feature matrix according to one embodiment of the present invention
- Figure 19 shows a normalized feature vector matrix according to one embodiment of the present invention.
- Figure 20 shows a chart that shows a fraction of time a score may win for a particular raw score value according to one embodiment of the present invention
- Figure 21 shows a graph of actual binning and best fit function of that binning according to one embodiment of the present invention
- Figure 22 shows a chart showing true scores for platform pairs according to one embodiment of the present invention
- Figure 23 shows a chart showing random binning sampling with replacement according to one embodiment of the present invention
- Figure 24 shows a sorted version of the chart in Figure 23 according to one embodiment of the present invention
- Figure 25 shows an example bin mapping according to one embodiment of the present invention.
- Figure 26 shows a normalized feature vector according to one embodiment of the present invention.
- systems and methods are provided for analyzing user device information to determine different users among a number of devices.
- user's behavior when using the devices is observed, and responsive to that observed behavior, is the determined whether a particular users are distinguishable from other users.
- it may be useful to distinguish between users that operate the same devices or exist within a similar network e.g., a home network, a workplace network, etc.
- a similar network e.g., a home network, a workplace network, etc.
- Figure 1 shows a distributed system 100 including one or more users 102 that operate one or more devices 103 that execute one or more applications 104.
- users 102 may operate one or more PCs, mobile devices, or other computing device that accesses a communication network (e.g., the Internet).
- Such devices may have one or more applications (e.g., applications 104) that communicate with one or more systems within a distributed system 100.
- applications may include, for example, a browser application, another type of application that communicates with one or more systems, or any other application that can collect behavior (e.g. behavior 105) of the user.
- Such devices may communicate with a system 101 that is capable of observing such behavior.
- System 101 may include one or more processing engines including, but not limited to, scoring engine 106 that is capable of determining a similarity score for particular users.
- System 101 may also include a storage entity 109 on which data may be stored associated with system 101. For instance, system 101 may create and maintain a device graph 107 that identifies, based on collective behavior information, access patterns between devices, users, and other entities.
- Storage 109 may also include segmentation information 108 which identifies particular collections of devices, users and other entities that are recognized from system 101 as being similar.
- Figure 2 shows a process 200 for analyzing user device information according to one embodiment of the present invention.
- process 200 begins.
- the system e.g., system 101
- sighting data may include, for example, application use information by one or more users, website visits within a browser program performed by one or more users, or any other behavioral information that could be collected from a computing device that indicates user behavior.
- a sighting may include, for example, an observation of a device with a given context at a particular time.
- a website sighting may be an observation (e.g., by a system) that a particular device visited a website. For instance, device abc was observed on website www.tapad.com at 13:00 on 2015/01/01.
- An application or app sighting might include a sighting (e.g., by the device or other system) of an app on a device. For example, use of the Yelp app was observed on device abc on 2015/01/01.
- the system selects a device pair for comparison.
- the system determines a similarity between behavioral data based on the sighting data collected. The system then may compute a similarity score between the devices, based on the sighting data at block 205.
- process 200 ends.
- Figure 3 shows a process 300 for analyzing device data according to one embodiment of the present invention.
- process 300 begins.
- the system e.g., system 101
- the system collects sighting data as discussed above with reference to Figure 2.
- the system determines one or more contexts associated with a particular device.
- the contexts may be clustered into groups at block 304.
- the system determines attribution matrices associated with the contexts. Based on the attribution matrices, the system may determine a score at block 306 for a particular device.
- process 300 ends.
- Figure 4 shows a process for analyzing device information according to one embodiment of the present invention.
- process 400 begins.
- the process may include one or more aspects that relate to training a scoring engine (e.g., scoring engine 106).
- the system trains a scoring engine based on sighting information collected for one or more devices.
- the system may create attribution matrices.
- the system creates score mappings which may be used to produce scores based on received data.
- the system may receive behavior data for one or more users at block 405.
- the system may create a feature vector using the attribution matrices determined within the training process.
- the system compares vectors and score mappings to create a raw score based on the behavior.
- the system determines a final score based on score mappings.
- process 400 ends.
- FIG. 5 shows another example process for analyzing device information according to one embodiment of the present invention.
- one or more components of system 101 may perform one or more processes within FIG. 5.
- a training process occurs where sighting data (e.g., app and browser sighting information) is used to train a scoring engine (e.g., engine 106) that can determine, based on future sightings, relatively how positive that behavior on any two devices are related. This information may be used to conclude, for example, that the users exhibit similar behavior, or for two connected devices, whether a particular user uses both devices.
- sighting data e.g., app and browser sighting information
- a scoring engine e.g., engine 106
- This information may be used to conclude, for example, that the users exhibit similar behavior, or for two connected devices, whether a particular user uses both devices.
- sighting data 503 is collected and used to train the system during a training procedure 501.
- fraud detection and context filtering may be performed to remove fraudulent information from the sighting data. For instance, it is appreciated that there may be bots that attempt to cheat an advertisement system by creating fake traffic. To avoid having fraudulent data incorporated into a scoring model, fraud detection techniques may be used to filter out such sighting data.
- One method might also include finding devices and context that are highly clustered, i.e. devices and contexts that are only seen together. These contexts and devices may be then removed from the sighting data.
- the system may perform context clustering. Prior to building a scoring matrix, it might be helpful to cluster context within the sighting data into super contexts. This capability allows for a reduction in dimensionality and may result in a better-behaved scoring matrix. Clustering may be performed using various methods including modularity clustering and/or clustering based on a reduction in the similarity measures such as correlation. In one embodiment, clustering may be an optional step.
- the system determines an attribution matrix M. In general, the attribution matrix is a way to map raw sightings per device into a feature vector.
- Various methods may be used to determine an attribution matrix, including, but not limited to, a Pearson correlation matrix, a Pearson correlation matrix with reduction in dimensionality (SVD), an inverse Cholesky-root of a Pearson covariance matrix, an inverse Cholesky-root of a Pearson covariance matrix with reduction in dimensionality (SVD), OASIS, a modified OASIS, or other method. It is noted that each of the above methods may be used with data that is trained on a per platform pair basis.
- the system calculates final score mappings h which is used in scoring to map a raw score to a final individual score.
- a mapping may be determined based on a "win rate" followed by a uniform discretization.
- the win rate is the rate at which a randomly selected unrelated pair of devices of the same platform pair chosen from the training data would have a raw score that exceeds R.
- a win rate of 50% implies that half of the randomly selected device peers appeared to be better related than the given device pair.
- actual sighting data is used to create a feature vector using the M matrix determined during the training process.
- the system compares to feature vectors to obtain a raw score, and the raw scores map to a final individual score using this scoring mappers determined from mappings h determined during training.
- the scoring process yields one or more scores 512 for device pairs, which indicates the level to which the selected device pairs are related or connected. Such information may be used in association with a device graph as described below with respect to FIGS. 6-9.
- FIG. 6 is a diagram illustrating an exemplary device graph 600, according to one embodiment that may be used to analyze user behavior on a number devices according to various aspects of the present invention.
- the device graph 600 includes six device identifiers, device identifier A, device identifier B, device identifier C, device identifier D, device identifier E, and device identifier F.
- the device identifier may be an identifier used by a device when the device accesses content (e.g., accesses a news website).
- device identifier A may be, for example, a cookie ID for a browser application on a smartphone
- device identifier B is an MAC address for a computing device that has been hashed using the Message Digest-5 (MD-5) hashing function/algorithm
- device identifier C is an iOS® Identifier For Advertisers (IDFA)
- IDFA iOS® Identifier For Advertisers
- device identifier D is a Android® device ID
- device identifier E is a cookie ID for a browser application on a desktop computer
- device identifier F is a cookie ID for a browser application on an Android® device.
- a device identifier may represent a device that associated with the device identifier (e.g., may represent the device that is using the device identifier).
- a device may be a computing device and/or an application, software, software modules, and/or other component on the computing device.
- the device may be one or more of a desktop computer, a laptop computer, a server computer, a PDA, smartphone, web-enabled television set, smart television set, a gaming console, and/or any other device capable of processing, managing and/or transmitting data.
- the device may be software, a software module, an application, and/or other component on a computing device.
- the device may be one or more of a web browser, an application (e.g., a mail application or a social networking application installed on smartphone ), a game application, a media player application (e.g., a video player or a music player), a software module (e.g., an add-on or a component for a browser application), etc.
- an application e.g., a mail application or a social networking application installed on smartphone
- a game application e.g., a game application, a media player application (e.g., a video player or a music player), a software module (e.g., an add-on or a component for a browser application), etc.
- a software module e.g., an add-on or a component for a browser application
- Each device may have one or more device identifiers.
- a smartphone e.g., a device
- a serial number e.g., a serial number from a manufacturer an Open Device Identification Number (ODIN), a Unique Device Identifier (UDID), an OpenUDID, a Globally Unique Identifier (GUID), an IMEI number, an IP address, etc.
- applications, software, and/or software modules may also have device identifiers.
- an application on a computing device may have a serial number which may be the device identifier for the application.
- a web browser application may have cookie which includes an identifier and the identifier in the cookie (e.g., the cookie ID) may be the device identifier for the web browser application.
- an application may use different types of device identifiers (e.g., a GUID, a UUID, etc.).
- device identifiers may include, but are not limited to, IP addresses, a MAC addresses, IMEI numbers, serial numbers, ODINs, UDIDs, OpenUDIDs, GUIDs, cookie IDs, an iOS® IDFA, an Identifier for Vendors (IDFV), and/or any other data/information which may be used to identify a device (e.g., an application, software, and/or a computing device).
- a device identifier may be a number (e.g., 734598238742), an alphanumeric value (e.g., A984FDSJL334), a string of characters (e.g., HZ$98!324* 1), or any type of value that may be used to identify a device (e.g., an application, software, and/or a computing device).
- a device e.g., a computing device, an application, software, a software module, etc.
- the application e.g., a device
- the application may generate a device identifier based on an address associated with the computing device.
- a computing device e.g., a device, such as a smartphone
- the smartphone may generate a device identifier based on an IMEI number or a UDID for the smartphone.
- the device e.g., a computing device, an application, etc.
- an application on a computing device may use a cryptographic hash function (e.g., Secure HashAlgorithm- 1 (SHA-1), Secure Hash Algorithm-2 (SHA-2), Message Digest-5 (MD-5), etc.) to generate a device identifier for the application based on an IMEI for the computing device.
- a computing device e.g., a tablet computer
- a random number generator e.g., a Pseudo-Random Number Generator (PRNG)
- the device identifier A is connected to device identifier B via connection 602, to device identifier C via connection 601, and to device identifier D via connection 605.
- the device identifier B is connected to the device identifier C via connection 606 and to device identifier D via connection 606.
- the device identifier C is connected to the device identifier D via connection 604.
- Device identifier E and device identifier F are not connected to other device identifiers.
- Each of connections 601-606 has a weight associated with the connection.
- Connection 602 has a weight of 5, connection 601 has a weight of 7, connection 603 has a weight of 3, connection 604 does not include a weight, connection 605 has a weight of 1, and connection 606 has a weight of 5.
- connections in the device graph 600 may be identified based on (i) network data (e.g., identified based on whether device identifiers used the same network/network gateway), (ii) login information (e.g., based on whether two devices for two device identifiers used the same login information to access content from a partner such as a publisher, an advertiser, etc.), and/or (iii) both network data and login information.
- network data e.g., identified based on whether device identifiers used the same network/network gateway
- login information e.g., based on whether two devices for two device identifiers used the same login information to access content from a partner such as a publisher, an advertiser, etc.
- the weight of the connection may be indicative of how closely or how strongly two device identifiers are associated with each other.
- the weight for each connection may be a numeric value (e.g., a number). A higher numeric value may indicate that two device identifiers are closely associated with each other and a lower numeric value may indicate that two device identifiers are less closely associated with each other. In one embodiment, a closer association between two device identifiers may indicate that the two device identifiers originate from the same computing device.
- a first device identifier may be an identifier for a computing device (e.g., a smartphone) and a second device identifier may be for an application (e.g., a web browser) that is used on the computing device.
- a first device identifier may be an identifier for a first application (e.g., a web browser) on a computing device and the second device identifier may be for a second application (e.g., a game or social networking application) on the same computing device.
- a closer association between two device identifiers may indicate that the two device identifiers originate from computing devices that are used by the same user, or that are used by users who know each other (e.g., members of a household, friends, etc.).
- the first device identifier may be for a first computing device (e.g., a smartphone) used by a user and the second device identifier may be for a second computing device (e.g., a laptop computer) used by the same user.
- the first device identifier may be for a first computing device (e.g., a tablet computer) used by a user and the second device identifier may be for a second computing device (e.g., a laptop computer) used by a family member (e.g., husband, wife brother, sister, son, daughter, etc.) of the user.
- a first computing device e.g., a tablet computer
- the second device identifier may be for a second computing device (e.g., a laptop computer) used by a family member (e.g., husband, wife brother, sister, son, daughter, etc.) of the user.
- the connections 601-606 may also include other information in addition to the weights.
- the connection 604 includes information (e.g., the text/value "login") indicating that the connection 604 was established because the device associated with device identifier C and the device associated with device identifier D used the same login information (e.g., same username/password) to access content from a partner (e.g., publisher, advertiser).
- a partner e.g., publisher, advertiser
- the connection 605 includes information (e.g., the text/value "login”) indicating that the connection 605 was identified because the device for device identifier A and the device for device identifier D used the same login information to access content from a publisher (e.g., used the same username/password to access an email account), and information (e.g., the weight " 1") indicating that the connection 605 was also identified because device identifier A and device identifier D used the same network gateway (e.g., accessed the publisher service via the same Wi-Fi network).
- the connection may also include information indicating the name of the partner (e.g., a publisher, an advertiser, etc.) that provided the login information.
- connection may include the text/value "login-Gmail®” indicating that the login information was received from Google®, or the connection may include the text/value "login/Pandora®” indicating that the login information was received from Pandora®.
- connection may also include information indicating where the network data (that was used to identify the connection) was obtained from (e.g., a merchant server, a content server, an advertisement server, a network data server, etc.).
- strings or text are illustrated in FIG. 6 to represent the additional information, in other embodiments, the additional information (e.g., information indicating how a collection was identified) may be represented using other strings, numbers, alphanumeric values, and/or other representations.
- the text "network" may be included in a connection to indicate that the connection was identified because two device identifiers used the same network.
- the connections between the device identifiers may not include weights. For example, when a connection between two device identifiers is identified using login information, the connection may not include a weight.
- connections 601-606, and the weights for the connections are determined or identified using network data and/or login information (e.g., username and/or password) that is used by different devices to access a partner's (e.g., publisher's, advertiser's) website.
- network data and/or login information e.g., username and/or password
- partner's e.g., publisher's, advertiser's
- the network data may include a device identifier (e.g., an iOS® IDFA, a Android® device ID, an identifier in a cookie, etc.) for a device (e.g., a computing device and/or an application), a type of a network used by the device (e.g., whether the network is a residential/home network, or a commercial/business/corporate network), an identifier for the network (e.g., an IP address for the network, such as the IP address for a Wi-Fi router) that is used by the device, a type for the content that was accessed by the computing device and/or application (e.g., news content, music content, video content, etc.), an identifier for the servers that host the content (e.g., an IP address or a domain name for the servers), user agent information (e.g., the type of a browser, such as Mozilla®, Safari®, Firefox®, Chrome®, Internet Explorer®, etc.), website header information (e.g., HTTP headers
- the network data may be generated, obtained and/or inferred by a server (e.g., tracking server 305, network data server 325, etc., as shown in FIG. 3) based on other data (e.g., based on other network data).
- a server e.g., tracking server 305, network data server 325, etc., as shown in FIG. 3
- the server may be able to determine the type of the network (e.g., whether the network is a residential or a commercial network) by performing an IP address lookup.
- the servers that host or store the content (e.g., websites, multimedia content, etc.) that is accessed by the devices may obtain and/or record network data when the device accesses the content.
- a server that hosts a news website may obtain a cookie ID for a web browser on a smartphone.
- a server that hosts social networking content may obtain an IP address for a tablet computer that accesses the social networking content.
- these servers may provide the network data (e.g., device identifiers, network types, identifiers for the servers, etc.) to an identifier module.
- the identifier module may obtain and/or create a device graph (such as device graph 600) based on network data received from the servers.
- the servers may provide the network data to a network data server (e.g., a server for an online data vendor/partner which collects and stores the network).
- a network data server e.g., a server for an online data vendor/partner which collects and stores the network.
- the network data server may collect network data from multiple servers that host different content.
- one or more of a content server, a merchant server, and an advertisement server may provide the network data to the identifier module.
- the network data server may provide the network data to the identifier module.
- the identifier module may obtain and/or create a device graph (such as device graph 600) based on network data received from the servers.
- an application/software on a computing may obtain device identifiers (e.g., a MAC address, an IP address, an IMEI number, a UDID, among others) and may provide these identifiers to the identifier module.
- a game application that is installed on a smartphone may obtain the iOS® ID FA for the smartphone and may provide the iOS® ID FA to the identifier module.
- the identifier module may obtain login information and network data from different servers.
- the identifier module may identify connections between different device identifiers, based on the login information. For example, the identifier module may obtain hashed login information that was used to access an email server.
- the identifier module may determine that two different devices (each with different device identifiers) used the same login information (e.g., same username and/or password) to access an email account.
- the identifier module may create a connection between the two device identifiers (e.g., between the two devices) because the two devices associated with the two device identifiers used the same login information to access the email account.
- the weights for the connections 601-606, and the weights may be increased or decreased based on different criteria, rules, factors, and/or conditions.
- the weight for the connection 602 may be increased if device identifier A and device identifier B are seen on the same residential network within a period of time (e.g., seven days).
- the weight for connection 606 may be decreased of device identifier B and device identifier C have not been seen on the same residential network for a period of time (e.g., two weeks).
- the identifier module may increase and/or decrease the weights for the connections 601-606 based on the network data received from the servers and/or the network data server.
- device identifiers may be removed from the device graph 600.
- device identifier E is not connected to other device identifiers. After a period of time (e.g., 30 days, 7 days, etc.), the device identifier E may be removed from the device graph 600 if the device identifier E is still not connected to other identifiers or the device identifier has not been refreshed in a threshold period of time.
- device identifiers may be added to the device graph 600. For example, a new device identifier may be added to the device graph and a new connection may connect the new device identifier to device identifier F (not shown in the figures).
- a server may manage, update, and/or track all device identifiers (e.g., all identifiers for computing devices, software, application, etc.) and the connections between all the device identifiers using a single device graph. Because all device identifiers may be included in a single device graph, the single device graph may include disjointed groups of connected device identifiers and single device identifiers that have no connections. In another embodiment, there may be separate device graphs that contain different device identifiers and connections.
- FIG. 7 is a diagram illustrating an exemplary device graph 700, according to another embodiment.
- the device graph 700 also includes device identifier A, device identifier B, device identifier C, device identifier D, device identifier E, device identifier F, and device identifier G.
- Device identifier A is a cookie ID for a browser application on a smartphone
- device identifier B is an address for a computing device (e.g., a MAC or other type of address) that has been hashed using the Message Digest-5 (MD-5) hashing function/algorithm
- device identifier C is an iOS® Identifier For Advertisers (IDFA)
- device identifier D is a Android® device ID
- device identifier E is a cookie ID for a browser application on a desktop computer
- device identifier F is a cookie ID for a browser application on an Android® device
- device identifier G is an IMEI for a computing device that has been hashed using the Secure Hashing Algorithm-1 (SHA-1).
- the graph 700 also includes a merged identifier ABC.
- the merged identifier ABC aggregates device identifier A, device identifier B, and device identifier C, so that they are equivalent.
- Merged identifier ABC is connected to device identifier D via connection 701 and connection 701 has a weight of 4.
- Device identifier D is connected to device identifier G via connection 702 and connection 702 has a weight of 4.
- Device identifier F is connected to device identifier G via connection 703 and the connection 703 has a weight of 2.
- the connection 703 also includes information (e.g., the text/value "login") indicating that the connection 703 was identified because both the device identifier F and the device identifier G accessed content (e.g., a website, a service, etc.) using the same login information (e.g., same username/pas sword).
- device identifiers may be merged if the device identifiers originate from the same computing device.
- device identifier A may be an identifier for the computing device's operating system (e.g., an iOS® IDFA)
- device identifier B may be an identifier for a web browser (e.g., a cookie ID)
- device identifier C may be an identifier for another application (e.g., a game application) on the computing device (e.g., a serial number for a second application).
- One of the applications e.g., the game application
- One of the applications may obtain different identifiers for the computing device and/or the applications on the computing device.
- the game application may obtain the IMEI number, the iOS® IDFA, and a cookie ID.
- the game application may provide these identifiers to the identifier module and may indicate that these three identifiers originate from the same computing device and/or user.
- the identifier module may merge the different identifiers in a device graph, based on this information.
- the merged device identifiers are stored as being equivalent to each other. For example, when the identifier module accesses device graph 700 to find device identifier A, the device identifier A is stored in the device graph 700 as being equivalent to device identifier B and/or device identifier C.
- device identifier may be merged when cookie syncing is performed.
- an advertiser or a publisher may implement cookie syncing between an advertisement exchange (e.g., an ad exchange where bids for advertisements are processed).
- the advertiser/publisher may use a first cookie ID (e.g., cookie IDA) and the advertisement exchange may use a second cookie ID (e.g., cookie ID B).
- the identifier module may merge the first cookie ID and the second cookie ID because the publisher and the advertisement exchange have synced the two cookie IDs (e.g., both cookie IDs belong to the same device).
- device identifiers may not be merged even though the device identifiers are associated with the same login information or user credentials.
- the identifier module may identify a connection between two device identifiers because the two device identifiers are associated with the same login information (e.g., same user credentials).
- the identifier module may not merge the two device identifiers, but may indicate in the connection (e.g., in the extra information included in the connection) that the connection was identified based on login information.
- device identifiers may be merged if the device identifiers are associated with the same user credentials for a server.
- a user may use desktop computer at work to log into an email website.
- the desktop computer may have the device identifier A.
- the user may also use an email application on a smartphone to access the same email website when the user is away from a computer.
- the email application may have the device identifier B.
- the user may also use a web browser on a tablet computer to access the email website when the user is at home (e.g., at the user's residence).
- the table computer may have the device identifier C.
- the server that hosts the email website may provide network data indicating that device identifiers A, B, and C, are all associated with the same user credentials.
- the device identifiers A, B, and C may be merged or combined into the merged identifier ABC, based on the network data.
- devices may not be merged if different users utilize the same device (e.g., a shared family PC, gaming system, etc.). In this case, it may be useful to discern which user is operating a common device at a specific instance.
- the connections and the weights for the multiple device identifiers may also be merged or combined.
- device identifier A is connected to device identifier D via connection 605 that has a weight of 1
- device identifier B is connected to device identifier D via connection 603 that has a weight of 3
- device identifier C is connected to device identifier D via connection 604 that was established using login information.
- the merged identifier ABC includes device identifiers A, B, and C.
- the merged identifier ABC is connected to device identifier D via connection 701.
- the connection 701 has a weight 4, which is the sum of the weights for connections 603-605 shown in FIG. 6.
- the weights for connections that have been merged due to the merging of device identifiers may be obtained using other methods. For example, instead of summing the weights of the merged connections, the average weight of the merged connections may be used.
- FIGS. 6 and 7 illustrate device identifiers, connections between the device identifiers, and weights for the connections using a graph
- this information may be represented using different representations and/or data structures.
- the identifiers, connections, and weights for the connections illustrated in FIG. 7 may be represented using a table.
- the device identifiers, connections, and weights for the connections may be represented using other data structures and/or representations, such as graphs, tables, charts, trees, etc.
- one or more device graphs may be used to determine whether certain behavior by user is more likely to be the same user among multiple connected devices. For instance, as shown in FIG. 8, each square shown may be used to represent a device or "node" in the device graph. For instance, according to one model, each device or node in the graph includes several pieces of data associated with it. For instance, each node may be identified by what type of platform it is, what site(s) the note has visited as well as its neighbors within the device graph. For example, nodes "1" and "2" might have information associated with them such as outlined in the following example:
- Websites/Apps Visited My Weather Android App, MyFavoriteAndroidGameApp Neighbors: Nodes 1, 3, and 4
- the process may consider edges in the graph of one platform type. For example, here we will only consider (in the toy example) iPhone and AndroidT ablet edges (that means iPhones connected to AndroidT ablets and vice versa but no other devices). This training process is repeated for each platform pair in turn and new matrices/mapping functions are created for each platform pair until the final step.
- behavior information is propagated to connected neighboring nodes.
- the process may "propagate" the websites and applications visited to the neighbors (again, considering only edges that are consistent with our chosen platform pair). It is appreciated that because there is generally very little overlap between App based devices (devices that only use apps like many mobile devices) and Browser based devices (devices that only go to websites like computers), one assumption is that by using propagation, the process will add more signal than noise. That is, it is appreciated that devices connected in the graph are more likely to belong to the same user than random devices not connected in the graph.
- Websites/App Visited My Weather Android App, MyFavoriteAndroidGameApp, (+any new websites/apps from abc.com devices 3 and/or 4 if 3 and/or 4 are Android Tablets) Neighbors: Nodes 2, 3, and 4.
- Websites/Apps Visited My Weather Android App, MyFavoriteAndroidGameApp, abc.com, (+any new websites/apps from devices 3 and/or 4 if 3 and/or 4 are iPhones )
- Figure 9 shows how the websites/apps could potentially propagate along the whole example graph. Including the propagated websites/apps, simple counts may be recorded of how many devices contain each website/app (note that in this example, multiple sitings of the same site/app on a single device is not considered). Therefore, if a user has been to abc.com 50 times, the process will still only feature one occurrence in the statistics.
- the following counts may be observed by one or more computer systems as shown in Figure 10. For instance, a number or count of sightings may be collected from multiple sites/apps and may be analyzed. As shown in Figure 10, the node has various counts of visiting abc.com, MyFavorite Android App, MyFavoriteAndroidGameApp, def.com, xyz.net, awesomeSite.com, 123.com and bestSite.net. The process may then compute how many times each pair occurs together on a device with such propagated contexts (the data is already starting to look like a matrix) as shown by way of example in Figure 11.
- the correlation may be computed between them (e.g., by determining a Pearson Correlation). Assume for the example that there is a need to compute the correlation between context "A” and "B". Where P(A) is the probability a device has context "A”, P(B) is the probability a device has context B and P(A,B) is the joint probability of A,B meaning that the probability a device has context A and B.
- MyFavoriteAndroidGameApp MyWeal, JwrAndroklAp , yFavmiteAiKliOidGamoApp
- the Full Pearson Matrix for the above co-occurrence counts are as shown by way of example in Figure 12. Then, the values may be determined that represent the highest correlations, and these highly-correlated entries may be flagged as possible bot correlations. This may be
- the process could flag one or more sites as possible botnet sites as shown in Figure 13.
- the system may then remove any pair that contains a website that is known to be popular (e.g., sites that are listed in the Alexa top 10,000 rankings (e.g., as found at http://s3.amazonaws.com/alexa-static/top-lm.csv.zip).
- sites may be removed, as they are not distinctive among multiple users because of their 5 popularity. For example, imagine that the site abc.com is a very common site. The system may be adapted to determine this and could remove the entry. This would leave the sites shown in Figure 14.
- the system may then cluster the remaining ones together using the Pearson correlation as an edge weight. All remaining flagged websites that are in a group equal to or larger than n (e.g., three (3) in the example) are flagged as bots and removed from consideration. This means that pairs which just have a high correlation between them are not flagged as anomalous, only large groups of interconnected devices.
- "modularity" clustering may be used to cluster contexts using "community detection”. Here, one such cluster exists which consists of sites def.com, 123.com, and xyz.net, and this cluster is removed from the subset. In one implementation, these websites are flagged as fraud/bots and are not considered in scoring.
- a(i,j) is a coefficient which maps feature i onto feature j.
- This mapping means instead of doing 20k*20k computations, the system may perform 20k* lk computations. This reduces the matrix algebra to l/20th of the previous complexity, which is a big savings in time and processing when data is large. If all the features in the space are independent, then in principle, all of the components are needed to full specify the space. However, it is appreciated that some features/combination of features are much more important than others. A Singular Value Decomposition (SVD) is a good choose for such reduction.
- This algorithm may be used by the system to find the "k” most important new features by decomposing the matrix into eigenvectors and eigenvalues. The "k” eigenvectors with the k largest eigvenalues are kept.
- a second training set of edges (again of a single platform pair type - same as considered above - iPhone/AndroidT ablet) may be used to obtain a win rate function, h.
- a win rate function h.
- Websites/ Apps Visited My Weather Android App, MyFavoriteAndroidGameApp Neighbors: Nodes 1, 3, and 4
- each device will have a binary feature vector assigned to it - in this calculation step, the context is not propagated to other nodes. Rather, propagation of contexts between connected nodes is used in the training environment.
- Binary feature vectors are used to characterize observed behavior. For instance, the entry for that element will be 0 if the device did not visit that site or app, and 1.0 if it visited it any number of times. For node 1 and 2 this would resemble the diagram shown in Figure 17. These vectors are then multiplied by the dimensionality reduced M matrix to yield the matrix shown in Figure 18.
- the system may then determine the dot product of these normalized vectors. In this case:
- This score may be referred to as a "true” raw score because it is a score between a pair of devices which are connected in the graph - a true positive that indicates a connection. Now, the win rate for this value may be determined. We inspect more edges that have the same platform pair (they connect an iPhone and an Android Tablet in this case).
- edges For each of these edges, two false edges are created. For instance, taking nodel (the iPhone) and compute a score between it and the Android Tablet of edge i (which is not connected to nodel). The process does the same procedure with node2 and the iPhone of edge i. This may be referred to as a "false" score because it is a score between two devices which are not connected in the graph (this score corresponds to a false positive).
- this score is greater than or less than the value of 0.63 as determined above. If the value is less than 0.63 then the algorithm "wins” this round. That is, the algorithm has scored a true positive higher than a false positive. If the value is greater than 0.63 then the algorithm "loses" this round.
- the system may construct a histogram and determine for each value of the true, raw score, what is the fraction of time that that score will "win". If the score is very high (e.g., close to 1.0) then the algorithm will "win” most of the time and the win rate will be close to a value of 1. If the score is very low, then the algorithm will "lose” most of the time and the win rate will be close to 0. However, in the middle, there may a place where the algorithm wins as often as it loses (basically a coin flip) and the win rate is around 0.5 - this is the system cannot distinguish a True edge from a False edge. For example, the binning might look like this as shown by way of example in Figure 20. This binning can be fit to a logistic function of the form:
- the f mapping may be computed and the process considers all platform pairs together again.
- the training data used to fit the function h is then used to "bin" the data.
- all of the raw scores are converted into h scores by using the a & b parameters for each platform pair to convert the raw score into an h score.
- the bins may be computed as follows:
- Each comparison has the following data: h(raw score I platformPair) (e.g., the raw score is what was referred to as the "true raw score” above)
- An equal number of comparisons may be performed to train the h's as shown by way of example in Figure 22.
- scoring may include the M matrix, the parameters that map raw score to an h score, and an f mapping that maps an h (win rate) score into an approximate bin.
- an SVD- reduced M Matrix may be determined for each platform pair and a dictionary indicating which index belong to which feature to perform the matrix multiplication/dot product.
- fit parameters a, b are determined for each platform pair to map raw score to an h score as discussed above.
- an f mapping of the h (win rate) score onto the final equal- mass approximate bin is determined.
- the two feature vectors are compared to obtain a raw score, for example, by taking the dot product of these vectors to obtain the raw score.
- the h function for that platform pair is used to convert the raw score into a winRate (h) score.
- h winRate
- the previously defined nodel, node2 example above may be used. Recall that after the feature vectors were created, multiplied each by M, the resultant vectors were normalized, and the dot product was taken, a raw score of 0.63 was obtained.
- the system may then map the raw score to a final individual score using scoring mappers f. Finally, the system may then use the h to bin mapping function to map that h score onto a bin.
- references to "or” may be construed as inclusive so that any terms described using “or” may indicate any of a single, more than one, and all of the described terms. Any references to front and back, left and right, top and bottom, upper and lower, and vertical and horizontal are intended for convenience of description, not to limit the present systems and methods or their components to any one positional or spatial orientation.
Abstract
Description
Claims
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP21154719.5A EP3848881A1 (en) | 2015-09-01 | 2016-09-01 | System and method for analyzing user device information |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/842,530 US10754913B2 (en) | 2011-11-15 | 2015-09-01 | System and method for analyzing user device information |
PCT/US2016/049930 WO2017040799A1 (en) | 2015-09-01 | 2016-09-01 | System and method for analyzing user device information |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP21154719.5A Division EP3848881A1 (en) | 2015-09-01 | 2016-09-01 | System and method for analyzing user device information |
Publications (1)
Publication Number | Publication Date |
---|---|
EP3345146A1 true EP3345146A1 (en) | 2018-07-11 |
Family
ID=56940395
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP21154719.5A Withdrawn EP3848881A1 (en) | 2015-09-01 | 2016-09-01 | System and method for analyzing user device information |
EP16766741.9A Ceased EP3345146A1 (en) | 2015-09-01 | 2016-09-01 | System and method for analyzing user device information |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP21154719.5A Withdrawn EP3848881A1 (en) | 2015-09-01 | 2016-09-01 | System and method for analyzing user device information |
Country Status (5)
Country | Link |
---|---|
EP (2) | EP3848881A1 (en) |
AU (1) | AU2016317920A1 (en) |
CA (1) | CA3034030A1 (en) |
HK (1) | HK1257819A1 (en) |
WO (1) | WO2017040799A1 (en) |
Families Citing this family (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10999298B2 (en) | 2004-03-02 | 2021-05-04 | The 41St Parameter, Inc. | Method and system for identifying users and detecting fraud by use of the internet |
US11301585B2 (en) | 2005-12-16 | 2022-04-12 | The 41St Parameter, Inc. | Methods and apparatus for securely displaying digital images |
US8151327B2 (en) | 2006-03-31 | 2012-04-03 | The 41St Parameter, Inc. | Systems and methods for detection of session tampering and fraud prevention |
US9112850B1 (en) | 2009-03-25 | 2015-08-18 | The 41St Parameter, Inc. | Systems and methods of sharing information through a tag-based consortium |
US10754913B2 (en) | 2011-11-15 | 2020-08-25 | Tapad, Inc. | System and method for analyzing user device information |
US9633201B1 (en) | 2012-03-01 | 2017-04-25 | The 41St Parameter, Inc. | Methods and systems for fraud containment |
US9521551B2 (en) | 2012-03-22 | 2016-12-13 | The 41St Parameter, Inc. | Methods and systems for persistent cross-application mobile device identification |
EP2880619A1 (en) | 2012-08-02 | 2015-06-10 | The 41st Parameter, Inc. | Systems and methods for accessing records via derivative locators |
WO2014078569A1 (en) | 2012-11-14 | 2014-05-22 | The 41St Parameter, Inc. | Systems and methods of global identification |
US10902327B1 (en) | 2013-08-30 | 2021-01-26 | The 41St Parameter, Inc. | System and method for device identification and uniqueness |
US10091312B1 (en) | 2014-10-14 | 2018-10-02 | The 41St Parameter, Inc. | Data structures for intelligently resolving deterministic and probabilistic device identifiers to device profiles and/or groups |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8935249B2 (en) * | 2007-06-26 | 2015-01-13 | Oracle Otc Subsidiary Llc | Visualization of concepts within a collection of information |
US20150142767A1 (en) * | 2010-12-07 | 2015-05-21 | Google Inc. | Scoring authors of social network content |
US20140180811A1 (en) * | 2012-12-22 | 2014-06-26 | Coupons.Com Incorporated | Automatic recommendation of electronic offers to an offer provider based on historical transaction data and offer data |
US9317614B2 (en) * | 2013-07-30 | 2016-04-19 | Facebook, Inc. | Static rankings for search queries on online social networks |
US20150120717A1 (en) * | 2013-10-25 | 2015-04-30 | Marketwire L.P. | Systems and methods for determining influencers in a social data network and ranking data objects based on influencers |
-
2016
- 2016-09-01 EP EP21154719.5A patent/EP3848881A1/en not_active Withdrawn
- 2016-09-01 CA CA3034030A patent/CA3034030A1/en active Pending
- 2016-09-01 AU AU2016317920A patent/AU2016317920A1/en not_active Abandoned
- 2016-09-01 WO PCT/US2016/049930 patent/WO2017040799A1/en active Application Filing
- 2016-09-01 EP EP16766741.9A patent/EP3345146A1/en not_active Ceased
-
2019
- 2019-01-04 HK HK19100170.1A patent/HK1257819A1/en unknown
Also Published As
Publication number | Publication date |
---|---|
CA3034030A1 (en) | 2017-03-09 |
AU2016317920A1 (en) | 2018-04-26 |
WO2017040799A1 (en) | 2017-03-09 |
EP3848881A1 (en) | 2021-07-14 |
HK1257819A1 (en) | 2019-11-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11314838B2 (en) | System and method for analyzing user device information | |
EP3848881A1 (en) | System and method for analyzing user device information | |
Stringhini et al. | Follow the green: growth and dynamics in twitter follower markets | |
TWI684875B (en) | Method for determining user behavior preference, method and device for displaying recommended information | |
JP6092362B2 (en) | How to target stories based on influencer scores | |
AU2010266611B2 (en) | Gathering information about connections in a social networking service | |
JP5635590B2 (en) | Using information in social networks as an insulator to target ads by inference | |
US9485318B1 (en) | System and method for identifying influential social media and providing location-based alerts | |
US10084870B1 (en) | Identifying user segment assignments | |
EP3625748B1 (en) | Distributed node cluster for establishing a digital touchpoint across multiple devices on a digital communications network | |
Lim et al. | A topological approach for detecting twitter communities with common interests | |
Khayyambashi et al. | An approach for detecting profile cloning in online social networks | |
Zhang et al. | You are where you have been: Sybil detection via geo-location analysis in OSNs | |
Nasraoui et al. | Performance of recommendation systems in dynamic streaming environments | |
Guo et al. | Safer: Social capital-based friend recommendation to defend against phishing attacks | |
CN111885011B (en) | Method and system for analyzing and mining safety of service data network | |
CN111125747B (en) | Commodity browsing privacy protection method and system for commercial website user | |
JP2016500457A (en) | Generate metrics based on client device ownership | |
Li et al. | Toward a monopoly botnet market | |
Niazi Torshiz et al. | Collusion-resistant worker selection in social crowdsensing systems | |
Voronin et al. | Crowdturfing on instagram-the influence of profile characteristics on the engagement of others | |
US20140280124A1 (en) | Social Graph Sybils | |
Fallah et al. | Practical detection of click spams using efficient classification-based algorithms | |
Hammed et al. | CLASSIFICATION OF PHISHING ATTACKS IN SOCIAL MEDIA USING ASSOCIATIVE RULE MINING AUGMENTED WITH FIREFLY ALGORITHM | |
US10078844B1 (en) | Method and system for combining offline and online identities with associated purchasing intention indicators in view of a geographic location |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20180518 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20190820 |
|
REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 1257819 Country of ref document: HK |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R003 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN REFUSED |
|
18R | Application refused |
Effective date: 20200922 |