TW201839633A - Method of analyzing a URL to generate a user profile having a profile analysis by using chi-square test and variance analysis techniques to find out significant attributes - Google Patents

Abstract

The invention relates to a method for analyzing a URL to generate a user profile, which mainly selects a plurality of users in a URL browsing log according to an application category, and uses the usage information according to the number, times, or period of browsing of the users to obtain a weight value corresponding to the application category, and then a score of the user's record is calculated based on the weight value, and sorting the same to find the user group with a high usage amount, and performing a profile analysis by using chi-square test and variance analysis of some attributes related to the user group to find out significant attributes among them and depict the user profile of the application category.

Description

 Method of analyzing a URL to generate a user profile  

The present invention relates to a method of generating a user profile by selecting an application category to analyze a method of linking or browsing a profile of a particular URL (Uniform Resource Locator) address.

With the popularization of mobile devices and broadband networks, people's reliance on network convenience has gradually increased. While people have access to a variety of services that are diversified over the Internet, they also generate a large amount of use. Record. From a service provider's point of view, in order to meet the business challenges caused by various potential corporate competitions and changes in customer preferences, there is a need for better ways for companies to provide products and services that better meet customer needs. The marketing method has been shifted from the mass marketing and product differentiation marketing in the past to targeted marketing. In order to achieve such targeted marketing, it is necessary to collect a considerable amount of information for a certain customer. In this network era, customers The record left by using the Internet has undoubtedly become an analytical target with considerable value.

There are several methods in the prior art that effectively use these materials, such as a keyword-oriented method, which can accurately track the behavior on a particular client device and analyze the content of the network resources browsed by the client to capture multiple clients. Information such as content keywords, source and creation time of the website browsing, further establishes the matching relationship between the activities to be marketed and the network keywords.

There are other similar methods in the prior art, which can determine the user according to the weighted average of the user characteristics through the ranking algorithm by analyzing the user characteristics, such as the number of times of browsing the website, the time spent, and the content of the webpage. The combination of features to respond to different outcomes of customers.

However, there is still a lack of technology in the current technology that can mine a significant attribute of a customer for a particular category or attribute to perform a more comprehensive profile on the customer.

In order to make up for the deficiencies of the prior art, the present invention provides a user profile description method combining a huge amount of data and statistical analysis, the main purpose of which is to provide a user with the ability to analyze through defined URLs (Uniform Resource Locators). Profiling the user of a particular domain and the process of describing the characteristics of the profile.

The secondary object of the present invention is to identify whether a user with a higher usage and a remaining user in a specific user domain have statistically significant differences, and such analysis results can be used to construct significant attributes of the user group, such as user basics. Attributes, usage status, interest preferences, service requests, etc. The specific contours of the significant attribute information can be further used as a basis for commercial marketing planning, and can also assist business units to understand the characteristics of the target users. It communicates with customers more effectively to communicate appropriate marketing plans and plan for more appropriate activities.

Based on the above object, the present invention provides a method for analyzing a URL to generate a user profile, which is executable by a profile analysis server capable of obtaining a user's original browsing log, and includes at least the following steps: selecting an application category, including a plurality of A user group is selected in an original browsing log of the user according to the application category. The profile analysis server may be connected to the network to obtain the original browsing log, wherein the application category is a predefined content homogeneous or approximate URL (Uniform Resource Locator) address category, and the application category includes At least one representative URL address having the same or similar content, wherein if the users are recorded in the original browsing log with a URL address that has browsed the application category, the users will be selected into the user group. . In addition, the original browsing log may be structured data stored in a huge data computing technology such as a Hadoop Hive distributed database, and the original browsing log is stored in a decentralized structure according to a certain format, for example, selection basis ("user identification" Codes, "Browse Dates", "Browse URLs") are stored in these formats. This greatly reduces the amount of data that needs to be analyzed, so that the method can be processed by a single machine with statistical computing capabilities.

The usage information of the URL addresses in the application category of the user groups in the user group is analyzed to generate a user record. The usage information includes the Start Date & End Date or Date Duration, the Days Count, the Hours Count, and the number of views of the URLs of the user categories in the application category. Count), the access frequency (Access Frequency), the number of times of each date category, etc., the date category refers to the meaning that the date may represent, for example, a date may belong to the holiday, workday or even false.

The machine training of the variable analysis of the application category is performed by using a URL of a positive or negative example known in the movie application category to obtain a weight value corresponding to the application category. The training of the variable analysis is performed by using the above-mentioned usage information (such as the continuous browsing date, the browsing period, the number of browsing times of each date category, etc.) for generating the user record, through the neural network or logic analysis. (Logistic Regression) model or the like analyzes the input variables to generate the weight values.

Performing a score calculation on the user record according to the weight value, and sorting the calculated scores, and then acquiring at least one user with a high usage amount from the user group according to the sorted score, wherein the at least one user is selected from the user record The scores obtained from the high score to the low score are the users of the percentage ratio (Top N) with higher scores after sorting.

Performing profile analysis on the at least one user to generate a user profile corresponding to the application category, wherein the profile analysis is statistically analyzed according to an attribute previously assigned to the application category to generate at least one significant attribute of the at least one user, and further determining the The at least one significant attribute is related to the at least one user to describe the user profile based on the at least one significant attribute. The attributes that are pre-assigned to the application category may include general user attributes, such as the user's gender, age, education level, etc. In addition, the user may also define the attributes of the jade inquiry, that is, describe the high-volume users. There are attributes such as representative behaviors worthy of attention, for example, if the purpose of the present invention is to find potential high-volume video users, it is possible to define whether the user is a paying user, a bid rate, and a recent bill average. The amount of money, usage data traffic, transmission time, number of roaming base stations, etc.

Wherein, the statistical analysis method used in the foregoing contour analysis may include a Chi-square Test process for performing category variable analysis in the cross-analysis to select χ ² (Chi-square) The value that is significantly larger is output as the at least one significant attribute. The statistical analysis method used in the contour analysis may include an analysis of variance (ANOVA) process for performing continuous variable analysis in the cross-analysis to determine the F-test (F-test). The value that is significantly larger is output as the at least one significant attribute.

The foregoing contour analysis may determine, by using a Logistic Regression model, that the at least one significant attribute is positively or negatively correlated with the at least one user, to find out the influence of the significant attribute on the high-volume user variable. For example, if you find the positive or negative correlation between the age of the user applying for the service and the high-volume user community, you can describe the profile of the high-volume user family that belongs to the prime or younger group. In addition, the model for performing positive and negative correlation judgments may be any current or available model, and is not limited to a logistic regression model.

In addition, the present invention can also perform the above steps again by selecting other application categories to generate user profiles for other application categories.

Through the method of the present invention, the user can select the application categories of the predefined URLs (Uniform Resource Locators) through the profile analysis server, and then analyze the original browsing log to establish a specific user profile, and can distinguish the users with higher usage. Whether there are statistically significant differences with other users on specific attributes.

S100~S300‧‧‧Step process

110‧‧‧Database of the original browsing log

120‧‧‧User records for the video application category

210‧‧‧ Regular and negative data

S220~S230‧‧‧Step procedure

240‧‧‧High-volume user list

S310~S340‧‧‧Step procedure

350‧‧‧ significant attributes

360‧‧‧User profile

1 is a schematic diagram of the main steps of the present invention; FIG. 2 is a schematic diagram showing the detailed steps of the browsing log URL analysis according to the present invention; FIG. 3 is a schematic diagram showing the detailed steps of the score calculation sorting according to the present invention; 5 is a schematic diagram of a detailed step of contour analysis; FIG. 6 is a schematic diagram of a second example of the present invention; FIG. 7 is a schematic diagram of a third example of the present invention; FIG. 8 is a schematic diagram of a fourth example of the present invention; A schematic diagram of the fifth example.

The invention will be further illustrated by the following examples in conjunction with the drawings.

First, please refer to FIG. 1 , which is a schematic flowchart of the main steps of the present invention, which includes three steps, which are S100 browsing log URL analysis, S200 score calculation sorting, and S300 description user profile, wherein, as described in the Summary of the Invention The present invention first selects a user group that matches a specific application category and usage information of the user group from the original browsing log, for example, a continuous browsing date, a browsing time, a browsing period, and the like, and then performs the use according to the step S200. The weight value of the information is calculated and sorted by the users in the user group to find a high-volume user, and then the contour analysis is performed through step S300 to describe the contour of the user conforming to the specific application category, and the user contour can be used. For marketing or event planning for users of a specific application category.

In this embodiment, the original browsing log as the analyzed data includes the connection record data of the plurality of user networks, and the time interval of the original browsing log is from January to February in the week of 2015, and the number of weeks is eight weeks.

The profile analysis server of the method of the present invention can filter out a user group according to a predefined video application category in the original browsing log by Hadoop Hive. In this embodiment, the video application category includes user predefined The method of the present invention is to compare the connection records in the original browsing log that match the first application category. In this embodiment, the amount of original data collected daily is about 500 GB.

Please refer to FIG. 2 , which is a schematic diagram of the detailed steps of browsing the log URL analysis in step S100. According to the above embodiment, according to the user's selection, the representative URL of the video application of the predefined video application category is extracted, for example, for example. The URL of the api.3g.tudou.com, kankan.1kxun.com, etc. of Tudou. The server of the present invention filters out the users who have browsed the URLs from the database 110 of the original browsing log, and Start Date & End Date or Date Duration, Days Count, Hours Count, Access Count, Access Frequency, and date categories for these users The usage information, such as the number of views, is analyzed as an input variable to generate a user record 120 that conforms to the category of the video application.

Please refer to FIG. 3 again, which is a schematic diagram of the detailed steps of the step calculation of the step S200. It is worth noting that before performing this step, the server of the present invention has been executed in advance for a plurality of application categories (including the video application category). Step S220, the weight analysis is to generate a weight for each of the application categories, wherein the weight analysis for the movie application category is to input the positive example and the negative training data 210 corresponding to the movie category into the neuron. The Neural Network model is trained in a single layer model. The output model will contain the weights of the aforementioned input variables, and the weight is the weight value of the film application category.

Next, the server of the present invention combines the user record 120 of the movie application category with the weight value corresponding to the input variable of the video application category, and performs step S230 user score calculation. After calculation, the scores of the users who have browsed the web sites in the video application category will be obtained, and then the scores will be sorted according to the scores from high to low, and the user-determined N values according to the invention are selected to be used. The top N percent of high-volume users list 240. Please refer to the first example diagram of FIG. 4 . In this embodiment, the user browses the video content through the mobile network or the broadband network. Therefore, the user can separately take the mobile network user and the broadband network user separately. The top 20% (13 points or more) and the top 15% (32 points or more) of the total score are used as positive samples. In addition, you can also refer to statistics such as the mean and median of the usage distribution. For mobile network users, the average is 9.0 points, the median is 3 points, and the average number of broadband users is 14.4 points. The median is 10 points.

Please refer to FIG. 5 , which is a detailed step diagram of the step analysis of the step S300. The profile analysis step takes the high-volume user list 240 as an input, and performs step S310 to set a complex attribute to be analyzed for the movie application category. The attributes include general user attributes, such as the user's gender, age, education level, etc. In addition, the user can set attributes according to requirements, for example, if the user's purpose is to find potential high-volume video users, It can be set, for example, whether the user is a paying user, a bid rate, a recent bill average amount, a download upload data traffic, a main transmission period, and a number of base stations used for roaming.

After the high-volume user list 240 sets the complex attribute to be analyzed via step S310, the server of the present invention performs cross-analysis, one of which is the chi-square verification of step S320, which is used to analyze the category attribute variable, after the chi-square verification, ² The value is significantly larger, that is, it can be selected as a significant attribute 350 of the user; for example, in the embodiment, please refer to the second example of FIG. 6 , and the majority of the high-volume users who frequently watch popular movies and TVs are mostly women. The number is 1.24 times that of men, while among other users, the proportion of women to men is only 0.87 times. Similarly, in the chi-square verification in step S320, the second attribute selected as the significant attribute 350 has a geographical distribution about the user, and is observed according to the distribution of the high-volume user living in the county and the city, and resides in the northern metropolitan area. Among the mobile users in the Central and Southern Territory, a higher proportion is paid for high-volume users of video applications. The proportion of paying users in Taipei, New Taipei, Taoyuan, Hsinchu, Taichung, and Tainan is higher than the average.

In addition, the server of the present invention will also perform the variance analysis (ANOVA) of step S330 to analyze the continuous attribute variables, and select the user attribute with a significantly larger F-verification value as the salient attribute 350. For example, in this embodiment, the user wants to find out whether the age attribute of the user has a significant influence on whether the user is a high-activity user, and analyzes the positive and negative correlation between the significant attribute and the user in step S340. The algorithm observes that the regression coefficient is -0.006774, which is a negative value, which indicates that the user age has a negative relationship with the distribution of high-volume user groups, and further observes the user's age distribution, as shown in the third example of Figure 7. It can be found that the average age of high-volume users (37.9 years old) in the video application category is younger than other users (45.1 years old), and the high-volume users show the highest ratio in the 18 to 24 years old range. Three significant attributes are 350.

Similarly, the variation analysis in step S330 can further analyze the fourth significant attribute 350, which is related to the user's mobile attribute, according to the fourth example diagram of FIG. 8, the user's moving distance and the utterance type, etc. Statistical data is observed, and users with high usage are mostly users who like to move around, make friends and have a large circle of life. During weekdays, the mobile distance of high-volume users is 151 km more than that of other users. During weekends, the mobile distance of high-volume users is 55 km more than other users. It can be inferred from this phenomenon that high-volume users may be mostly commuters, and they may watch the film and television time when they are on a train, subway or bus. On the other hand, it can be observed that the number of users in high-volume users, the number of base stations used for sending calls, the number of minutes of total calls, the number of total calls, and the number of outgoing objects outside the network account for a high ratio. The high-volume users are obviously A group with more social activities.

In this embodiment, according to the analysis of the variance of step S330, a fifth saliency attribute 350 can be found, which is related to the attribute of the degree of payment, which indicates that the high-volume user is willing to spend a higher purchase amount on the mobile phone, The ratio is 1.5 times the average of other users, and the average purchase period is shorter, which is 0.8 times the average of other users. In addition, in terms of the user's bidding for the telecommunications business, the proportion of high-volume users who eat the full-fledged users is 13.83%, which is much higher than the proportion of high-volume users who are not full users. Moreover, if the average amount of bills in the past three months is observed, the average amount of bills for high-volume users is 1.7 times that of other users, and the amount of data services and value-added services also accounts for the proportion of their bills. high.

In this embodiment, according to the chi-square verification of step S320, a sixth saliency attribute 350 can be found, which is related to the attribute of the user action service plan, that is, in the high-volume broadband user, observe the purchase of the high-volume user. The company's business portfolio situation. The user who has subscribed to the mobile phone number of a certain communication company can be specially marked so that the communication company's customer service can quickly find the customer's attribute at the time of contact, in order to try to recommend the user to match the purchase or the new contract. The communication company's special movie marketing plan, etc., or the user who has not rented the communication company's mobile number, can plan a code-carrying service plus a film and television program to attract high-volume users to purchase more of the communication company. Services and products.

In this embodiment, according to the chi-square verification of step S320, the seventh significant attribute 350 about the user's Internet access rate can also be found. As shown in the fifth example diagram of FIG. 9, the user who can find the application for high-speed Internet access is observed. The higher proportion is for high-volume users, and the amount of downlink data of users is also significantly higher, 1.5 times that of other users. In addition, the average amount of data connected by high-volume paying users is higher than that of high-volume users. Among them, the downlink volume is 104% higher and the uplink volume is 68% higher. In addition, the average connection time for film and television high-paying users is longer and the number of connections is smaller.

Finally, according to the analysis of the variance of step S330, the eighth significant attribute 350 can be found, which is a geographical distribution area of the number of broadband video users. For example, high-definition video users are densely concentrated in the northern part of Taiwan, including Keelung and Taipei. , Xinbei, Taoyuan, Hsinchu and other regions, the number of users in these areas accounted for 44.59% of all users, which can be concluded that one out of every five high-volume users who watch movies and TV is paid users (accounting for the number of users in the region) 21%~26.97%).

In the present embodiment, by the method of the present invention, for the high-volume users of the video application category, finally, eight significant attributes 350 can be found, which can be used to describe the high-volume users of the video application category. User profile 360.

It can be seen that the present invention is a method for the user to perform a series of association analysis on the URL of the URL browsed by the user according to the application category to be analyzed, thereby depicting the outline of the user, and the method according to the present invention can be effective. Excavate potential customers to achieve the benefits of accurate marketing.

Compared with the prior art, the present invention has the following advantages:

1. Provide a way to find the outline of a particular group of users through a user-defined URL.

2. A method for calculating a user score, comprising a weight value analysis method, which can perform a single layer model training on each application category through a neural network to generate a weight including each input variable, which can be provided The score calculation method preferably sets the weight so that the calculation method of the user score can be applied to multiple application fields and maintains better user identification, and the weight value analysis method can also be replaced by logistic regression or other algorithms.

3. Provide a method for finding the outline of a specific group of users. You can set pre-defined hundreds of user attributes or user-defined attributes, cross-analyze through chi-square verification and variance analysis to find high-volume users and Other usage users have statistically significant differences in attributes to design a suitable marketing solution with such contours.

4. Analysis of significant attributes through a logistic regression algorithm is positively or negatively correlated with respect to high-volume user lines.

5. Combining the current huge amount of data computing technology with the method of the present invention, it can solve the problem that the traditional single machine is difficult to process a large number of users or long-term accumulated data, and provides a better user profile analysis method than the prior art.

In summary, the present invention is innovative in terms of technical ideas, and also has various functions that are not in the prior art, and has fully complied with the statutory invention patent requirements of novelty and progressiveness, and has filed a patent application according to law, and invites you to approve the invention. The patent application was inspired to invent, and it was a matter of feeling.

Claims (6)

 A method for analyzing a URL to generate a user profile, the method comprising the steps of: selecting a user group according to an application category in an original browsing log, wherein the application category is a predefined content homogenous or approximate URL (Uniform Resource Locator) a location class; analyzing usage information such as the number, time, or period of the URL address of the user group link or browsing the application category to generate a user record; corresponding to the known positive or negative URL address Using information to perform machine training to obtain a weight value corresponding to the application category; calculating a score for the user record according to the weight value and sorting, to obtain at least one user with a high usage amount from the user group according to the sorted score; The at least one user performs profile analysis to generate a user profile corresponding to the application category, wherein the profile analysis sets a complex attribute to be analyzed for the application category, and the attributes are statistically analyzed to generate at least one of the at least one user a significant attribute, and more than a correlation of the at least one significant attribute with respect to the at least one user, At least a significant attribute describes the user profile.    The method of claim 1, wherein the weight value is generated by performing a variable analysis training on the use information by a Neural Network or a Logistic Regression model.    The method of claim 1, wherein the at least one user selects a user whose number of percentages is higher since the score recorded by the user is reduced.   The method of claim 1, wherein the statistical analysis method used in the contour analysis comprises at least a Chi-square Test process, and the chi-square verification process is used for performing cross-analysis in the category. A variable analysis is performed to output a significantly larger χ ² value as the at least one significant attribute.  The method of claim 1, wherein the statistical analysis method used in the contour analysis comprises at least an analysis of variance (ANOVA) process for performing cross-analysis. Continuous variable analysis to output a significantly larger F-test value to the at least one significant attribute.    The method of claim 1, wherein the profile analysis determines that the at least one significant attribute is positively or negatively correlated with the at least one user by a Logistic Regression model.