JP2016177728A - Data analysis apparatus and data analysis method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique capable of appropriately evaluating the similarity among screen transitions for the analysis of a user behavior pattern in a website.SOLUTION: According to the present invention, similar screen transitions are clustered in such a way as to belong to the same cluster and the frequency of each cluster is accumulated, thereby quantitatively representing a user behavior pattern in a website. More specifically, a system according to the present invention includes a function that accumulates access logs and derives data per session; a function that accumulates screen transition patterns in the data per session; a function that clusters screen transition patterns; and a function that accumulates a ratio of the clustered screen transition patterns for every user.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a data analysis apparatus and a data analysis method, for example, a technique for analyzing a user's screen transition pattern based on an access log of a website.

  Big data analysis technology has attracted attention, and in various fields, it has been attempted to obtain knowledge useful for business by analyzing data. Data analysis technology has also been proposed for measures to expand sales and to investigate illegal login measures. In particular, in the provision of products and services through websites, many data analysis techniques have been proposed because customer actions can be acquired as access logs.

  Non-Patent Document 1 and Patent Document 1 propose a method of deriving a screen transition (access route) for each session from an access log of a website and obtaining a Longest Common Subsequence (LCS) for each screen transition. . LCS represents the longest of the partial series of list X and the partial series of list Y that are included in both lists in common. For example, the LCS (X, Y) of X = (A, F, B, D, E) and Y = (A, B, C, D, E) is (A, B, D, E). Patent Document 1 proposes a method in which the most frequently obtained LCS is used as a frequent access pattern.

  If the access pattern tendency (similarity) can be analyzed in this way, it becomes possible to extract a user access behavior pattern in a Web site that requires login (for example, a financial Web site). If such a pattern can be extracted, the identity of the access action can be detected, and unauthorized access due to unauthorized login can be detected. Such similarity information can also be used for marketing. In other words, it is a hint of how to improve Web pages based on user access patterns so that they can be used efficiently.

JP 2004-152209 A

Seiji Toda, Haruo Yokota: Evaluation of Scalability Improvement Methods for LCS Analysis of Web Logs, The Database Society of Japan Letters, Vol.2 (No.3), 2003 Needleman, S.B. and Wunsch, C.D. “A general method applicable to the search for similarities in the amino acid sequence of two proteins,” Journal of Molecular Biology, vol.48, pp.443-453, 1970.

  However, since the LCS is the same if, for example, the start and end of the screen transition match, the method of analyzing the tendency of the access pattern by the LCS may not be able to appropriately express the similarity of the screen transition. For example, the LCS (P, Q) of P = (A, S, F) and Q = (A, T, F) is (A, F). On the other hand, LCS (P, R) of P = (A, S, F) and R = (A, V, W, X, Y, Z, F) is also (A, F). In this way, even in the same LCS, the original screen transitions are not always similar, which is a problem in access pattern analysis by LCS.

  The present invention has been made in view of such a situation, and provides a technique capable of appropriately evaluating the similarity between screen transitions in order to analyze a user's behavior pattern on a Web site.

  In order to solve the above problems, in the present invention, clustering is performed so that similar screen transitions belong to the same cluster, and the frequency of each cluster is aggregated to quantitatively represent a user's behavior pattern on the website. Provide technology. That is, in the present invention, data representing screen transition for each session is converted as screen transition data from the access log. Then, the similarity between the screen transition patterns in the screen transition data is calculated using the distance calculation between character strings, and the screen transition patterns are clustered based on the similarity. The clustering information is stored in a storage device and used for access log analysis.

  According to the present invention, it is possible to accurately and quantitatively grasp a user's macro screen transition pattern on a website. This makes it possible to improve the efficiency of access analysis, such as the construction of a website according to the user's screen transition pattern, analysis of changes in user behavior before and after campaign measures, and detection of users who impersonated regular users and logged in it can.

  Problems, configurations, and effects other than those described above will become apparent from the following embodiments for carrying out the present invention and the accompanying drawings.

It is a figure showing a schematic structure of a system (data analysis device) concerning an embodiment of the present invention. It is a figure which shows an example of access log data. It is a figure which shows an example of session data. It is a figure which shows an example of screen transition data. It is a figure which shows an example of user pattern data. It is a figure for demonstrating the example of a clustering process. It is a flowchart for demonstrating a session data generation process. It is a flowchart for demonstrating a screen transition total process. It is a flowchart for demonstrating a clustering process. It is a flowchart for demonstrating a user pattern total process.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be noted that this embodiment is merely an example for realizing the present invention, and does not limit the technical scope of the present invention. In each drawing, the same reference numerals are assigned to common components.

  In the following description, “program” will be the subject, but the program is executed by the processor, and processing determined by using the memory and communication port (communication control device) will be performed. It is good. Further, the processing disclosed with the program as the subject may be processing performed by a computer such as a management server or an information processing apparatus. Part or all of the program may be realized by dedicated hardware, or may be modularized. Various programs may be installed in each computer by a program distribution server or a storage medium.

<Configuration of data analyzer>
FIG. 1 is a functional block diagram showing a schematic configuration of a data analysis apparatus according to an embodiment of the present invention. The data analysis apparatus includes a central processing unit (processor) 100 that performs necessary arithmetic processing and control processing, an input / output device 110 for inputting and outputting data, and a program required for processing in the central processing unit 100. And a storage device 130 for storing data to be processed by the central processing unit 100 or data after processing.

  The input / output device 110 includes a display device 111 for displaying data, an output device including a printer (not shown), and a keyboard for performing operations such as selecting a menu for the displayed data. 112, a pointing device 113 such as a mouse.

  The program memory 120 analyzes the access log on the website and converts it into session unit data, a session data generation program 121, a screen transition tabulation program 122 that tabulates screen transition patterns in the session unit data, A clustering program 123 for clustering patterns and a user pattern totaling program 124 are stored. Each processing program is stored in the program memory 120 as a program code, and each processing is realized by the central processing unit 100 executing each program code.

  The storage device 130 stores the access log data 131 storing the access log accumulated in advance, the session data 132 storing the information for each session extracted from the access log data, and the screen transition pattern in the session data. The screen transition data 133 and user pattern data 134 for storing the ratio of the screen transition clustering results for each user are stored. The storage device 130 may be a storage system that is remotely located via a network.

  The processing program, data, each program, etc. described above can be provided by being stored in various recording media such as a CD-ROM, DVD-ROM, MO, floppy (registered trademark) disk, USB memory or the like.

<Access log data>
FIG. 2 is a diagram illustrating an example of the access log data 131 in the storage device 130. The access log data stores information when the user accesses each screen on the Web site from which the access log is acquired. The session ID 201 is an ID that uniquely represents a session. The session ID is assigned the same session ID after the user logs in to the Web site until the user transitions to another Web site or closes the Web browser. The access date 202 represents the date when the user accessed the screen. The user ID 203 is an ID that uniquely represents the user ID, and stores the user ID of the user who accessed the screen. The access screen ID 204 represents the ID of the screen accessed by the user. The access screen ID is uniquely determined for each screen.

  The Web site to be accessed is premised on, for example, login (user authentication) in order to acquire a user ID.

<Session data>
FIG. 3 is a diagram illustrating an example of the session data 132 in the storage device 130. The session ID 301 is the same as the session ID 201 in the access log data 131. The access start date and time 302 is an access date and time when the first screen is accessed during screen transition in the same session. The user ID 303 is the same as the user ID 203 in the access log data 131. The screen transition 304 is a list of access screen IDs of screens transitioned in the same session in the order of access. For example, (A, B, C) indicates that the screens whose access screen IDs are “A”, “B”, and “C” are accessed in order.

  In FIG. 3, the screen transition when the session ID is d00001234 is (A, B, C), and the screen transition when d00001236 is (E, B, C, D). The same access screen ID (for example, B or C) indicates the same screen, but the top page as the entrance is different between A and E. This is because a plurality of types of top pages serving as entrances are provided. For example, the screen A shows the PC top page, and the screen E shows the mobile top page.

<Screen transition data>
FIG. 4 is a diagram illustrating an example of the screen transition data 133 in the storage device 130. No. 401 is an ID that uniquely represents each record, that is, a screen transition. The screen transition 402 is a list of access screen IDs of screens transitioned in the same session in the order of access, like the screen transition 304 in the session data 132. In the screen transition data 133, the same screen transition is collected into one record. The cluster ID 403 is an ID representing a cluster corresponding to the result of clustering the screen transition. The screen transition data represents the correspondence between the original screen transition and the cluster to which the screen transition belongs.

<User pattern data>
FIG. 5 is a diagram showing an example of user pattern data 134 in the storage device 130. The user ID 501 is the same as the user ID 203 in the access log data 131. The cluster ID 502 is the same as the cluster ID 403 in the screen transition data 133, and the screen transition of the user is replaced with the corresponding cluster ID in the screen transition data. The ratio 503 represents the ratio of the number of sessions having the cluster ID to the total number of sessions for the user.

<Outline of processing in data analyzer>
Processing performed in the data analysis apparatus having the above-described configuration will be described. First, the central processing unit 100 uses the session data generation program 121 to read the access log data 131 in the storage device 130 and generate session data 132.

  Next, the screen transition aggregation program 122 is executed. The session data 132 is read from the storage device 130, the screen transitions are tabulated in a comprehensive and non-overlapping state, and the obtained screen transition data 133 is stored in the storage device 130. Next, the clustering program 123 is executed. Clustering is a process of classifying similar screen transition patterns into the same cluster. The clustering result is written in the screen transition data and stored in the storage device 130.

  Subsequently, a user pattern totaling program is executed. The session data 132 and the screen transition data 133 are read from the storage device 130, the session data is aggregated for each user, the screen transition of each user is replaced with the clustering result, and the ratio of each cluster to the number of sessions of each user is calculated. The pattern data 134 is stored in the storage device 130. Each process will be described in detail below.

<Session data generation processing>
FIG. 7 is a flowchart for explaining session data generation processing executed by the session data generation program 121. The operating subject here is a session data generation program. In the session data generation process, a process for generating session data as shown in FIG. 3 is performed based on the access log data as shown in FIG.

  In step 701, the access log data 131 is read.

  In step 702, an unprocessed session ID is extracted. In other words, since processing is performed in units of session IDs from step 703 to step 706, session IDs that have not been subjected to these processings are extracted.

  In step 703, the record having the session ID extracted in step 702 is extracted.

  In step 704, the access date / time is acquired from the records having the oldest access date / time among the records extracted in step 703, and this is set as the access start date / time.

  In step 705, data (screen transition data) listing access screen IDs in ascending order of access date and time is generated from the record extracted in step 703. For example, in the record extracted in step 703, there are 3 records with the session ID “d00001234”, and the screen transition data when the access screen IDs are “A”, “B”, and “C” in order from the oldest access date (A, B, C).

  In step 706, the obtained data, that is, the session ID, access start date / time, user ID, and screen transition are stored as one record in the session data. Such processing is performed for all session IDs.

  In step 707, it is determined whether all session IDs have been processed. If the determination result is true, the process ends. If not, the process returns to step 702.

<Screen transition aggregation processing>
FIG. 8 is a flowchart for explaining the screen transition totaling process executed by the screen transition totaling program 122. The operating subject here is a screen transition counting program. In the screen transition totaling process, a process for generating screen transition data as shown in FIG. 4 is performed based on the session data as shown in FIG. Note that the screen transitions are totaled every predetermined period (for example, every three months).

  In step 801, the session data 132 is read.

  In step 802, an unprocessed record in the session data is read.

  In step 803, it is determined whether the screen transition in the record read in step 802 has been registered in the screen transition data 133. If registered, the process proceeds to step 805, and if not registered, the process proceeds to step 804.

  In step 804, the screen transition in the record read in step 802 is registered in the screen transition data 133. That is, No. and screen transition are stored as one record in the screen transition data 133. No. assigns a unique number arbitrarily. At this time, no data is stored in the cluster ID (Null value).

  In step 805, it is determined whether all records in the session data 132 have been processed. If not processed, the process returns to step 802, and if processed, the process is terminated.

<Clustering processing>
FIG. 9 is a flowchart for explaining the clustering process executed by the clustering program 123. The operation subject here is a clustering program. In the clustering processing, clustering processing is performed on the screen transition data as shown in FIG. 4 based on the screen transition 402, and processing for obtaining the cluster ID 403 is performed.

  FIG. 6 is a schematic diagram for explaining the clustering process. Hereinafter, description will be made along the flowchart of FIG. 9 using the examples of FIGS.

  In step 901, screen transition data is read. As an example, assume that data (1) to (5) in FIG. 4 are stored. The screen structure at this time is defined as shown in FIG. In FIG. 6A, each rectangle represents a screen, and the attached character represents a screen ID. Moreover, the line between screens represents a link.

  In step 902, the similarity between clusters is obtained. In the initial state, all screen transitions start from a singleton, that is, a cluster state composed only of the screen transitions. Similarity between clusters is obtained for all pairs as shown in FIG. At this time, the similarity is obtained according to equation (1).

          S (a, b) = Len (LCS (a, b)) / Dist (a, b) (1)

  In equation (1), S (a, b) represents the similarity between screen transition a and screen transition b. LCS (a, b) represents the LCS of screen transition a and screen transition b. Len (a) represents the number of screens constituting the screen transition a. Dist (a, b) represents the distance between character strings between screen transition a and screen transition b. The distance between character strings is information that represents how much two character strings differ as a distance. Examples of the distance between character strings include a Levenshtein distance and a Jaro-Winkler distance. Although the distance between character strings can be arbitrarily selected, in the present embodiment, description will be made using the Levenshtein distance. As a method for obtaining the Levenshtein distance, there is a method called DP matching as shown in Non-Patent Document 2. As shown in Non-Patent Document 2, DP matching is a matching method in which a non-linear expansion / contraction between two objects (characters, DNA, speech, etc.) and the most matched state is used as a matching result. In DP matching, there are a penalty Pz for deviation and a penalty Pn for mismatch as measures of inconsistency between objects. Empirically, the penalty setting value is preferably set so that Pz: Pn is 1: 3. When applied to screen transition, for example, when Pz = 1, Pn = 3, a = (A, B, C), b = (A, B, D), Dist (a, b) = 3. Also, the similarity of P = (A, S, F), Q = (A, T, F), R = (A, V, W, X, Y, Z, F) described above using equation (1) When calculating the degree, Dist (P, Q) = 3, Dist (P, R) = 19, so S (P, Q) = 2/3, S (P, R) = 2/19, LCS More precise matching is possible than in the case of using only. LCS is a problem for precise matching even if there are many mismatched screens between screen transitions, but it is not necessary for LCS by combining the distance between character strings to obtain the similarity. Since the mismatched screen can be considered as a penalty, more precise matching is possible. If the cluster contains multiple screen transitions, separate processing is required, which will be described later.

Returning to the flowchart, in step 903, the maximum similarity S _max is obtained from the obtained inter-cluster similarity. In FIG. 6B, since 1.67 in the case of the cluster pair (4)-(5) is the maximum value, this value is set as S _max .

In step 904, S _max is determined whether or similarity threshold S _T. S _T can be changed depending on whether considered to be similar to how much screen transition. In the present embodiment, description will be made assuming that S _T = 0.8. If the determination result is true, the process proceeds to step 905, and if it is false, the process proceeds to step 906.

  In step 905, the cluster pair having the maximum similarity in step 903 is merged. That is, the screen transitions included in the cluster pair are included in the same cluster and given the same cluster ID. In the example of FIG. 6, a cluster ID C1 is assigned to the pair (4)-(5).

  Returning to step 902, the similarity between clusters is obtained again. In the first loop, all clusters were singletons, but in the second loop, there are clusters that contain multiple screen transitions, so the method of obtaining similarity is different. Specifically, the similarity is obtained by brute force between screen transitions included in both clusters, and the minimum value is set as the similarity between the clusters. For example, when calculating the similarity between (1) and C1, calculate the similarity between the screen transitions included in C1 and (1), and set the minimum value as the similarity between (1) and C1. . Since C1 includes (4) and (5), the similarity between (1)-(4) and (1)-(5) is obtained. Since the similarity between (1)-(4) is 0.06, the similarity between (1)-(5) is 0.15, and the minimum value is 0.06, this is the similarity between (1) and C1. By adopting the minimum value of the similarity in the screen transitions included in the cluster, all the screen transitions in the cluster have a certain degree of similarity, and precise clustering is possible. In this way, the cluster pair similarity is obtained as shown in FIG.

  In step 903, Smax is obtained. In the case of FIG. 6C, the similarity between (1) and (2) is 1.0, which is the maximum value.

In step 904, it is determined whether Smax = 1.0 is S _T = 0.8 or more. Since it is true, merge (1) and (2). That is, a cluster ID that uniquely represents the cluster is assigned. In this example, C2 is assigned.

Thereafter, step 904 is false, i.e. Smax repeats a loop until less than S _T. If false, the process proceeds to step 906, and a cluster ID is assigned to the screen transition of the singleton. That is, a cluster ID is assigned to a screen transition that has not become the same cluster as any screen transition in the processing from step 902 to step 905. In the example of FIG. 6, since (3) is a singleton, a cluster ID of C3 is given by this processing.

  In step 907, the cluster information thus obtained, that is, the cluster ID of each screen transition is written in the screen transition data and stored in the storage device 130. FIG. 4 shows the screen transition data finally obtained in this example.

<User pattern aggregation processing>
FIG. 10 is a flowchart for explaining the user pattern totaling process executed by the user pattern totaling program 124. The operating subject here is a user pattern totaling program. In the user pattern totaling process, the cluster ID ratio for each user is obtained from the session data as shown in FIG. 3 using the screen transition data as shown in FIG. 4, and the user pattern totaling data as shown in FIG. 5 is generated. I do.

  In step 1001, session data and screen transition data are read.

  In step 1002, session data is totaled for each user. That is, a process of extracting records of the same user from the session data using the user ID as a key and totaling them for each user.

  In step 1003, an unprocessed user session group is selected from among the session groups tabulated for each user.

  In step 1004, the cluster ID corresponding to the screen transition of each session in the session group selected in step 1003 is obtained by referring to the screen transition data, and the screen transition is replaced with the cluster ID.

  In step 1005, the ratio of each cluster ID to the number of sessions in the session group is obtained, and the cluster ID and the ratio are stored in the user pattern data. For example, if the number of sessions in the session group is 100 and the number of clusters C1 is 12, the ratio is 0.12.

  In Step 1006, it is determined whether or not all user session data has been processed. If the determination result is false, the process returns to step 1003, and if true, the process ends.

<Summary>
In the data analysis apparatus according to the embodiment of the present invention, an access log of a website is acquired in advance. In this access log, information from when the user logs in to the target website until it leaves the target website is recorded. Based on the access log, the data is converted into session unit data (session data). In this, the transition information of the screen accessed by the user in the same session is recorded. Next, the data analysis apparatus comprehensively obtains the types of screen transitions based on the session data. Furthermore, the data analysis apparatus obtains a similarity between screen transitions based on the session data, and performs screen transition clustering based on the similarity. At this time, when calculating the similarity, not only LCS but also calculation of distance between character strings (DP matching as an example) enables more precise matching of the user's macro screen transition tendency. The data analysis apparatus classifies the session data in units of users and then converts each screen transition of the user into a cluster ID with reference to the screen transition data. Then, the ratio of each cluster ID to the number of user sessions is obtained, and user pattern data is generated. The obtained user pattern holds the ratio after the screen transition accessed by the user is replaced with the cluster ID. The clustering result obtained in this way represents a screen transition pattern from a more macro viewpoint, with the difference between fine screen transitions absorbed. Therefore, it is possible to more easily and accurately grasp what screen transition the user has performed on the website.

  Note that the present invention is not limited to the embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiments. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  In addition, each configuration, function, processing unit, processing unit, and the like described in the embodiments may be realized in hardware by designing a part or all of them with, for example, an integrated circuit. Further, each of the above-described configurations, functions, etc. may be realized by software by the processor interpreting and executing a program that realizes each function. Information such as programs, tables, and files for realizing each function is stored in a recording or storage device such as a memory, hard disk, or SSD (Solid State Drive), or in a recording or storage medium such as an IC card, SD card, or DVD. be able to.

  Furthermore, in the above-described embodiment, control lines and information lines are those that are considered necessary for explanation, and not all control lines and information lines on the product are necessarily shown. All the components may be connected to each other.

100 Central processing unit (processor)
110 I / O devices
111 Display device
112 keyboard
113 mouse
120 program memory
121 Session data generator
122 Screen transition summary program
123 Clustering program
124 User pattern aggregation program
130 storage devices
131 Access log data
132 Session data
133 Screen transition data
134 User pattern data

Claims (14)

A data analysis device for analyzing access logs,
A processor that executes a program for analyzing the access log;
A storage device for storing management information for enabling analysis of the access log,
The processor is
From the access log, a process of converting data representing screen transition for each session as screen transition data;
A process of calculating a similarity between screen transition patterns in the screen transition data using a distance calculation between character strings, and clustering the screen transition patterns based on the similarity,
Processing for storing the clustering information in the storage device;
Data analysis device that executes. In claim 1,
The data analysis apparatus further executes a process of applying a result of the clustering process to the screen transition in a session group for each user and totaling a ratio of screen transitions after clustering. In claim 1,
The data analysis apparatus, wherein the processor clusters the screen transition patterns for each user in the clustering process. In claim 1,
The data analysis apparatus, wherein in the clustering process, the processor classifies the pair into one cluster when the similarity degree of the screen transition pattern pair indicates a predetermined value or more. In claim 4,
In the clustering process, the processor includes a pattern included in the screen transition pattern classified into the cluster when there is no similarity between the screen transition pattern pair and the similarity is equal to or greater than a predetermined value. A data analysis apparatus that classifies each of the screen transition patterns other than as a single cluster. In claim 1,
The data analysis apparatus, wherein the screen transition pattern includes screen transition in the same session. In claim 1,
The data analysis apparatus, wherein the access log is an access log for a website that requires user authentication. A data analysis method for analyzing access logs,
A processor that executes a program for analyzing an access log converts data representing a screen transition for each session as screen transition data from the access log;
The processor calculates a similarity between screen transition patterns in the screen transition data using a distance calculation between character strings, and clusters the screen transition patterns based on the similarity;
The processor storing the clustering information in a storage device for storing management information for enabling the access log to be analyzed;
Data analysis method including. The claim 8, further comprising:
A data analysis method including a step in which the processor applies a result of the clustering process to the screen transition in a session group for each user, and totals a ratio of screen transitions after clustering. In claim 8,
The data analysis method, wherein in the clustering step, the processor clusters the screen transition patterns for each user. In claim 8,
The data analysis method, wherein in the clustering step, the processor classifies the pair into one cluster when the similarity degree of the screen transition pattern pair indicates a predetermined value or more. In claim 11,
In the clustering step, the processor includes a pattern included in the screen transition pattern classified into the cluster when there are no more similarities in the screen transition pattern pair indicating the predetermined value. A data analysis method for classifying each of the screen transition patterns other than as a single cluster. In claim 8,
The data analysis method, wherein the screen transition pattern includes screen transition in the same session. In claim 8,
The data analysis method, wherein the access log is an access log for a website that requires user authentication.

Images