KR100496414B1 - An user clustering method using hybrid som for web information recommending and a recordable media thereof - Google Patents

Abstract

The present invention relates to a user clustering method using hybrid self-organizing feature maps for recommending web information used to provide customized information necessary for a user on the Internet.

Recently, researches on the web information recommendation system for providing personalized information necessary for each user to the web sites such as internet shopping malls are actively conducted. Among many types of web information recommendation systems, the information recommendation method using clustering is web. After clustering a user, a method of performing differentiated web information service using characteristics of a new user and representative characteristics of a corresponding cluster most similar to the user is performed.

By the way, as one of the methods for clustering as described above, a conventional self-organization map (SOM) model is used, but there are various problems such as fast performance and ease of use, but it cannot guarantee convergence.

Therefore, the present invention proposes a hybrid self-organization shape map (HSOM) model to solve the conventional problems.

The HSOM model proposed by the present invention is a model in which probability distribution is applied to initial weight determination and weight update of an existing SOM model, thereby ensuring accurate convergence in clustering.

Description

User clustering method and recording medium using hybrid self-organizing shape map for web information recommendation {AN USER CLUSTERING METHOD USING HYBRID SOM FOR WEB INFORMATION RECOMMENDING AND A RECORDABLE MEDIA THEREOF}

The present invention provides a method and method for clustering users using hybrid self-organizing feature maps (hereinafter referred to as “HSOM”) for recommending web information used to provide personalized information necessary for a user on the Internet. The present invention relates to a computer-readable recording medium having recorded thereon a program for realizing this.

Recently, researches on the web information recommendation system for providing personalized information necessary for each user to the web sites such as internet shopping malls are actively conducted. Among many types of web information recommendation systems, the information recommendation method using clustering is web. After clustering a user, a method of performing differentiated web information service using characteristics of a new user and representative characteristics of a corresponding cluster most similar to the user is performed.

Clustering is a method of classifying each individual object into several homogeneous clusters based on similarity of various characteristics in a given learning data. The desirable result of such clustering is that there is some kind of close similarity between individuals belonging to the same community, and there must be relative incompatibility between individuals belonging to different clusters. Discriminant analysis is a method that knows which clusters each entity belongs to and derives a discriminant function using the given data, and uses them to classify each unknown entity. This is a method of assigning objects to homogeneous clusters using data that does not know whether they belong to a cluster and how many total clusters.

In general, there is an AHC (Agglomerative Hierarchical Clustering) algorithm among the clustering methods that have been widely used in the prior art, this algorithm has a disadvantage that the speed is slow when there is a lot of target data. Another method is a single-link and group-average method, which takes a long time to perform the algorithm.

On the other hand, the clustering method using SOM of neural network is Self-Organizing feature Maps (hereinafter referred to as “SOM”) that best models the human brain structure among various neural network models. The neural networks model proposed by Kohonen at the beginning is the SOM, which has an unsupervised learning structure in which learning is performed without knowing the result of the learning data among neural networks.

However, although the above-mentioned SOM has been widely applied and many important studies have been made, there are a number of problems that are still unresolved.

First, SOM does not have a density function defined in the data space. In other words, it does not have a way of formalizing the relationship between the distribution of the training data and the weight distribution.

Second, there is no process of optimizing an objective function that manages errors in the learning algorithm of SOM. Therefore, it is necessary to build and optimize the objective function using the stochastic distribution that plays the role of the objective function.

Third, there is no general guarantee that SOM's learning algorithms always converge. In particular, the final model after learning often falls into local optimal values. Therefore, there is a need for a method using a training data that converges to a global optimal value within tolerance.

Fourth, SOM does not have a theoretical framework that determines the optimal value chosen as the model parameter. In other words, SOM is a model through the update of the weight through the connection from the input layer to the output layer, but the description of the prediction model is absolutely insufficient because it gives meaning to the prediction value itself rather than the process of generating the prediction value when calculating the prediction value for the new input. . For example, there is no theoretical structure for determining the initial value for the learning rate, changing rules, functions for decreasing the neighborhood function, and update distribution of weights. In this neural network model that lacks the theoretical basis, there is a need for a method that enables a model interpretation of prediction results.

Fifth, SOM cannot be compared with other SOM models or learning models of heterogeneous architectures. This is because no explicit mathematical rules for the model are defined, and the mathematical measures of general model evaluation should also be applied to the SOM model.

Accordingly, an object of the present invention is to provide a method for overcoming the model limitations that cannot be solved in the SOM by proposing a HSOM that guarantees convergence of clustering results while maintaining fast learning ability.

A user clustering method for recommending web information according to the present invention for solving the above problems includes an initialization step; Determining a winner node; Parameter updating of the weight distribution; A clustering step, wherein the initializing step comprises: initializing a weight vector; Initializing the learning rate; Initializing a neighbor function, wherein determining a winner node comprises: initializing an input vector; Selecting a weight distribution; Selecting a victory node using a Euclidean distance between an input vector and an output vector, wherein updating the parameters of the weight distribution comprises: updating with new weights; Replacing the previous update distribution with a new update distribution, wherein the initializing and weighting updating of the weight vector are characterized by ensuring convergence by applying a Bayesian inference technique based on the prior probability distribution of the exponential group.

Hereinafter, the present invention will be described in more detail with reference to equations and drawings.

In general, cluster analysis calculates a distance matrix representing dissimilarity among all the individuals using p cluster variables for a total of N individuals, and then uses the distance matrix to form a cluster. The process takes place. There are two types of clusters: hierarchical clusters and disjoint clusters.

Hierarchical clusters are a type in which one cluster is contained within another cluster, but no overlap is allowed between clusters, but in the form of a tree-like dendogram. Say what type it belongs to. When p cluster variables are used for a total of N objects, this can be expressed as an N × p data matrix as shown in [Equation 1]. Here p cluster variables use standardized variables. And, the basic method for measuring the similarity in each case is between two observation vectors x _i '= (x _1i , x _2i ,…, x _pi ) and x _j ' = (x _1j , x _2j ,…, x _pj ) The distance d _ij = d (x _i , x _j ) is calculated to produce a distance matrix of size N × N , as shown in Equation 2 below. This distance matrix is used to bring objects that are relatively close together to form the same cluster. Measured distances for two vectors x ₁ and x ₂ are Euclidean distance, Euclidean distance square, Chebychev distance, Manhattan distance, and Maharanobis distance ( Mahalanobis distance) and the like can be used. Euclidean distance, the most widely used distance measure among these, is defined as shown in [Equation 3]. , Where Q stands for ∑. In addition, Minkowski distance, Power distance, correlation coefficient, and cosine between two vectors can be used.

delete

delete

delete

delete

delete

delete

As described above, hierarchical clustering methods in a case where an N × N distance matrix is given include an agglomerative method and a division method. The merging method is a method of making clusters by tying nearby objects together, while the partitioning method is a method of dividing distant objects. The process of merging or separating these clusters can be expressed simply using a tree-shaped figure. The process of forming a cluster using a merging method is as follows.

1) Start with N clusters first, then create an N × N distance matrix, D.

2) Combine two groups with the smallest distance between them into one cluster.

3) Create a new distance matrix D1 by calculating the distance between the new cluster formed in 2) and the remaining clusters.

4) Repeat the process of 2) and 3) N-1 times.

In addition, the merging method can be defined as the single linkage method, the longest linkage method, the centroid linkage method, the median linkage method, and the average linkage method. linkage method). For example, the shortest connecting method among these methods is a method using a shortest distance of the distance between two objects belonging to the distance between two clusters C _1, C ₂ of each cluster can be expressed as [Equation 4]. d {C ₁ , C ₂ } = min {d (x, y) | x∈C ₁ , y∈C ₂ } The longest linking method uses a similar process and uses the longest distance between the two groups in the distance between the clusters. [Equation 5] shows the equation of the longest link method. d {C ₁ , C ₂ } = max {d (x, y) | x∈C ₁ , y∈C ₂ }

delete

delete

delete

Meanwhile, the SOM applied to various fields such as speech recognition, character recognition, and syntax analysis has a forward monolayer neural network structure composed of an input layer and an output layer as shown in FIG. 1. The weight, which is the connection strength of the SOM, acts like the centroid value of the output node corresponding to the normalized input vector, and the output node with the closest measure (Euclidian distance) to the input vector during training is the winner. ), Only the weights of this winner node and neighboring nodes are updated. In particular, SOM consists of two very simple layers compared to supervised learning models such as multi-layer perceptrons, and projects multidimensional data into two-dimensional feature maps. So that you can do competitive learning yourself. In this case, the two layers are an input layer having an input vector and an output layer having a shape map. Competitive layers as output layers are generally two-dimensional maps. Self-organized shape map models, unlike error back propagation models, use only one feedforward flow without multiple levels of feedback and are fully connected from the input layer to the output layer. It has a structure.

In SOM, the initial strength of neurons is properly initialized with random numbers generated from a uniform distribution between 0 and 1. At this time, the input vector is normalized to have a value between 0 and 1. The learning after this initialization calculates the distance of each value of the input vector corresponding to the weight of each node of the output layer, and the distance measure usually uses Euclidean distance. Finally, the output node with the smallest distance to the given input vector wins. Only this winner node will be output. However, the update learning of the weight may be performed by the winner node and neighboring nodes. In other words, the learning rules of SOM are winner take all, and all output and updates are centered on the winner node. The neighboring region of the winner node does not limit only the very closest nodes, and at the beginning of learning, it extends the neighborhood size to all the neurons in the layer and gradually reduces it during the learning process. Finally, only the weight of the winner node is updated. The weight update around the winner node is calculated as shown in [Equation 6]. W ^(new) = W ^(old) + α (X-W ^(old) ) In Equation 6, W ^(old) is a connection strength vector before being updated, and W ^(new) is a new connection strength vector after being updated. X is an input vector and α is a learning rate, and if α is increased, the update width of the weight increases.

delete

delete

SOM learning simply calculates the difference between the link strength vector and the input vector and adds this value to the previous link strength vector by the learning rate. In this case, not only the connection strength of the winner node is changed but also the nodes in the neighboring region of the winner node are changed in the same manner. In other words, in this case, only the winner node may be updated or the winner node and neighboring nodes may be updated together, and updating only the winner node corresponds to a special case where the range of the neighbor is zero.

While the SOM's data is being learned, the input vector is projected onto a two-dimensional shape map, resulting in competitive learning of the output nodes. In other words, the multidimensional input vector is projected onto the shape map of the SOM, and the dimension reduction occurs, and the individual learning materials form clusters competitively with each other without limitation on the number of clusters.

In SOM, there is a condition to terminate learning because repetitive learning occurs. Kohonen's proposed convergence condition of learning termination is that learning ends when the distance between each data node in the shape map is smaller than the specified convergence value before learning. do. A maximum number of iterations of learning may be predetermined and used in parallel with convergence conditions.

However, as mentioned above, the SOM does not have a general guarantee that the learning algorithm always converges, and in particular, the final model that has been learned often falls into a local optimal value. Therefore, there is a need for a method using a training data that converges to a global optimal value within tolerance.

Therefore, the present invention proposes an HSOM that guarantees convergence of clustering results while maintaining fast learning ability. The HSOM proposed by the present invention applies probability distribution to initial weight determination and weight update of the existing SOM model. Here, the probability distribution means the probability of the value that the random variable can take. In the present invention, the weight of the neural network is defined as the random variable and the probability distribution for the weight is obtained.

Hereinafter, the HSOM proposed by the present invention will be described in more detail.

2 is a flowchart illustrating a process of the HSOM algorithm. As can be seen from the HSOM algorithm first initializes the relevant parameters (S100). In the next step, the winner node is determined according to a predetermined method (S110), and the weight is updated (S120). The HSOM algorithm repeats the determining of the winner node and the updating of the weight until the given condition is satisfied. FIG. 3 is a flowchart illustrating the initialization of HSOM, and [Algorithm 1] is List the code that performs the initialization. [Algorithm 1]

delete

delete

 Algorithm: Initialize_HSOM (Input [i])

 // initialization of HSOM 1: initialization of network constants

Initialize network parameter; Input_layer (int i, int o, int init_neigh_size)

       num_inputs = i, num_outputs = o // Initialize the input and output vectors

neighborhood_size = init_neigh_size // Initialize the neighbor radius of the shape map

weights = new float [num_inputs * num_outputs] // specify weights

outputs = new float [num_outputs] // specify outputs

 // initialization of HSOM 1: initialization of probability distribution

 Initialize bayesian distributions // Determine Bayesian Probability Distributions

Initialization of the weight vector, w _j (0) to have

probabilistic distribution, N (0, 1).

        // the distribution of the initial weights is a Gaussian with mean and variance of 0 and 1, respectively

           Determined by distribution

Initialization of the learning rate α (0), α (t) ∝t ^-α ; 0 <α <1 ,.

// determine the learning rate function

Initialization of the neighborhood function K (j, j ^* ) ,

K decreases as to increase | jj ^* | // Determine neighbor radius function

delete

Where i is the input vector, o is the output vector, and init_neigh_size is the neighbor radius.

As shown in FIG. 3 and the above algorithm, the initialization process of the HSOM first initializes the weight vector (S200). At this time, the initial weight vector w _j (0) has a probability distribution. Next, the learning rate is initialized (S210). The initial learning rate α (0) must satisfy the conditions α (t) ∝ ^t-α and 0 <α <1 . In the next step, the neighbor function (or “neighbor radius function”) K (j, j ^* ) newly introduced in the present invention is initialized (S220). Neighbor function | jj ^* | As the value increases, the K value decreases.

This initial stage of HSOM must first consider the prior probability distribution of the weights. This distribution has prior knowledge of the weights from the structure of the model to be constructed. In general, the initial weight distribution is the prior probability distribution of the exponential family as shown in Equation 7. distribution). Here, Z _w (α) is a normalization factor as shown in Equation 8 below, and α is a hyper parameter. In addition, the probability distribution E _w has the form as shown in [Equation 9]. In Equation 9, M is the total number of weights of the model. Specifying the prior probability distribution as an exponential group makes it easier to calculate the posterior probability distribution. Of course, the prior probability distribution can be determined by any distribution other than the exponential group.

delete

delete

delete

delete

delete

delete

In addition, the parameter of this prior probability distribution has a hyperparameter. For example, assuming that the weight vector is a random variable following a Gaussian distribution, this Gaussian distribution has different parameters for each component in the weight vector. That is, the i th weight ω _i follows a Gaussian distribution with mean μ _i and variance σ _i ² , where μ has a prior distribution of N (0, 1) , where variance is It is possible to determine with a known fixed value or consider an initial weight with a prior distribution of an inverse gamma distribution with position and shape parameters.

In order to remove the influence of different scales between the nodes in the p-dimensional input vector, normalization of each input node is required. The standardization of the i th input node x _i = (x _i1 ,…, x _ip ) is calculated by subtracting the total mean of each node from the values of each node and dividing this value by the total standard deviation of each node. That is, the i- th input node normalized is represented by Equation 10 below. As a measure for determining the victory node in the shape map of the HSOM, the euclidean distance between the standardized input vector and the output node is used. The Euclidean distance between the weight of the i th input node and the j th output node can be calculated using Equation 11. The determination of the victory node w for the i th input vector is made with the node having the smallest distance, dist (x _i ^normal , w _j ) . In HSOM, the region size of the winning node is initially determined to be quite wide. However, as learning progresses, the range gradually decreases. This decreases the range of true values of the parameters as learning progresses.

delete

delete

delete

delete

For each learning, HSOM, which updates the weights of victory nodes and nodes within its neighborhood size, needs to consider the distribution of update distributions that are not considered in SOM. This is because, unlike the mean of the distribution, variance needs to be gradually reduced as the model is optimized through learning.

The variance of the post-probability distribution by the likelihood function of the weights and the prior probability distribution for the weights is smaller than the variance of the prior distribution by Bayes' theorem. Since this post-distribution becomes a pre-distribution of new learning data, the variance of the update distribution becomes small. As the model becomes intelligent, the variance becomes smaller, so consideration of the reduction of variance along with the learning rate of SOM is naturally made in HSOM.

When the prior probability distribution and the likelihood function are determined, the Bayes' theorem gives the form of the posterior probability distribution of Equation 12. Here, S (w) = βE _D + αE _w , and Z _s (α, β) = ∫exp (−βE _D −αE _w ) dw . Solving S (w) again and summarizing [Equation 13]. The mean of the new weight distribution to be updated in Equation 13 is the expectation for the mean of the posterior probability distribution by the product of the prior probability distribution and the likelihood function of the training data, which are considered beforehand regardless of the training data. Is calculated. In this case, the integral for calculating the expected value of the posterior probability distribution is obtained by the Monte Carlo method. Among these methods, the Metropolis algorithm is commonly used. In the markov chain defined by the Metropolis algorithm, the new state, θ ^{(t + 1),} becomes the candidate state of the specified proposal distribution generated from the previous state, θ ^(t) . . This candidate state is either accepted or rejected by the probabilities of the previous state and the original distribution. [Algorithm 2] shows the process of determining the size of the HSOM layer.

delete

delete

delete

delete

delete

delete

Algorithm: HSOM_Size (Layer [k])

Determine initial size of feature maps // Determine dimension of shape maps

for (i = 0; i <= max; i ++)

choose layer_size,

input_layer_size [0]

feature_layer_size [i] // Determine dimension of input layer and shape map

end

Set layers // Determine hierarchy of shape map

current layer replaces as new feature_layer (layer_size [i])

                                      // Determine hierarchy structure according to the change of dimension

for (i = 0; i <= max; i ++)

if (layer_ptr [i] == convergence_const)

// investigation of the convergence of dimensions

end 」The process of determining the size of the HSOM layer consists of two hierarchical structures. First, the initial hierarchical structure of the HSOM is determined before learning using the given training data. In this step, the input structure of the learning data is initialized, and the hierarchical size of the shape map for the first learning from this data is determined. The structure of the input data once determined does not change until the end of HSOM's final learning, but the hierarchical size of the shape map increases sequentially during the learning, until the given conditions are satisfied and the convergence of the optimal cluster results. 4, 5, and [Algorithm 3] show an update procedure for the weight distribution of HSOM using the winning node determination of the SOM model and the probabilistic distribution update strategy of the weights.

delete

delete

Algorithm 3

Algorithm: Train_HSOM (Train [j])

   // update the Bayesian probability distribution of HSOM

Determine the winner node // determine the winner node

        // normalize input vector along Gaussian distribution

Normalization of input vector, Gaussian distribution with mean 0, variance 1

        // determine hyper = parameter of weight

Choose the distribution of weights, w ～ f (θ)

        // determine the node with the minimum Euclidean distance as the winner

Choose the winner node j ^* = arg max y , using Euclidean criteria.

Update of parameters // Update Parameter of Weight Distribution

w _j ^New = w _j ^Old + α (j) K (j, j *) (Xw _j ^Old ),

where, K (j, j ^* ) ; Neighborhood function

end

 // iterate the parameter update task of the weight distribution

Replace old distribution by current.

 // Determine Victory Node Of HSOM

Set Winner_index = 0,

     maxval = -1000000 // Specify maximum number of iterations

Find the winner neuron // Determine Winning Node

for (j = 0; j <= num_outputs; j ++)

for (i = 0; i <num_inputs; i ++)

// node with minimum Euclidean distance becomes the winning node

winner [i, j] = arg min ∥ x (k)-w _j ∥

end

 // update weight distribution

Set m = (int) alpha, delta = alpha-m

// create parameter from Gaussian distribution

while (count <data_size)

v = rand () / 32767.0, w = pow (y / m, delta) / (1+ (y / m-1) * delta)

end

 // iterate until convergence

Repeat Until given criteria satisfaction.

As shown in FIG. 4 and the above algorithm, the present invention first initializes an input vector (S300) and generates weights to be used for nodes of each shape map in an initial Gaussian distribution (S310). The initial predistribution uses a Gaussian distribution with a mean of 0 and a variance of 1. This is because the Euclidean distance is used to determine the triumph node in the shape map of HSOM, and the values of the input vectors are initialized such that the mean follows zero variance. Therefore, updating the weight from the beginning is updating the parameters of the distribution to which they belong. These values are updated by subsequent learning.

Next, the victory node of the HSOM is determined (S320). In this step, the maximum range of a given iterative learning is specified, and the node of the shape map whose final weight value is to be updated is determined.

5 shows a procedure for updating the distribution of weights. Calculation of the new weight (S400) is performed as shown in Equation 14 below. w _j ^New = w _j ^Old + α (j) K (j, j *) (Xw _j ^Old )

delete

Here, K (j, j ^* ) means a neighbor function. At this time, an update is made from the distribution of the hyperparameters for the parameters of the Gaussian distribution, which is the weight distribution (S410). The algorithm uses a gamma distribution as a distribution for the hyperparameters, because the Bayesian posterior probability becomes a conjugate distribution, making calculations easier.

The methods shown and described in the present invention have shown only one embodiment, and various changes can be readily implemented by those skilled in the art without departing from the scope and spirit of the invention.

In addition, the method of the present invention may be implemented by hardware and a suitably programmed computer. A “computer program” is to be understood as meaning any software product that is stored on a computer readable medium, such as a floppy disk or CD, can be downloaded over a network such as the Internet, or sold in any other way.

As described above, the present invention proposes a clustering method using the HSOM model in which a probability distribution is applied to initial weight determination and weight update of an existing SOM model.

The clustering method using the HSOM model according to the present invention can be easily used to calculate the representative preference for each cluster by solving the non-convergence problem of the existing SOM by applying the Bayesian inference method based on probability distribution to initial weight determination and weight update. Can be.

1 is a structural diagram of a conventional magnetic shape map (SOM)

2 is a flowchart illustrating a process of HSOM according to the present invention.

3 is a flowchart illustrating an initialization process of the HSOM according to the present invention.

4 is a flowchart illustrating a process of determining a victory node of HSOM according to the present invention.

5 is a flowchart illustrating an update procedure for a weight distribution of HSOM according to the present invention.

Claims (6)

In the user clustering method for recommending web information: An initialization step; Determining a winner node; Parameter updating of the weight distribution; In the clustering stage, The initializing step may include initializing a weight vector; Initializing the learning rate; Including the initialization of the neighbor function, The determining of the winner node may include initializing an input vector; Selecting a weight distribution; Selecting the victory node using the Euclidean distance between the input vector and the output vector, The parameter updating of the weight distribution may include updating with new weights; Replacing the previous update distribution with a new update distribution, The user clustering method using a hybrid self-organizing shape map, wherein the initializing and weight updating steps of the weight vector ensure convergence by applying a Bayesian inference technique based on a prior probability distribution of the exponential group. delete delete In the user clustering method for recommending web information, An initialization step; Determining a winner node; Parameter updating of the weight distribution; In the clustering stage, The initializing step may include initializing a weight vector; Initializing the learning rate; Including the initialization of the neighbor function, The determining of the winner node may include initializing an input vector; Selecting a weight distribution; Selecting the victory node using the Euclidean distance between the input vector and the output vector, The parameter updating of the weight distribution may include updating with new weights; Replacing the previous update distribution with a new update distribution, And a step of initializing the weight vector and updating the weight vector to ensure convergence by applying Bayesian inference techniques based on the prior probability distribution of the exponential group. delete delete