KR20190047576A - Alternating AutoencoderMethod and System for recommendation System - Google Patents

Abstract

Disclosed are a selective auto-encoding method and an auto-encoding system for a recommendation system. The auto-encoding method based on collaborative filtering includes: a step of performing first auto-encoding to a matrix M in which pre-collected item information is set as a column of the matrix M corresponding to an input layer of a deep neural network; a step of determining whether to perform auto-encoding with another auto-encoder based on a preset threshold value and a root mean square error after the first auto-encoding; and a step of performing second auto-encoding based on a transpose matrix M^T of the matrix M, in which pre-collected user information is set as a column, in accordance with the determination that the second auto-encoding is performed with another auto-encoder.

Description

TECHNICAL FIELD The present invention relates to an automatic encoding method and an auto encoding system for a recommendation system,

The present invention relates to a technique for applying an autoencoder used for performing feature extraction to extract meaningful values in data to a deep neural network based on collaborative filtering .

An autoencoder is a technique used in machine learning learning, and is described in the following non-patent document [1] by Ian Goodfellow , Yoshua Bengio , and Aaron Courville, Deep learning, MIT Press, 2016. http: //www.deep learningbook . org . Is used for feature extraction as in the deep learning learning presented in Fig.

For example, unlike other neural networks, the auto encoder learns that the input and the output are the same as shown in Table 1 below.

As shown in Table 1, in an autoencoder that learns that the input and the output are the same, the loss function can be expressed as Equation 1 below. ≪ EMI ID =

Consumer satisfaction with products is a very important factor in designing a recommendation system, but there is a difficulty in ensuring sufficient satisfaction data because voluntary cooperation of consumers is required.

Accordingly, there is a need for a technique of predicting the entire data through learning based on collected (i.e., secured) partial satisfaction data (i.e., a small amount of satisfaction data).

[1] Ian Goodfellow, Yoshua Bengio, and Aaron Courville, Deep learning, MIT Press, 2016. http: //www.deep learningbook.org. [2] Suvash Sedhain, Menon Krishna, Scott Scanner and Lexing Xie. Autorec: Autoencdersmeet collaborative filtering. Proceedings of 24th ACM International Conference on World Wide Web, pp. 111-112, 2015. [3] YunhongZhou, Dennis Wilkinson, Robert Schreiber and Rong Pan. Large-scale parallel collaborative filtering for the Netflix prize. Proceedings of the 4th international conference on Algorithmic Aspects in Information and Management, pp. 377-348, 2008. [4] Joonseok Lee, Seungyeon Kim, Guy Lebanon and Yoram Singer. Local low-rank matrix approximation. Proceedings of the 30th International Conference on Machine Learning, pp. 82-90, 2013. [5] Dongsheng Li, Chao Chen, Qin Lv, Junchi Yan, Li Shang and Stephn M. Chu. Low-rank matrix approximation with stability. Proceedings of the 33rd International Conference on Machine Learning, pp. 295-303, 2016.

The present invention relates to cooperative filtering based auto-encoding techniques for predicting overall data based on some satisfaction data collected in connection with a particular product (e.g., movie, music, broadcast, physical product, etc.).

A method of autoencoding based on collaborative filtering, the method comprising the steps of: acquiring a matrix M of item information previously collected into a column of a matrix M corresponding to an input layer of a Deep Neural Network; Determining whether to perform auto encoding based on a different auto encoder based on a root mean square error after the first auto encoding and a predetermined threshold value, And a second auto encoding based on the transpose matrix M ^T of the matrix M in which the previously collected user information is set as a column, And performing the steps of:

According to an aspect, the step of determining whether to perform the auto-encoding includes calculating an average square root error of the current auto-encoder for each epoch in which learning is performed, and calculating the average square root error of the current auto- And determining to change the current auto-encoder to the other auto-encoder if the current auto-encoder is smaller than the current auto-encoder.

According to another aspect, whenever it is decided to perform auto encoding in the other auto encoder, the encoder output of the current auto encoder can be set to the weight matrix of the decoder of another auto encoder.

According to another aspect of the present invention, the step of performing the first auto encoding may further include the step of, based on back propagation, setting the weight of the decoder of the first auto- And updating the output of the encoder of the first auto encoder.

According to another aspect of the present invention, the step of performing the first auto encoding includes setting a column of a matrix M corresponding to an input layer of the Deep Neural Network on the basis of the item information And performing an encoding on the matrix M at an encoder of a first autoencoder to estimate a transpose matrix V ^T of a matrix V of matrix factorization schemes .

을 추정하는 단계를 더 포함할 수 있다.According to another aspect, the step of performing the first auto encoding comprises the steps of performing a decoding based on the estimated transposed matrix V ^T to estimate a matrix U of matrix factorization, A matrix corresponding to an output layer of the Deep Neural Network based on the matrix U and the transposed matrix V ^T

May be estimated.

According to another aspect of the present invention, the step of performing the second auto encoding includes setting the user information to a column of a transposed matrix M ^T of a matrix M corresponding to an input layer of the Deep Neural Network And performing an encoding on the matrix M at an encoder of a second autoencoder to estimate a transposed matrix U ^T of a matrix U of matrix factorization schemes .

을 추정하는 단계를 더 포함할 수 있다.According to another aspect, the step of performing the second auto encoding comprises the steps of: performing decoding based on the estimated transposed matrix U ^T to estimate a matrix V of matrix factorization, A matrix corresponding to the output layer of the Deep Neural Network based on the transpose matrix U ^T and the matrix V,

May be estimated.

In a collaborative filtering based autoencoding system, a matrix M, in which item information previously collected is set as a column of a matrix M corresponding to an input layer of a Deep Neural Network, A first auto encoder for performing a first auto encoding on the basis of a first auto encoding and a root mean square error after the first auto encoding and a second auto encoder based on a predetermined threshold, And the second auto encoding is performed by the other auto encoder, the pre-collected user information is determined based on the transpose matrix M ^T of the matrix M set in a column The second auto encoding may comprise a second auto encoder.

According to an aspect of the present invention, the determining unit may calculate an average square root error of a current auto encoder for each epoch in which learning is performed, and if the calculated average square root error is less than the threshold value, It can be determined to change to an encoder.

According to another aspect, whenever it is decided to perform auto encoding in the other auto encoder, the encoder output of the current auto encoder can be set to the weight matrix of the decoder of another auto encoder.

According to another aspect of the present invention, the first auto-encoder is configured to perform, on the basis of error propagation back propagation, the first auto-encoder based on the first auto- Lt; RTI ID = 0.0 > encoder < / RTI >

According to another aspect of the present invention, the first auto encoder sets a column of a matrix M corresponding to the input layer of the Deep Neural Network based on the item information, An encoder of an autoencoder can perform encoding on the matrix M to estimate the transpose matrix V ^T of the matrix V of matrix factorization.

을 추정할 수 있다.According to another aspect, the first auto encoder performs decoding based on the estimated transposed matrix V ^T to estimate a matrix U of matrix factorization, and estimates the matrix U and the transpose A matrix corresponding to the output layer of the Deep Neural Network based on the matrix V ^T

Can be estimated.

According to another aspect of the present invention, the second auto encoder sets the user information to a column of a transpose matrix M ^T of a matrix M corresponding to an input layer of the Deep Neural Network, In an encoder of an autoencoder, encoding is performed on the matrix M to estimate a transposed matrix U ^T of a matrix U of matrix factorization.

을 추정할 수 있다.According to another aspect, the second auto encoder performs decoding based on the estimated transpose matrix U ^T to estimate a matrix V of matrix factorization, and the estimated transpose matrix U ^T and A matrix corresponding to the output layer of the Deep Neural Network based on the matrix V,

Can be estimated.

According to embodiments of the present invention, a plurality of auto encoders based on collaborative filtering-based in-depth neural networks are used to acquire some satisfaction data collected in association with a specific product (e.g., movie, music, broadcast, physical product, The entire data can be predicted.

In addition, based on the average square root error (RMSE), the conversion from the current auto encoder to the other auto encoder can be performed to improve the performance of the conventional cooperative filtering based auto encoder comparison system. That is, the prediction accuracy can be improved.

1 is a diagram illustrating a layer structure of an auto-encoding system based on cooperative filtering in an embodiment of the present invention.
FIG. 2 is a flowchart illustrating a cooperative filtering-based auto-encoding method in an embodiment of the present invention.
3 is a block diagram illustrating an internal configuration of an auto-encoding system based on cooperative filtering, according to an embodiment of the present invention.
4 is a block diagram illustrating a structure of an auto-encoding system including a plurality of different auto-encoders according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The present invention relates to an auto-encoding technique for predicting overall data (i.e., satisfaction data) through a neural network based on collaborative filtering, and more particularly, to a method and apparatus for automatically selecting a plurality of different auto- The present invention relates to a technique for predicting whole data by using the above- That is, the present invention relates to a technique for predicting all data from a part of data by successively performing a plurality of different auto-encoders.

In the present embodiments, " satisfaction data " indicates data indicating satisfaction with the product by the user. The data includes user information data indicating satisfaction of the user collected in advance with respect to the specific product, And may include product information data. In the present embodiments, it can be assumed that the satisfaction data in the matrix factorization model appears as a product of user information and product information. Here, information on latent variables common to user information and product information can be displayed. For example, if the latent variable is an action genre, the user information may indicate the degree to which a user likes the action, and the item information may indicate the degree to which the item represents the action genre well.

In the present embodiments, " current auto encoder " can represent an auto encoder currently used for performing auto encoding, and other auto encoders can represent one auto encoder other than the current auto encoder. For example, when two auto encoders are used (I-AE, U-AE), the current auto encoder may be either I-AE or U- (For example, U-AE) other than the auto encoder set to the auto encoder (for example, AE). In the following embodiments, a matrix corresponding to the output layer of the neural network, that is, a prediction result, is sequentially output by performing I-AE and U-AE sequentially. The A-AE may be performed after the U-AE is performed first, and the I-AE may be sequentially performed after the U-AE is performed.

In the present embodiments, an autoencoder used for collaborative filtering may include an I-AE that takes a column of a matrix as an input and a U-AE that takes a row of a matrix as an input.

In the present embodiments, the I-AE is a non-patent document [2] Suvash Singhain , Menon Krishna, Scott Scanner and Lexing Xie . Autorec: Autoencdersmeet collaborative filtering. Proceedings of the 24th ACM International Conference on World Wide Web, pp. 111-112, 2015. In the co-filtering based auto-encoding algorithm, the auto-encoder performs auto encoding based on item information, and U- May represent an auto-encoder that performs auto-encoding based on user information.

Hereinafter, the case where the U-AE is sequentially performed after the I-AE is performed will be described as an example.

1 is a diagram illustrating a layer structure of an auto-encoding system based on cooperative filtering in an embodiment of the present invention.

Referring to FIG. 1, a collaborative filtering based auto-encoding system 100 may include a plurality of autoencoders. For example, the auto encoder may include a first auto encoder (e.g., I-AE) and a second auto encoder (e.g., U-AE). At this time, each of the auto encoders may include an encoder 110 and a decoder 120.

At this time, the input 130 of the encoder 110 corresponds to the input layer of the neural network, and the output 140 of the decoder 120 corresponds to the output layer of the neural network. That is, a matrix based on a part of product information corresponding to a predetermined reference amount, and a matrix based on a part of user information corresponding to the reference amount may be set as an input of the encoder 110 of each auto encoder. The output 140 corresponding to the output layer of the neural network can be predicted by adjusting the weights in at least one hidden layer 150 based on the input matrix.

FIG. 2 is a flowchart illustrating a cooperative filtering-based auto-encoding method according to an exemplary embodiment of the present invention. FIG. 3 illustrates an internal configuration of a cooperative filtering-based auto-encoding system according to an exemplary embodiment of the present invention. Block diagram.

2 and 3 below illustrate a case where U-AE is sequentially performed after performing I-AE, but this corresponds to the embodiment. After performing U-AE, I-AE is sequentially .

Referring to FIG. 3, the auto-encoding system 300 may include a first auto encoder 310, a determination unit 320, and a second auto encoder 330. Each of the steps 210 to 230 of FIG. 2 corresponds to the first auto encoder 310, the determination unit 320, and the second auto encoder 330, which are components of the auto-encoding system 300 of FIG. ≪ / RTI > Here, each of the first auto encoder 310 and the second auto encoder 330 may include a corresponding encoder and a decoder.

In step 210, the first auto encoder 310 receives a matrix M, in which item information previously collected is set as a column of a matrix M corresponding to an input layer of a Deep Neural Network, Auto encoding can be performed.

For example, the first auto encoder 310 may set a column of a matrix M corresponding to an input layer of a Deep Neural Network based on item information. Then, the encoder of the first auto encoder 310 may perform encoding on the matrix M to estimate the transpose matrix V ^T of the matrix V of the matrix factorization method. That is, the first auto encoder 310 encodes the matrix M including the product information as column components to generate the transposed matrix V ^T (which is a feature matrix composed of product-related feature values) Can be estimated. At this time, in a state where the weight of the decoder of the first auto encoder 310 is fixed, the output V ^T = H of the encoder of the first auto encoder 310 can be updated based on the back propagation method.

이 추정될 수 있다.The decoder of the first auto encoder 310 may perform decoding based on the estimated transposed matrix V ^T to estimate a matrix U of matrix factorization. That is, the decoder of the first auto encoder 310 can estimate a user feature value matrix U corresponding to a feature value of user information based on the transposed matrix V ^T. Thus, based on the estimated matrix U and the transpose matrix V ^T , a matrix corresponding to the output layer of the Deep Neural Network

Can be estimated.

In step 220, the decision unit 320 calculates a root mean square error (RMSE) after the first auto encoding in the first auto encoder 310, and calculates a root mean square error (RMSE) It is possible to determine whether or not to perform auto encoding based on another auto encoder (i.e., the second auto encoder 330) based on the specified threshold value.

For example, the determining unit 320 may calculate a mean square root mean square error (RMSE) of the first auto encoder 310, which is the current auto encoder, for each epoch in which learning is performed. If the calculated mean square root error is smaller than the predetermined threshold value, it may be determined that the current auto encoder is changed from the first auto encoder 310 to the second auto encoder 330.

At this time, each time the decision unit 320 decides to change the current auto encoder to a different auto encoder instead of holding the current auto encoder, that is, the first auto encoder or the second auto encoder, 1 The encoder output H (= V ^T ) of the current encoder can be set to the weight matrix of the decoder of another auto encoder whenever it is decided to change to the auto encoder. For example, when the current auto encoder is the first auto encoder 310, the matrix H, which is the output of the encoder of the first auto encoder 110, may be set to the weight matrix of the decoder of the second auto encoder 330. The first has an output H (= V ^T), (= V ^T), an updated matrix H when the update of the automatic encoder 310 can be set to a weight matrix of the decoder of the second auto-encoder 330.

On the other hand, when the RMSE of the first auto encoder 310, which is the current auto encoder, is equal to or greater than the predetermined threshold value, it can be determined that the first auto encoder 310 which is the current encoder is maintained as it is.

In step 230, the second auto encoder 330 determines to perform the second auto encoding in the second auto encoder 330, so that the pre-collected user information is stored in the transpose matrix of the matrix M set in the column the second auto encoding can be performed based on the transpose matrix M ^T.

For example, when the average square root error (RMSE) of the first auto encoder 310, which is the current auto encoder, is smaller than a predetermined threshold value, the current encoder is changed from the first auto encoder 310 to the second auto encoder 320 If so, the second auto encoder 330 may set the user information to a column of the transpose matrix M ^T. Then, the second automatic encoder (autoencoder, 330) of the encoder (encoder) is, by performing the encoding intended for the transposed matrix M ^T to estimate the transpose matrix U ^T of the matrix U of the matrix factorization techniques (matrix factorization) have.

를 추정할 수 있다. 이때, 제2 오토인코더(autoencoder, 330)의 디코더(decoder)는 추정된 상기 전치행렬 U^T과 상기 제1 오토인코더(310)의 출력 H(=V^T)을 기반으로 설정된 가중치 행렬

을 기반으로 디코딩을 수행하여 심층신경망(Deep Neural Network)의 출력층에 해당하는 행렬

를 추정할 수 있다. 이때, 추정된 상기 행렬 V는 상기 제1 오토인코더(310)의 디코더의 가중치 행렬로 설정될 수 있다.The decoder of the second auto encoder 330 may perform decoding based on the estimated transpose matrix U ^T to estimate a matrix V of matrix factorization. Then, based on the estimated transposed matrix U ^T and the matrix V, a matrix corresponding to the output layer of the Deep Neural Network

Can be estimated. At this time, the second auto-encoder decoder (decoder) is a weighting matrix is set based on the output H (= V ^T) of the first auto-encoder 310 and an estimated U ^T the transposed matrix of (autoencoder, 330)

And a matrix corresponding to the output layer of the Deep Neural Network

Can be estimated. At this time, the estimated matrix V may be set as a weight matrix of the decoder of the first auto encoder 310.

4 is a block diagram illustrating a structure of an auto-encoding system including a plurality of different auto-encoders according to an embodiment of the present invention.

In Fig. 4, the configuration in which the output of the current auto encoder is set to the weight matrix of the other auto encoder every time the setting of the current auto encoder is changed will be described.

와 완전 행렬(complete rating matrix)

의 차원(rank)이

일 때,

로 나타내지는 것을 의미할 수 있다. 이때, 행렬 U 와 행렬 V는

일 수 있다. 행렬 U와 V를 결정하기 위해 최소제곱법 최적화기법(least squares optimization problem)이 이용될 수 있다. 예컨대, 아래의 수학식 2에 기초하여 최적화 문제의 해(solution)를 구함으로써, 행렬 U와 V가 계산될 수 있다. Collaborative filtering may be based on a rank factorization theorem under row rank assumption. The rank factorization theorem is an incomplete rating matrix,

And complete rating matrix.

The rank of

when,

. ≪ / RTI > At this time, the matrix U and the matrix V

Lt; / RTI > A least squares optimization problem can be used to determine the matrices U and V. For example, by obtaining a solution of the optimization problem based on Equation (2) below, the matrices U and V can be calculated.

&Quot; (2) "

In Equation (2), M _{i, j} represents the (i, j) th entity of M, and I _{i, j} represents For matrix M 1 if the (i, j) th entity exists, and if the (i, j) th entity does not exist, And [lambda] can represent a randomization coefficient.

를 계산하고, 제1 오토인코더(I-AE)의 디코더(420)는 상기 행렬 U에

를 곱함으로써, 협력 필터링을 수행할 수 있다. 예를 들어, 제1 오토인코더(I-AE)의 디코더(420)의 가중치 행렬 W는 이전 단계(step)에서 제2 오토인코더(U-AE)의 출력인 행렬 U로 설정되므로, 제1 오토인코더(I-AE)의 디코더(420)는 상기 행렬 U에 제1 오토인코더(I-AE)의 인코더(410)의 출력인 V ^T 를 곱함으로써, 심층신경망의 출력층에 해당하는 행렬

을 추정할 수 있다.Referring to FIG. 4, based on Equation 2 above, the encoder 430 of the second auto-encoder (U-AE)

And the decoder 420 of the first auto encoder (I-AE) calculates the matrix U

To perform cooperative filtering. For example, since the weight matrix W of the decoder 420 of the first auto encoder (I-AE) is set to the matrix U which is the output of the second auto encoder (U-AE) in the previous step, The decoder 420 of the encoder I-AE multiplies the matrix U by V ^T , which is the output of the encoder 410 of the first auto encoder (I-AE), so that a matrix corresponding to the output layer of the deep-

Can be estimated.

), σ는 비선형 활성함수(nonlinear active function)인 경우, 제1 오토인코더(I-AE)의 출력, 즉, 상기 출력층에 해당하는 행렬

(= y_i)은 아래의 수학식 3과 같이 표현될 수 있다.In this case, the matrix m _i corresponding to the input layer of the neural network is composed of the i-th column vectors of the matrix M

) and σ is a nonlinear active function, the output of the first auto encoder (I-AE), that is, the matrix corresponding to the output layer

(= y _i ) can be expressed by Equation (3) below.

&Quot; (3) "

In Equation (3), w _r may represent the weighting matrix of the encoder and decoder. For example, in the case of including the _{w 1, w 2, w 3} , w 4 , as shown in FIG 1, w _1, w ₂ denotes a weight matrix of the encoder _{(410, 430), w 3} , w 4 is a decoder (420 , 440). And b ₁ and b ₂ may represent a bias corresponding to the weights w ₁ and w ₂ of the encoders 410 and 430, respectively.

The weight and bias for each layer of the first auto-encoder (e.g., I-AE) shown in Equation (3) are based on the back-propagation technique Can be calculated by solving the solution of the optimization problem. For example, the weight (W ₁ , W ₂ ) and the bias of the encoder of the first auto encoder can be calculated based on the following equation (4).

&Quot; (4) "

는 표시 함수(indicator function)를 나타내는 것으로서, 사용자 정보 i가 상품 정보 j와 관련된 경우에는 1, 아닌 경우에는 0으로 설정될 수 있다. 그리고, λ는 정규화 계수(randomization coefficient)를 나타낼 수 있다. In Equation (4) above,

Is indicative of an indicator function and may be set to 1 if user information i is related to product information j, or to 0 otherwise. And [lambda] can represent a randomization coefficient.

(4) can be used to solve the problem based on rank factorization of cooperative filtering through an auto-encoder (e.g., I-AE, U-AE) have.

은

로 표현될 수 있다. 여기서, W는 I-AE의 디코더의 가중치 행렬

을 포함하고,

는 I-AE의 인코더의 출력을 나타내는 것으로서,

로 표현될 수 있다. 여기서,

는 시그모이드(sigmoid) 함수를 나타내고

는 ReLU(Rectified Linear Unit) 함수를 나타낼 수 있다. W ₁ , W ₂ 는 오토인코더의 인코더의 가중치 행렬을 나타낼 수 있다.In the above equation (4), in the case of I-AE having three hidden _{(hidden layer, n L = 3} ) as shown in FIG. 1, M _{i, j} is the matrix M has a default value (default) is set to contain 0 . 1, the matrix m _i corresponding to the input layer may be composed of an entity corresponding to the i-th column of the matrix M , and the output of the I-AE

silver

. ≪ / RTI > Where W is the weight matrix of the decoder of I-AE

/ RTI >

Represents the output of the encoder of the I-AE,

. ≪ / RTI > here,

Represents a sigmoid function and < RTI ID = 0.0 >

Can represent a ReLU (Rectified Linear Unit) function. W ₁ , W ₂ can represent the weighting matrix of the encoder of the auto-encoder.

Referring again to FIG. 4, in the first auto encoder or the second auto encoder, the relationship between the encoders 410 and 430 of the respective auto encoders and the decoders 420 and 440 can be expressed by Equation (5) below.

&Quot; (5) "

로 표현될 수 있다. W는 가중치 행렬을 나타내고, H는 오토인코더(즉, 오토인코더의 인코더)의 출력 행렬을 나타내는 것으로서,

로 표현될 수 있다. 그리고, 수학식 5의 행렬들은 차원(rank) r을 가질 수 있다.In Equation (5), Y is an output matrix of an auto encoder (i.e., a decoder of an auto encoder), and may represent an estimated matrix corresponding to an output layer. for example,

. ≪ / RTI > W denotes a weight matrix, H denotes an output matrix of an auto-encoder (i.e., an encoder of an auto-encoder)

. ≪ / RTI > The matrices of Equation (5) may have a rank r.

In the case of such an auto encoder, since the decoder of the auto encoder depends on the least squares method of the encoder of the auto encoder, in order to find the optimal U and V as shown in Equation (2) Layer multilayer perceptron may be used.

, 상기 출력 행렬 V를 추정하기 위한 멀티레이어 페셉트론

이 이용될 수 있다. 여기서, 멀티레이어 퍼셉트론

의 파라미터는

,

의 파라미터는

이고, 각 퍼셉트론의 입력 x는 불완전 행렬(incomplete matrix)의 로우 벡터들(row vectors)과 컬럼 벡터들(column vectors)로 구성될 수 있다. 상기 퍼셉트론

와

를 이용하여 최소화할 최적화 문제는 아래의 수학식 6과 같이 표현될 수 있다.For example, two different auto-encoders (i.e., a first auto-encoder and a second auto-encoder) can be used to estimate the output matrices U, V of the encoders of each auto-encoder. That is, the multilayer perceptron for estimating the output matrix U

A multi-layered cepstrum for estimating the output matrix V ,

Can be used. Here, the multi-layer perceptron

The parameters of

,

The parameters of

, And the input x of each perceptron may be composed of row vectors and column vectors of an incomplete matrix. The perceptron

Wow

The optimization problem to be minimized can be expressed by Equation (6) below.

&Quot; (6) "

또는 제2 오토인코더가 현재 오토인코더로 설정된 경우의 출력 행렬

이 생성될 수 있다. 수학식 4에서 설명한 바와 같이, 최적화 문제를 기반으로 상기 U와 V를 추정하기 위해 먼저 랭크 인수분해(rank factorization)가 수행되며, 랭크 인수분해That is, the output V of the encoder 410 of the first auto encoder and the output U of the encoder 430 of the second auto encoder are estimated based on Equation (6) above, and the estimated outputs U and V and the weight matrix W The output matrix to be estimated when the first auto-encoder is currently set to the auto-encoder

Or the output matrix when the second auto encoder is currently set to auto encoder

Can be generated. As described in Equation (4), rank factorization is first performed to estimate U and V based on the optimization problem, and rank factorization

At this time, the operation in which the current auto encoder changes the setting from I-AE to U-AE or from U-AE to I-AE as shown in FIG. 4 can be performed based on the algorithm shown in Table 2 below.

)로 설정될 수 있다. 위의 표 2에서

는 k번째 에폭(epoch)에서의 RMSE를 나타내고,

는 k번째 에폭(epoch)에서의 임계값을 나타낼 수 있다. 여기서, 임계값

는

로 설정될 수 있다.위의 표 2에서는 I-AE에서 학습을 시작한 이후 U-AE에서 순차적으로 학습을 수행하는 것을 나타내고 있으나, 이는 실시예에 해당되며, U-AE에서 먼저 학습을 시작하고 순차적을 I-AE에서 학습이 수행될 수 있다. 그리고, 표 2의 단계(step) 3 내지 7에 따르면, 현재 오토인코더로 설정된 I-AE의 인코더의 출력

는 업데이트될 수 있다. 예컨대, 상기 I-AE의 인코더의 출력

는

에서 I-AE의 디코더(420)의 가중치 W _k 가 고정된(fixed) 동안 업데이트될 수 있다. 상기

의 업데이트는 계산된 RMSE가

를 만족할 때까지 반복될 수 있다. Referring to Table 2, when the current auto encoder is set to the first auto encoder (I-AE), after each training epoch of the I-AE, the decision unit 320 can calculate the RMSE of the I- have. The determining unit 320 may compare the calculated RMSE with a predefined threshold value. At this time, if RMSE is above the threshold value, the current auto encoder setting is maintained as I-AE, and if RMSE is less than the threshold value, the current auto encoder setting can be changed to I-AE. That is, the current auto encoder can be changed from I-AE to U-AE. Thus, whenever the current setting of the auto encoder is changed, the output of the encoder of the current auto encoder can be set to the weight matrix of the decoder of the auto encoder to be changed. For example, the weight matrix of the decoder 440 of the U-AE output H (= V ^T) of the encoder 410 of the I-AE (

). In Table 2 above,

Represents the RMSE in the k < th > epoch,

May represent a threshold value at the kth epoch. Here,

The

In the above Table 2, it is shown that the U-AE performs the learning sequentially in the U-AE after the learning starts, but this corresponds to the embodiment, and the U-AE first starts the learning, Learning can be performed in I-AE. According to the steps 3 to 7 of Table 2, the output of the encoder of the I-AE currently set to the auto encoder

Can be updated. For example, the output of the encoder of the I-AE

The

While in a weight W _k of the decoder 420 of the I-AE fixed (fixed) it can be updated. remind

The update of the calculated RMSE is

Can be repeated.

와 현재 오토인코더의 설정을 변경하는 변경 시간(alternation time)

이 업데이트될 수 있다. 유사한 방법으로, 제2 오토인코더인 U-AE의 파라미터들은 표 2의 단계(step) 9 내지 14에서 업데이트될 수 있다. 그리고, RMSE 및 임계값을 기반으로 현재 오토인코더의 설정은 I-AE에서 U-AE, U-AE에서 I-AE로 변경하는 동작은

될 때 까지 반복 수행될 수 있다.In step 8 of Table 2,

And the alternation time to change the setting of the current auto encoder.

May be updated. In a similar manner, the parameters of the second auto-encoder U-AE may be updated in steps 9-14 of Table 2. Based on the RMSE and the threshold value, the current auto encoder setting is changed from I-AE to U-AE, and from U-AE to I-AE

Lt; / RTI >

와 같이 표현되고, U-AE의 경우,

와 같이 표현될 수 있다.The input / output relationship after the k-th epoch is, as shown in Fig. 4, in the case of I-AE,

In the case of U-AE,

Can be expressed as

Table 3 below shows the performance of an algorithm that sequentially learns based on I-AE and U-AE based on a widely used Movielens dataset (i.e., the auto encoding system of the present invention).

The dataset used in Table 3 can contain 100,000 rating information between 943 users and 1,648 movies (an observation rate of 5.8%). Herein, each rating information has an integer value ranging from 1 to 5. For comparison, the above non-patent document [2] Suvash Singhain, Menon Krishna, Scott Scanner and Lexing Xie . Autorec: Autoencdersmeet collaborative filtering. Autorec, [3] YunhongZhou , Dennis Wilkinson, Robert Schreiber and Rong Pan in Proceedings of the 24th ACM International Conference on World Wide Web, pp. 111-112, Large-scale parallel collaborative filtering for the Netflix prize. ALS-WR, [4] Joonseok Lee, Seungyeon Kim, Guy Lebanon and Yoram Singer , Proceedings of the 4th international conference on Algorithmic Aspects in Information and Management , pp.337-348 , Local low-rank matrix approximation. Proceedings of the 30th International Conference on Machine Learning, pp.82-90, 2013. LLORMA, [5] Dongsheng Li , Chao Chen, Qin Lv, Junchi Yan , Li Shang and Stephn M. Chu . Low-rank matrix approximation with stability. The SMA presented in LLORMA presented in Proceedings of the 33rd International Conference on Machine Learning, pp. 295-303, 2016. can be used.

According to Table 3 above, it can be confirmed that the algorithm that sequentially learns based on I-AE and U-AE (i.e., the auto encoding system of the present invention) has the lowest RMSE and has the best performance. As described above, in the actual data of characteristics having extremely sparse characteristics, that is, data having a small amount of data as compared with the entire data, the U-AE based on the user information or the I- It is possible to confirm that the performance is excellent when auto encoding is performed using both I-AE and U-AE selectively based on the RMSE and the threshold value.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (16)

In a collaborative filtering based autoencoding method,
Performing a first auto encoding on a matrix M in which item information previously collected is set as a column of a matrix M corresponding to an input layer of a Deep Neural Network;
Determining whether to perform auto encoding based on a different auto encoder based on a root mean square error after the first auto encoding and a predetermined threshold; And
The second auto encoding is performed on the basis of the transpose matrix M ^T of the matrix M in which the previously collected user information is set as a column so that it is decided to perform the second auto encoding in the other auto encoder step
Wherein the cooperative filtering is based on the cooperative filtering. The method according to claim 1,
Wherein the step of determining whether to perform the auto-
Calculating an average square root error of the current auto encoder for each epoch in which learning is performed; And
Determining to change the current auto-encoder to the other auto-encoder if the calculated mean square root error is less than the threshold value
Wherein the cooperative filtering is based on the cooperative filtering. The method according to claim 1,
Whenever it is decided to perform auto encoding in the other auto encoder, the encoder output of the current auto encoder is set to the weight matrix of the decoder of the other auto encoder
And a cooperative filtering based auto encoding method. The method according to claim 1,
Wherein performing the first auto encoding comprises:
Updating the output of the encoder of the first auto-encoder based on back propagation, with the weight of the decoder of the first auto-encoder corresponding to the first auto-encoding being fixed;
Wherein the cooperative filtering is based on the cooperative filtering. The method according to claim 1,
Wherein performing the first auto encoding comprises:
Setting a column of a matrix M corresponding to an input layer of the Deep Neural Network based on the item information; And
Estimating a transpose matrix V ^T of a matrix V of matrix factorization by performing an encoding on the matrix M at an encoder of a first autoencoder,
Wherein the cooperative filtering is based on the cooperative filtering. 6. The method of claim 5,
Wherein performing the first auto encoding comprises:
Estimating a matrix U of matrix factorization by performing decoding based on the estimated transposed matrix V ^T ; And
A matrix corresponding to the output layer of the Deep Neural Network based on the estimated matrix U and the transposed matrix V ^T

≪ / RTI >
And a cooperative filtering based auto encoding method. The method according to claim 1,
Wherein performing the second auto-
Setting the user information as a column of a transpose matrix M ^T of a matrix M corresponding to an input layer of the Deep Neural Network; And
Estimating a transposed matrix U ^T of a matrix U of matrix factorization by performing an encoding on the matrix M at an encoder of a second auto encoder,
Wherein the cooperative filtering is based on the cooperative filtering. 8. The method of claim 7,
Wherein performing the second auto-
Estimating a matrix V of matrix factorization by performing decoding based on the estimated transpose matrix U ^T ; And
A matrix corresponding to the output layer of the Deep Neural Network based on the estimated transpose matrix U ^T and the matrix V,

≪ / RTI >
And a cooperative filtering based auto encoding method. In an autoencoding system based on collaborative filtering,
A first auto encoder for performing a first auto encoding on a matrix M in which item information previously collected is set as a column of a matrix M corresponding to an input layer of a Deep Neural Network;
A determination unit for determining whether to perform auto encoding based on another auto encoder based on a root mean square error after the first auto encoding and a predetermined threshold; And
The second auto encoding is performed on the basis of the transpose matrix M ^T of the matrix M in which the pre-collected user information is set to a column as the second auto encoding is determined to be performed in the other auto encoder, Auto encoder
A cooperative filtering based auto encoding system. 10. The method of claim 9,
Wherein,
Calculating an average square root error of the current auto encoder for each epoch in which learning is performed and determining to change the current auto encoder to the other auto encoder if the calculated mean square root error is less than the threshold value
Wherein the cooperative filtering based auto-encoding system comprises: 10. The method of claim 9,
Whenever it is decided to perform auto encoding in the other auto encoder, the encoder output of the current auto encoder is set to the weight matrix of the decoder of the other auto encoder
Wherein the cooperative filtering based auto-encoding system comprises: 10. The method of claim 9,
Wherein the first auto encoder comprises:
Updating the output of the encoder of the first auto-encoder based on back propagation with the weights of the decoders of the first auto-encoder corresponding to the first auto-encoding being fixed
Wherein the cooperative filtering based auto-encoding system comprises: 10. The method of claim 9,
Wherein the first auto encoder comprises:
A column of a matrix M corresponding to the input layer of the Deep Neural Network is set based on the item information and an encoder of the first auto encoder M to estimate the transpose matrix V ^T of the matrix V of the matrix factorization technique
Wherein the cooperative filtering based auto-encoding system comprises: 14. The method of claim 13,
Wherein the first auto encoder comprises:
Estimating a matrix U of matrix factorization by performing decoding based on the estimated transposed matrix V ^T and calculating a matrix U based on the estimated matrix U and the transposed matrix V ^T based on the Deep Neural Network ≪ / RTI >< RTI ID = 0.0 >

To estimate
Wherein the cooperative filtering based auto-encoding system comprises: 10. The method of claim 9,
Wherein the second auto encoder comprises:
The user information is set as a column of a transpose matrix M ^T of a matrix M corresponding to an input layer of the Deep Neural Network and the encoder of the second auto- M to estimate the transpose matrix U ^T of the matrix U of the matrix factorization scheme
Wherein the cooperative filtering based auto-encoding system comprises: 16. The method of claim 15,
Wherein the second auto encoder comprises:
To estimate the matrices V and, based on the estimated the transpose matrix U ^T, and the matrix V of matrix factorization techniques (matrix factorization) to perform decoding based on the estimated the transpose matrix U ^T the depth neural network (Deep Neural Network ≪ / RTI >< RTI ID = 0.0 >

To estimate
Wherein the cooperative filtering based auto-encoding system comprises:

Images