CN117077735A - Dimension-dependent integrated service quality prediction method based on convolutional neural network - Google Patents

Abstract

The invention provides a dimension-related integrated service quality prediction method based on a convolutional neural network, which specifically comprises the following steps: step A: collecting a WSDream service quality data set, and preprocessing the data set; and (B) step (B): designing an input layer for extracting initial feature vectors of users and services; step C: designing an embedding layer to obtain an embedding vector E of a user _u And embedded vector E of service _s The method comprises the steps of carrying out a first treatment on the surface of the Step D: designing an interaction layer based on an attention mechanism to obtain a feature vector X; step E: and predicting the service quality through the CNN network. The invention can reduce the reasoning time and energy consumption of QoS prediction under the condition of limited computing resources, and realize energy-saving and high-quality service.

Description

Dimension-dependent integrated service quality prediction method based on convolutional neural network

Technical Field

The invention belongs to the field of service quality prediction.

Background

With the increasing number of network services, there are more and more services that provide similar functionality to users. When faced with these services, it may be difficult for users to determine which service best meets their requirements. Therefore, recommending a premium service to a user becomes an important requirement for service recommendation. In order to make better recommendations, it is necessary to further compare non-functional properties of similar services, namely quality of service (QoS), including response time, throughput, reliability, etc. One intuitive way to compare QoS is for the user to invoke all candidate services to obtain an accurate QoS value. However, because the number of services may be very large, invoking all candidate services by the user can cause significant overhead and delay. Thus, it is impractical to actively evaluate the QoS values of these services.

In order to obtain and recommend QoS of a service to a user, a Collaborative Filtering (CF) -based QoS prediction method is widely used, which is one of main QoS prediction methods. CF-based QoS prediction methods can be further classified into a memory-based method and a model-based method. The main idea of the memory-based CF method is to collect histories of similar users or services to predict the missing QoS. For example, the literature "L.Shao, et al," Personalized QoS Prediction for Web Services via Collaborative Filtering, "IEEE International Conference on Web Services (ICWS 2007) pp.439-446" proposes a QoS prediction method based on a user neighborhood, which predicts QoS values from similar users of a target user. The literature "J.Liu, et al," Location-aware and personalized collaborative filtering for web service recommendation, "IEEE Transactions on Services Computing, vol.9, no.5, pp.686-699" states that QoS is generally dependent on the Location of users and services, proposes a Location-aware CF method that selects a neighborhood based on the Location information of users and services. The CF method based on memory is simple to operate and has strong interpretation. However, these methods rely on historical data of QoS values, and users typically invoke only a small number of services, resulting in extremely sparse user-service matrices, which can present difficulties in similarity calculation and neighborhood selection. However, qoS matrices are typically extremely sparse, which results in poor performance of CF-based QoS prediction methods.

Matrix Factorization (MF) is a model-based CF method that maps users and services to the same potential space, obtains their respective potential feature vectors, and then calculates QoS predictions by the inner product of the two vectors. MF can reduce the impact of data sparsity to some extent and tends to achieve better performance than the memory-based CF approach. For example, the literature "J.xu, et al," Web service personalized quality of service prediction via reputation-based matrix factorization, "IEEE Transactions on Reliability, vol.65, no.1, pp.28-37, 2016," proposes a method named Reputation Matrix Factorization (RMF) that integrates a user's reputation value into a matrix factorization to reduce the impact of an untrusted user on QoS predictions. Literature "y.zhang et al," Covering-based web service quality prediction via neighborhood-aware matrix factorization, "IEEE trans.services comput, vol.14, no.5, pp.1333-1344, oct.2019," propose a method named CNMF that finds neighbors of users and services by coverage-based clustering methods and improves the accuracy of predictions. Although the model-based CF methods can alleviate the sparsity problem and improve the accuracy of QoS prediction to a certain extent, only low-order linear characteristic interaction can be learned, and high-order nonlinear characteristic interaction is ignored. However, MF only considers low order linear feature interactions between potential features, and not high order nonlinear feature interactions.

With the development of deep learning, neural networks are becoming an effective method of QoS prediction. They perform QoS prediction by extracting deep features of users and services, and neural networks can automatically learn patterns and rules in data, compared to conventional CF methods. For example, the literature "h.wu, et al," Multiple attributes QoS prediction via deep neural model with contexts, "IEEE Transactions on Services Computing, vol.14, no.4, pp.10841096, 2021," proposes a deep learning based multi-attribute QoS prediction model. It takes into account the context of service invocation, effectively extracts high-order features through multi-layer MLP, which is the first effort to make multi-attribute QoS predictions using DNN. Literature "Y.Zhang, et al," Location-Aware Deep Collaborative Filtering for Service Recommendation, "in IEEE Transactions on Systems, man, and Cybernetics: systems, vol.51, no.6, pp.3796-3807, 2021 "propose a location aware method (LDCF) to perform feature extraction by multilayer MLP and to accomplish QoS prediction in combination with an Adaptive Corrector (AC).

However, these methods generally send the embedded vector directly into the prediction model, which rarely considers the dimensional correlation between the user and the service embedded vector, and the extraction of the higher-order features is insufficient, which results in the degradation of the prediction accuracy of the service quality, and the requirements of service recommendation cannot be satisfied well.

Disclosure of Invention

The invention aims to: in order to solve the problems in the prior art, the invention provides a dimension-dependent integrated service quality prediction method based on a convolutional neural network

The technical scheme is as follows: the invention provides a dimension-related integrated service quality prediction method based on a convolutional neural network, which is characterized by comprising the following steps of:

step A: according to a service quality prediction target of service requirements, a WSDream service quality data set is collected, and the data set is preprocessed;

and (B) step (B): designing an input layer for extracting initial feature vectors of users and services;

step C: designing an embedding layer for converting initial feature vectors of users and services into low-dimensional dense embedding vectors to obtain embedding vectors E of the users _u And embedded vector E of service _s ；

Step D: designing an interaction layer based on an attention mechanism, wherein the interaction layer is used for interacting an embedded vector of a user with an embedded vector of a service so as to obtain a feature vector X;

step E: adopting a characteristic arrangement sequence of a characteristic vector X of a shuffle layer self-adaptive learning, and obtaining a characteristic vector X'; reshape operation is performed on X', fromAnd X' is converted into a matrix form and denoted as M ₀ M is set to ₀ And inputting the service quality into a CNN network, and predicting the service quality through the CNN network.

Further, the pretreatment in the step a specifically includes: and selecting the sparseness degree d according to the service recommendation scene, and randomly reserving d% of data in the data set.

Further, the input layer in the step B extracts relevant explicit features from the region information of the user and the service to obtain a feature vector I of the user _u And feature vector I of service _s ：

I _u ＝[u _id ，u _rg ，u _as ].

I _s ＝[s _id ，s _rg ，s _as ].

Wherein u is _id 、u _rg And u _as Identifier, area and autonomous system, s, respectively, representing a user _id 、s _rg Sum s _as An identifier, an area, and an autonomous system, respectively, representing a service;

then adopt one-hot coding to make I _u And I _s The method is converted into a sparse vector consisting of 1 and 0, wherein the sparse vector is used as an initial feature vector of users and services.

Further, the embedded layer designed in the step C performs the following processing on the input features:

E _j ＝σ(W _j X _j +b _j )

wherein X is _j Represents the j-th feature input to the embedded layer, W _j Embedding a weight matrix, b _j Representing the bias term, sigma (), activation function, E _j An embedding vector representing a j-th feature;

then the user's identifier u _id Region u _rg And autonomous system u _as Is concatenated as the embedded vector E of the user _u The method comprises the steps of carrying out a first treatment on the surface of the Identifier s of the service _id Region u _rg And autonomous system u _as Is connected as an embedded vector E of a service _s 。

Further, the interaction layer processes the embedded vector of the user and the embedded vector of the service as follows:

step D1: performing an outer product on the two embedded vectors, thereby explicitly modeling the dimensional correlation, resulting in an interaction graph T:

wherein,representing an outer product operation;

step D2: introducing an attention mechanism, and obtaining an attention characteristic V by performing the following operation on the interaction diagram:

V＝T⊙sigmoid(T)

wherein sigmoid (-) is a sigmoid activation function, by which is meant multiplication element by element;

step D3: performing a Flatten operation on y to obtain a vector M _f Then M is taken up _f The embedded vector of the user is connected with the embedded vector of the service, and the feature vector X is obtained.

Further, in the step E, the feature map M output by the convolutional layer of the last layer of the CNN network is obtained _N And inputting the QoS prediction information into a full connection layer, and finally, fusing the characteristics extracted by the CNN through the full connection layer to complete QoS prediction, wherein the prediction value is calculated as follows:

Q＝σ(W _m M _N +b _m ).

wherein sigma is an identity function, Q is a predicted value of QoS, W _m 、b _m Respectively a weight matrix and a bias term.

The beneficial effects are that: the invention improves the prediction precision of QoS, reduces the reasoning time and energy consumption of QoS prediction under the condition of limited computing resources, and realizes energy-saving and high-quality service.

Drawings

FIG. 1 is a block diagram of a quality of service prediction method according to the present invention;

FIG. 2 is a schematic diagram of a location-aware service invocation scenario.

Detailed Description

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention.

The invention is further described below with reference to the accompanying drawings.

As shown in FIG. 1, the invention provides a dimension-dependent integrated service quality prediction method (QPCN) based on a convolutional neural network, and a prediction model consists of an input layer, an embedded layer, an interaction layer and a prediction layer. Considering the effect of the user and service location on the QoS value, the present embodiment uses one-hot coding at the input layer to represent the location information of the user and service along with the identifier. At the embedded layer, the one-hot coded vector is converted into a low-dimensional dense vector. Through the interaction layer, the dimension correlation of the embedded vector, i.e., the second order interaction feature, is obtained using an outer product operation and distinguished using an attention mechanism. At the prediction layer, the initial feature sequence is changed through a layer of Shuffle operation, and the output result Reshape is formed into a feature map. QoS predictions are then made on the feature map using CNN. The QPCN can also build deeper models, learn higher-order features between embedded dimensions, to improve the prediction accuracy of QoS. The invention considers the position information of the user and the service and the dimension relativity of the embedded vector, and completes the extraction of the high-order characteristics through CNN, thereby realizing accurate service quality prediction, meeting the requirement of recommending high-quality service to the user in service recommendation,

the method specifically comprises the following steps:

step A: acquiring a WSDream quality of service data set: and collecting a WSDream service quality data set according to a service quality prediction target of the service requirement.

According to the preferred technical scheme, based on the service quality data set in the step A, a sparseness degree d% is selected according to a service recommendation scene, and d% of data in the data set is randomly reserved.

And (B) step (B): based on the WSDraam dataset, an input layer is designed to obtain initial feature vectors for users and services.

In the step B, an input layer is designed based on the wsstream data set, and steps B1 to B2 are performed to obtain initial feature vectors of the user and the service obtained by the one-time encoding, respectively.

Step B1: qoS values are affected by the context of the user and service, including network address, network status, subnetwork, autonomous system, geographic location, and the like. Some studies, indicating that QoS values are greatly affected by location, tend to experience better service, lower latency, and faster response times when users invoke services closest to them. Thus, at the input layer, the present embodiment extracts relevant explicit features from the location information, i.e., the areas where the users and services are located, and the autonomous systems to which they belong. Specifically, this embodiment represents initial features of users and services as follows:

I _u ＝[u _id ，u _rg ，u _as ].

I _s ＝[s _id ，s _rg ，s _as ].

wherein u is _id 、u _rg And u _as Representing the user's identifier, area, and autonomous system, respectively. Similarly, s _id 、s _rg Sum s _as An identifier representing a service, an area, and an autonomous system.

Step B2: taking the location-aware service invocation scenario of fig. 2 as an example, it includes multiple users and multiple servers, on which multiple services are deployed, which are located in 6 autonomous systems in 3 regions. Because the number of users, services, autonomous systems, and regions are limited, they can be represented by non-negative integers as unique identifiers, respectively. For example, when user 1 invokes service 4, their initial feature vector may be represented as two vectors [1, 2]And [4,3,6 ]]Which contains the user's identifier and location. For a better representation of the features, the two initial feature vectors are then converted by one-hot encoding into sparse vectors consisting of 1 and 0, i.e Initial characteristics of the user and the service are obtained.

Step C: and designing an embedding layer based on the initial feature vectors of the user and the service, and acquiring the low-dimensional dense embedding vectors of the user and the service.

In the step C, an embedding layer is designed based on initial feature vectors of the user and the service, and a low-dimensional dense embedding vector of the user and the service is obtained to better represent the relationship between different features, thereby being beneficial to training of a network model.

Step C1: because of the large number of users and services, the initial characteristics obtained by the single thermal coding are high-dimensional sparse, and the direct input of the initial characteristics into the neural network is unfavorable for the training of the network. Thus, the present embodiment adds an embedding layer to convert it into a low-dimensional dense vector. Specifically, each feature of the user and service is mapped to the same potential feature space R ^d Where d is the dimension of the potential feature space. The embedded layer can be regarded as a special fully connected layer, the formula of which is as follows:

E _j ＝σ(W _j X _j +b _j )

wherein X is _j Represents the jth feature, W _j Representing its corresponding embedded weight matrix, b _j Represents the bias term, σ represents the activation function, E _j An embedded vector representing the jth feature.

Step C2: thereafter the user's identifier u _id Region u _rg And autonomous system u _as Is embedded in vector e of (a) _uid ，e _urg ，e _uas Embedded vector E connected as a user _u Identifier s of the service will also be _id Region u _rg And autonomous system u _as Is embedded in vector e of (a) _sid ，e _srg ，e _sas Embedded vector E connected as a service _s The formula is as follows:

E _u ＝[e _uid ，e _urg ，e _uas ]

E _s ＝[e _sid ，e _srg ，e _sas ]

the sample embodiment can obtain two low-dimensional dense vectors, reduces the risk of insufficient expression capacity of classification features in one-hot coding, and can better learn the relation between different features.

Step D: the interaction layer is designed based on the embedded vectors of the user and the service. The second-order interaction feature is obtained through embedding vector outer product, and after a layer of attention, the second-order interaction feature is connected with the initial embedded vector to obtain a vector containing first-order and second-order features.

In the step D, the dimensional correlation of the embedded vectors is considered and distinguished by attention, so as to obtain richer interaction features.

Step D1: at the interaction level, the present embodiment processes the embedded vectors, and explicitly models both vectors. Although fully connected MLPs can theoretically fit any continuous function, dimensional correlation is not easily captured. Explicit modeling of the dimensional relevance of embedded vectors of users and services helps to improve the generalization ability of the deep learning model on sparse data. Thus, the present embodiment performs an outer product on the two embedded vectors to explicitly model the dimensional correlation. Interaction graph T epsilon R generated by outer product ^d×d The definition is as follows:

E _u representing the user's embedded vector, E _s Representing the embedded vector of the service.

Step D2: since T is obtained from the outer product of the embedded vectors of the user and the service, it contains many redundant features, and therefore it is necessary to distinguish the importance of these features, a layer of attention mechanism is introduced for this embodiment. Typically, a softmax function is applied to the attention layer, but it is considered that a softmax function sometimes assigns a greater weight to some features and a weight near 0 to other features, i.e., results in one-hot activation. This embodiment can alleviate this situation to some extent by using sigmoid instead of softmax. In the attention layer, T is the input of a sigmoid function, and then the output of the sigmoid function performs element-by-element multiplication with T. This process is defined as follows:

V＝T⊙sigmoid(T).

step D3: in order to preserve the features of the initial embedded vector, the present embodiment performs the Flatten operation on V to obtain a vector M _f It is then connected to the embedded vectors of the user and the service, resulting in a feature vector X comprising the initial embedded vectors and their dimensional dependencies. Thereafter, the feature vector X is fed into the prediction layer.

Step E: and designing a prediction layer, firstly performing a Shuffle operation on an output vector of an interaction layer, forming a feature map by Reshape, then performing feature extraction by CNN, and finally obtaining a prediction value through a full connection layer.

In the step E, the initial feature vector is changed by Shuffle (channel shuffling), so that the influence of the initial fixed feature sequence is reduced, and the CNN is used for feature extraction, so that higher-order interaction features can be extracted.

Step E1: among the prediction layers, the choice of hidden layer can have a large impact on the accuracy of QoS prediction, a common approach being to use fully connected networks. Because a large number of dimension interaction features are generated through outer products at the interaction layer, the use of fully connected MLP can bring a large number of parameters, which is unfavorable for training of the model and easy to cause over fitting. In the embodiment, the CNN has fewer parameters in a parameter sharing and local connection mode, a deeper model than the MLP is easier to construct, and high-order correlation among features is extracted. Therefore, in order to solve the drawbacks of the MLP, the present embodiment uses CNN for feature extraction.

Since the feature vector X is obtained by embedding vectors of users and services and their dimension interaction feature connection, the direct participation model training is affected by the fixed feature sequence, so the initial feature sequence is changed by a layer of Shuffle, the formula is that

X′＝f(W ₁ X+b).

Wherein f represents a Relu activation function, W ₁ Representing a weight matrix, b representing a bias parameter. In this way, the feature arrangement sequence of the vector X is adaptively learned by the Shuffle layer, which is beneficial to the feature extraction of the CNN-based prediction network.

Step E2: next, a Reshape operation is performed on X' to convert it into a matrix of K, denoted M ₀ Then at M ₀ Feature extraction was accomplished above with a multi-layer CNN. In CNN, there are N convolutional layers, and the ith layer has P _i And a convolution kernel. The feature map extracted by the jth convolution kernel of the ith layer may be expressed as follows:

where x represents the convolution operation,the parameter representing the j-th convolution kernel of the i-th layer, bi, is the bias term. The output profile of the i-th layer can be expressed as:

the convolution kernel size adopted by each layer is 2 x 2, and the stride is set to be 2, so that the size of the characteristic diagram is reduced to half of the original size when one layer passes through. To preserve more features, this embodiment does not pool after the convolutional layer. This way, the characteristic diagram of 1 x T is obtained by multi-layer convolution until the last layer is recorded as

And finally, fusing the CNN extracted features through the full connection layer to complete QoS prediction. The predicted values are defined as follows:

Q＝σ(W _m M _N +b _m ).

wherein sigma is an identity function, Q is a predicted value of QoS, W _m 、b _m Respectively a weight matrix and a bias term.

The invention designs a position-aware service quality prediction method. The method consists of an input layer, an embedding layer, an interaction layer and a prediction layer, and QoS prediction is realized through forward propagation. Many studies indicate that QoS values are affected by the location of users and services, so location information and identifiers are thermally encoded singly in the form of classification variables at the input layer. Because the vector obtained by the input layer is high-dimensional sparse and is unfavorable for training of a model, an embedded layer is arranged to convert the high-dimensional sparse vector into a low-dimensional dense vector. Then, the model obtains second-order interaction characteristics of the embedded vector through outer product at the interaction layer, and the importance of the characteristics is distinguished through an attention mechanism. Next, the second order features are concatenated with the initial embedded vector and fed into the prediction layer together. At the prediction layer, the initial feature sequence is changed through a layer of shuffle, and the output result reshape is formed into a feature map. Since convolutional neural networks have better generalization capability and higher-order features are easier to extract, the present embodiment uses CNN to implement QoS prediction on this feature map.

In addition, the specific features described in the above embodiments may be combined in any suitable manner without contradiction. The various possible combinations of the invention are not described in detail in order to avoid unnecessary repetition.

Claims (6)

1. The dimension-related integrated service quality prediction method based on the convolutional neural network is characterized by comprising the following specific steps of:

step A: according to a service quality prediction target of service requirements, a WSDream service quality data set is collected, and the data set is preprocessed;

and (B) step (B): designing an input layer for extracting initial feature vectors of users and services;

step C: designing an embedding layer for converting initial feature vectors of users and services into low-dimensional dense embedding vectors to obtain embedding vectors E of the users _u And embedded vector E of service _s ；

Step D: designing an interaction layer based on an attention mechanism, wherein the interaction layer is used for interacting an embedded vector of a user with an embedded vector of a service so as to obtain a feature vector X;

step E: adopting a characteristic arrangement sequence of a characteristic vector X of a shuffle layer self-adaptive learning, and obtaining a characteristic vector X'; performing a Reshape operation on X 'to convert X' into a matrix form and denoted as M ₀ M is set to ₀ And inputting the service quality into a CNN network, and predicting the service quality through the CNN network.

2. The convolutional neural network-based dimension-dependent integrated quality of service prediction method of claim 1, wherein the preprocessing in step a specifically comprises: and selecting the sparseness degree d according to the service recommendation scene, and randomly reserving d% of data in the data set.

3. The method for predicting the dimension-dependent integrated service quality based on the convolutional neural network as recited in claim 1, wherein the input layer in the step B extracts relevant explicit features from the regional information of the user and the service to obtain the feature vector I of the user _u And feature vector I of service _s ：

I _u ＝[u _id ，u _rg ，u _as ].

I _s ＝[s _id ，s _rg ，s _as ].

Wherein u is _id 、u _rg And u _as Identifier, area and autonomous system, s, respectively, representing a user _id 、s _rg Sum s _as An identifier, an area, and an autonomous system, respectively, representing a service;

then adopt one-hot coding to make I _u And I _s Conversion to a sparse vector consisting of 1 and 0 as initial feature directions for users and servicesAmount of the components.

4. The convolutional neural network-based dimension-dependent integrated quality of service prediction method of claim 2, wherein the embedding layer designed in step C performs the following processing on the input features:

E _j ＝σ(W _j X _j +b _j )

wherein X is _j Represents the j-th feature input to the embedded layer, W _j Embedding a weight matrix, b _j Representing the bias term, sigma (), activation function, E _j An embedding vector representing a j-th feature;

then the user's identifier u _id Region u _rg And autonomous system u _as Is concatenated as the embedded vector E of the user _u The method comprises the steps of carrying out a first treatment on the surface of the Identifier s of the service _id Region u _rg And autonomous system u _as Is connected as an embedded vector E of a service _s 。

5. The convolutional neural network-based dimension-dependent integrated quality of service prediction method of claim 1, wherein the interaction layer processes the embedded vector of the user and the embedded vector of the service as follows:

step D1: performing an outer product on the two embedded vectors, thereby explicitly modeling the dimensional correlation, resulting in an interaction graph T:

wherein,representing an outer product operation;

step D2: introducing an attention mechanism, and obtaining an attention characteristic V by performing the following operation on the interaction diagram:

V＝T⊙sigmoid(T)

wherein sigmoid (-) is a sigmoid activation function, by which is meant multiplication element by element;

step D3: performing a Flatten operation on y to obtain a vector M _f Then M is taken up _f The embedded vector of the user is connected with the embedded vector of the service, and the feature vector X is obtained.

6. The method for predicting the dimension-dependent integrated service quality based on the convolutional neural network according to claim 1, wherein in the step E, the feature map M output by the convolutional layer of the last layer of the CNN network is obtained _N And inputting the QoS prediction information into a full connection layer, and finally, fusing the characteristics extracted by the CNN through the full connection layer to complete QoS prediction, wherein the prediction value is calculated as follows:

Q＝σ(W _m M _N +b _m ).

wherein sigma is an identity function, Q is a predicted value of QoS, W _m 、b _m Respectively a weight matrix and a bias term.