CN116910403A

CN116910403A - Page anomaly analysis method and big data system based on artificial intelligence

Info

Publication number: CN116910403A
Application number: CN202310217435.0A
Authority: CN
Inventors: 亢朝侠; 方媛
Original assignee: Individual
Current assignee: Individual
Priority date: 2022-09-15
Filing date: 2022-09-15
Publication date: 2023-10-20
Also published as: CN115422486B; CN115422486A

Abstract

The embodiment of the application provides an artificial intelligence-based page anomaly analysis method and a big data system, which can obtain first training cost according to decision page optimization distribution data and actual page optimization distribution data associated with corresponding page optimization training data after loading the page optimization training data each time in the training process of a page optimization distribution model, then adjust the first training cost based on a first training optimization index to determine the target training cost of the page optimization distribution model, and can improve the global distinction degree of information generated by models of each two model processing units in the target page optimization distribution model after training, so as to improve the model accuracy of the target page optimization distribution model, thereby improving the accuracy of page optimization distribution results.

Description

Page anomaly analysis method and big data system based on artificial intelligence

The application relates to a division application of China application with an application number 202211118653.0, an application date of 2022, 09 and 15, and an application and creation name of an artificial intelligence-based cloud service online page optimization method and a big data system.

Technical Field

The invention relates to the technical field of big data, in particular to an artificial intelligence-based page anomaly analysis method and a big data system.

Background

Currently, the construction of internet informatization has achieved excellent results, and various internet service providers use a cloud service platform to provide a wide variety of online services for users. Cloud service is provided for the Internet service provider through the cloud service platform, so that the Internet service provider can reduce the operation cost and improve the management efficiency.

However, for each internet service provider, the stability of the online page of the cloud service is related to the experience of the user, and once an abnormal event (such as a page crash event, a page abnormal jump event, etc.) occurs on the online page of the cloud service, the online service is interrupted, so that the internet service provider generally configures corresponding emergency optimization measures for the page abnormal event in advance so that the online page service can be repaired and run continuously in time. In the related art, when deciding the page optimization scheme, the page optimization scheme is usually matched only according to the abnormal fields, however, the final page optimization scheme is determined according to the matching quantity, and the accuracy of the page optimization allocation result of the idea is lower.

Disclosure of Invention

In order to at least overcome the defects in the prior art, the application aims to provide an artificial intelligence-based page anomaly analysis method and a big data system.

In a first aspect, the present application provides an artificial intelligence based cloud service online page optimization method, which is applied to a big data system, wherein the big data system is in communication connection with a plurality of page servers, and the method comprises:

extracting features of candidate page abnormal events of the cloud service online page, obtaining page abnormal path features of the candidate page abnormal events, and respectively obtaining page optimization logic knowledge point features of each page optimization scheme data to be distributed;

respectively loading the determined page abnormal path characteristics and each page optimization logic knowledge point characteristic into a target page optimization distribution model meeting model convergence conditions, and generating respective page optimization distribution support degree of each page optimization scheme data to be distributed;

the training step of the target page optimization distribution model comprises the following steps:

performing traversal model weight parameter optimization of a plurality of training stages on a page optimization distribution model for initializing model weight parameters according to a page optimization training data sequence until the model weight parameters of the page optimization distribution model are not changed, and taking the page optimization distribution model generated in the last training stage as a target page optimization distribution model, wherein the page optimization distribution model comprises a plurality of model processing units, each model processing unit is used for extracting characteristics of page optimization training data loaded into the page optimization distribution model from a page optimization label, and the following steps are executed in one traversal training stage:

Loading the page optimization training data acquired from the page optimization training data sequence to the page optimization distribution model, and determining decision page optimization distribution data corresponding to the page optimization training data;

determining corresponding first training cost based on decision page optimization allocation data and actual page optimization allocation data corresponding to the page optimization training data;

determining corresponding target training cost according to the first training cost and the first training optimization index; the first training optimization index is used for representing the global distinction of model generation information of each two model processing units determined according to corresponding page optimization training data, and the first training optimization index is inversely related to the target training cost;

and carrying out traversal model weight parameter optimization on the page optimization distribution model based on the target training cost.

In a possible implementation manner of the first aspect, the page optimization allocation support degree of each page optimization scheme data to be allocated in the respective page optimization scheme data to be allocated is: page optimization allocation support of a target page optimization label is preset;

The determined page abnormal path characteristics and each page optimization logic knowledge point characteristic are respectively loaded into a target page optimization distribution model meeting model convergence conditions, page optimization distribution support degree of each page optimization scheme data to be distributed is generated, and each page optimization distribution support degree of each page optimization scheme data to be distributed is generated, and the following steps are executed:

if the generated page optimization allocation support degree is larger than the first set support degree, taking the corresponding page optimization scheme data to be allocated as target page optimization scheme data;

sequentially sorting the determined target page optimization scheme data according to the respective page optimization allocation support degree, and determining a sequence sorting result;

and acquiring final selected target page optimization scheme data fed back by the target cloud service page based on the sequence arrangement result, and performing page optimization on the cloud service online page based on target restoration firmware data corresponding to the final selected target page optimization scheme data in a cloud restoration firmware library.

In a possible implementation manner of the first aspect, the page optimization allocation support degree of each page optimization scheme data to be allocated in the respective page optimization scheme data to be allocated includes page optimization allocation support degrees of page optimization labels for a plurality of preset pages;

The determined page abnormal path characteristics and the determined page optimization logic knowledge point characteristics are respectively loaded into a target page optimization distribution model meeting model convergence conditions, and in the process of generating the page optimization distribution support degree of each page optimization scheme data to be distributed, each page optimization distribution support degree of each page optimization scheme data to be distributed is generated, and one of the following steps is executed:

if one of the page optimization allocation support degrees of the generated plurality of page optimization labels is larger than the corresponding second setting support degree, taking corresponding page optimization scheme data to be allocated as target page optimization scheme data;

if a plurality of page optimization allocation support degrees of the generated plurality of page optimization labels are respectively larger than the respective third setting support degrees, taking corresponding page optimization scheme data to be allocated as target page optimization scheme data;

and carrying out order arrangement on the determined target page optimization scheme data according to page optimization allocation support degrees of the page optimization labels.

In a possible implementation manner of the first aspect, the loading the page optimization training data obtained from the page optimization training data sequence to the page optimization allocation model, and determining decision page optimization allocation data corresponding to the page optimization training data includes:

Loading the page optimization training data acquired from the page optimization training data sequence to the page optimization distribution model, respectively determining respective model generation information of the plurality of model processing units and respective corresponding influence coefficients of the plurality of model processing units, and determining decision page optimization distribution data corresponding to the page optimization training data based on the determined respective model generation information and the corresponding influence coefficients;

the first training optimization index is determined based on the following steps:

and respectively determining first distinguishing metric values between model generation information of each two model processing units in the plurality of model processing units, and determining the first training optimization index according to each determined first distinguishing metric value.

In a possible implementation manner of the first aspect, the model generation information of each of the plurality of model processing units generates a feature; the determining, respectively, a first difference metric value between model generation information corresponding to each two model processing units in the plurality of model processing units, and determining, according to each determined first difference metric value, the first training optimization index includes:

Respectively acquiring first feature deviation degrees between generated features corresponding to each two model processing units in the plurality of model processing units, and respectively taking the determined first feature deviation degrees as corresponding first distinguishing metric values;

weighting the determined first discrimination metric values to determine the first training optimization index.

In a possible implementation manner of the first aspect, the decision page optimization allocation data includes page optimization allocation support degrees for a preset plurality of page optimization tags;

the loading the page optimization training data obtained from the page optimization training data sequence to the page optimization distribution model, respectively determining respective model generation information of the plurality of model processing units and respective corresponding influence coefficients of the plurality of model processing units, and determining decision page optimization distribution data corresponding to the page optimization training data based on the determined respective model generation information and the corresponding influence coefficients, wherein the decision page optimization distribution data comprises:

loading the page optimization training data acquired from the page optimization training data sequence to the page optimization distribution model, respectively determining the model generation information of each of the plurality of model processing units, and obtaining the influence coefficients of each of the plurality of model processing units under the plurality of page optimization labels;

Based on the influence coefficients of the plurality of model processing units under the plurality of page optimization tags, respectively determining influence coefficient characteristics corresponding to the plurality of page optimization tags, wherein each characteristic member contained in the influence coefficient characteristics corresponding to each page optimization tag corresponds to the influence coefficient corresponding to the plurality of model processing units under the corresponding page optimization tag one by one;

optimizing labels for the plurality of pages, respectively executing the following steps: and determining the page optimization distribution support degree of the page optimization training data corresponding to one page optimization label based on the influence coefficient characteristics corresponding to the page optimization label and the model generation information of each model processing unit.

In a possible implementation manner of the first aspect, the page optimization allocation model further includes a plurality of allocation units, each allocation unit being configured to obtain a page optimization allocation support of a page optimization tag;

the determining the page optimization allocation support degree of the page optimization training data corresponding to the page optimization label based on the influence coefficient characteristics corresponding to the page optimization label and the model generation information of each of the plurality of model processing units comprises the following steps:

Each feature member in the influence coefficient features corresponding to the page optimization tag is respectively aggregated with the model generation information of the corresponding model processing unit to obtain final model generation information corresponding to the page optimization tag;

and loading the final model generation information corresponding to the one page optimization label to a corresponding distribution unit, and determining the page optimization distribution support degree corresponding to the page optimization training data under the one page optimization label.

In a possible implementation manner of the first aspect, after the determining the influence coefficient features corresponding to the respective page optimization tags, before determining the corresponding target training cost according to the first training cost and the first training optimization index, the method further includes:

respectively determining second characteristic deviation degrees between influence coefficient characteristics corresponding to each two page optimization labels in the plurality of page optimization labels, and taking the determined second characteristic deviation degrees as corresponding second distinguishing metric values;

weighting the determined second discrimination metric values to determine a second training optimization index; the second training optimization index is used for representing global distinction of influence coefficient characteristics of a plurality of model processing units corresponding to each two page optimization labels, and the second training optimization index is inversely related to the target training cost;

The determining the corresponding target training cost according to the first training cost and the first training optimization index includes:

and determining the target training cost according to the first training cost, the first training optimization index and the second training optimization index.

In a possible implementation manner of the first aspect, the step of extracting features of the candidate page abnormal event of the online page of the cloud service to obtain the page abnormal path feature of the candidate page abnormal event includes:

performing anomaly code analysis on the candidate page anomaly event based on a target page anomaly analysis model to determine corresponding first anomaly code analysis information, and determining first anomaly confidence degrees of the candidate page anomaly event corresponding to at least one derivative page running path and a plurality of initial page running paths respectively by combining the first anomaly code analysis information;

updating first abnormal confidence degrees corresponding to the at least one derivative page running path respectively based on the target page abnormality analysis model and on abnormality prior information to determine second abnormal confidence degrees corresponding to the at least one derivative page running path respectively, wherein the plurality of initial page running paths and the at least one derivative page running path are generated based on page running configuration data of different page service modes respectively, the page service priority of the page running configuration data corresponding to the initial page running paths is higher than the page service priority of the page running configuration data corresponding to the derivative page running paths, and each initial page running path are different;

Combining the first abnormal confidence degrees respectively corresponding to the plurality of initial page running paths and the second abnormal confidence degrees respectively corresponding to the at least one derivative page running path, and fusing and outputting a target page running path corresponding to the candidate page abnormal event from the plurality of initial page running paths and the at least one derivative page running path, and determining the path characteristics reflected by the target page running path as the page abnormal path characteristics of the candidate page abnormal event;

the model configuration step of the target page anomaly analysis model comprises the following steps:

inputting each template derived page abnormal event in the first template page abnormal event cluster into an initial page abnormal analysis model, further determining a corresponding second derived abnormal confidence level of the corresponding template derived page abnormal event in the at least one derived page running path based on the initial page abnormal analysis model, and determining corresponding second initial abnormal confidence levels of the corresponding template derived page abnormal event in the plurality of initial page running paths;

inputting each template initial page abnormal event in the second template page abnormal event cluster into the initial page abnormal analysis model, further determining a third derived abnormal confidence level of the corresponding template initial page abnormal event corresponding to the at least one derived page running path based on the initial page abnormal analysis model, and determining a third initial abnormal confidence level of the corresponding template initial page abnormal event corresponding to the plurality of initial page running paths;

Determining a first dimension loss function value by combining the second derived anomaly confidence level, the second initial anomaly confidence level, the third derived anomaly confidence level and the third initial anomaly confidence level;

combining the second initial anomaly confidence level and the third initial anomaly confidence level to determine a second dimension loss function value;

performing weighted calculation on the first dimension loss function value and the second dimension loss function value, determining a corresponding target loss function value, performing model iterative updating on the initial page abnormal analysis model based on the target loss function value, and determining a fuzzy page abnormal analysis model corresponding to the initial page abnormal analysis model when the loss function value calculated in a model iterative updating flow is smaller than a set value, wherein the first template page abnormal event cluster corresponds to at least one derivative page running path, and the second template page abnormal event cluster corresponds to a plurality of initial page running paths;

collecting a plurality of updating page running configuration data, inputting each updating page running configuration data into the fuzzy page abnormality analysis model, further carrying out page abnormality path analysis based on the fuzzy page abnormality analysis model, determining a first derivative abnormality confidence level corresponding to the corresponding updating page running configuration data in the at least one derivative page running path, and determining a first initial abnormality confidence level corresponding to the corresponding updating page running configuration data in the plurality of initial page running paths, wherein each updating page running configuration data corresponds to one derivative page running path or one initial page running path;

Determining derivative priori abnormal parameters corresponding to the at least one derivative page running path by combining the first derivative abnormal confidence coefficient;

determining initial priori abnormal parameters corresponding to the multiple initial page running paths by combining the first initial abnormal confidence degrees;

determining corresponding abnormality prior information based on the derived priori abnormality parameters and the initial priori abnormality parameters;

and configuring the abnormality prior information in a full-connection unit of the fuzzy page abnormality analysis model, and outputting a corresponding target page abnormality analysis model, wherein the abnormality prior information is used for updating derivative abnormality confidence corresponding to the at least one derivative page running path.

In a second aspect, the embodiment of the application also provides an artificial intelligence-based cloud service online page optimization system, which comprises a big data system and a plurality of page servers in communication connection with the big data system;

the big data system is used for:

In any aspect, in the training process of the page optimization allocation model, after the page optimization training data is loaded each time, the first training cost can be obtained according to decision page optimization allocation data and actual page optimization allocation data associated with the corresponding page optimization training data, and then the first training cost is adjusted based on the first training optimization index, so that the target training cost of the page optimization allocation model is determined.

The first training optimization index represents the global distinction degree of the model generation information of each two model processing units determined by the corresponding page optimization training data, and the first training optimization index is inversely related to the target training cost, so that when the first training optimization index is larger, the global distinction degree of the model generation information of each two model processing units is larger, and the target training cost is smaller; when the first training optimization index becomes smaller, the global distinction degree of the model generation information of each two model processing units becomes smaller, and then the target training cost becomes larger. Therefore, the global distinction degree of the model generation information of each two model processing units in the target page optimal allocation model after training can be improved, so that the model accuracy of the target page optimal allocation model is improved, and the accuracy of the page optimal allocation result is improved.

Drawings

Fig. 1 is a schematic flow chart of an artificial intelligence-based cloud service online page optimization method according to an embodiment of the present invention.

Detailed Description

The architecture of an artificial intelligence based cloud service online page optimization system 10 provided in accordance with one embodiment of the present invention is described below, the artificial intelligence based cloud service online page optimization system 10 may include a big data system 100 and a page server 200 communicatively coupled to the big data system 100. Wherein, the big data system 100 and the page server 200 in the cloud service online page optimization system 10 based on artificial intelligence can be combined to cooperate to execute the cloud service online page optimization method based on artificial intelligence described in the following method embodiments, and the execution steps of the big data system 100 and the page server 200 can be referred to in the following method embodiments.

The cloud service online page optimization method based on artificial intelligence provided in this embodiment may be executed by the big data system 100, and is described in detail below with reference to fig. 1.

And (3) processing S101, acquiring a page optimization training data sequence.

In some possible embodiments, each page optimization training data in the page optimization training data sequence includes a reference page anomaly path feature of a reference page anomaly event, and a reference page optimization logic knowledge point feature of reference page optimization scheme data, where the reference page anomaly path feature may include a path formed by operation flow features of various page applications when a page is abnormal (such as crashed, page destroyed, etc.), and the reference page optimization logic knowledge point feature may include an optimization path feature of the reference page optimization scheme data (such as an optimization logic knowledge point for a page optimization element (such as a page function control, etc.) in each page, etc.

And the processor S102 performs traversal model weight parameter optimization of a plurality of training stages on the page optimization distribution model for initializing the model weight parameters according to the page optimization training data sequence.

For example, in one traversal training phase in the process 102, the following steps may be performed:

processing S1021, loading the page optimization training data obtained from the page optimization training data sequence to a page optimization distribution model, and determining decision page optimization distribution data corresponding to the page optimization training data.

For example, the process 1021 may be implemented by the following scheme:

the processing S10211 loads the page optimization training data acquired from the page optimization training data sequence to a page optimization distribution model, respectively determines model generation information of each of a plurality of model processing units and influence coefficients corresponding to each of the plurality of model processing units, and determines decision page optimization distribution data corresponding to the page optimization training data based on the determined model generation information and the corresponding influence coefficients.

The multiple model processing units of the page optimization distribution model can adopt an AI model and are respectively used for extracting the characteristics of the input page optimization training data from the corresponding page optimization tags so as to realize the characteristic extraction of the multiple page optimization tags.

In some possible embodiments, when the page optimization distribution model adopts a multi-model processing unit to learn the network, a decision can be made for one model processing unit, and the one model processing unit can be understood as making a confidence decision from a preset target page optimization label; the decision can also be made for a plurality of model processing units at the same time, which can be understood as making confidence decisions from a preset plurality of page optimization tags. When confidence decisions are made for the plurality of model processing units, the influence coefficients corresponding to the plurality of model processing units determined in process s10021 include influence coefficients corresponding to the plurality of model processing units.

For example, the plurality of model processing units include a model processing unit 1, a model processing unit 2, and a model processing unit 3, and for the model processing unit 1 and the model processing unit 2, the influence coefficient corresponding to the model processing unit 1 includes an influence coefficient a1 of the model processing unit 1 and an influence coefficient a2 of the model processing unit 2, the influence coefficient corresponding to the model processing unit 2 includes an influence coefficient b1 of the model processing unit 1 and an influence coefficient b2 of the model processing unit 2, and the influence coefficient corresponding to the model processing unit 3 includes an influence coefficient c1 of the model processing unit 1 and an influence coefficient c2 of the model processing unit 2. That is, the influence coefficients of the model processing unit 1 to the model processing unit 1, the model processing unit 2, and the model processing unit 3 are a1, b1, and c1, respectively, and the influence coefficients of the model processing unit 2 to the model processing unit 1, the model processing unit 2, and the model processing unit 3 are a2, b2, and c2, respectively.

In some possible implementations, the page optimization allocation model further includes a plurality of recurrent neural networks, a plurality of aggregation units, and a plurality of decision units; each cyclic neural network is used for generating an influence coefficient characteristic corresponding to a corresponding page optimization label; each aggregation unit is used for aggregating the model generation information of each of the plurality of model processing units based on the influence coefficient characteristics corresponding to the corresponding page optimization label; each allocation unit is used for obtaining page optimization allocation support of a corresponding page optimization tag. Wherein each recurrent neural network may employ, but is not limited to, a Softmax function, and each distribution unit may employ a deep learning network, including, but not limited to, a feed forward neural network, a convolutional neural network, and the like.

Therefore, when the page optimization training data is loaded to the page optimization distribution model, the page optimization training data can be simultaneously loaded to the plurality of model processing units and the plurality of cyclic neural networks so as to respectively determine model generation information of each of the plurality of model processing units and influence coefficients corresponding to each of the plurality of model processing units.

In some possible embodiments, when the page optimization allocation model makes decisions for multiple model processing units at the same time, the decision page optimization allocation data includes page optimization allocation support for a preset multiple page optimization tags, for example, the process 10211 may be implemented by the following scheme:

The processing S10211_1 loads the page optimization training data acquired from the page optimization training data sequence to a page optimization distribution model, respectively determines the model generation information of each of the plurality of model processing units, and obtains the influence coefficients of each of the plurality of model processing units under the plurality of page optimization tags.

For example, the obtained page optimization training data may be simultaneously loaded to a plurality of model processing units and a plurality of cyclic neural networks of the page optimization distribution model, to determine model generation information of each of the plurality of model processing units and influence coefficients of each of the plurality of model processing units under a plurality of page optimization tags, respectively.

And the ProcessS10211_2 is used for respectively determining influence coefficient characteristics corresponding to each of the plurality of page optimization tags based on the influence coefficients of each of the plurality of model processing units under the plurality of page optimization tags, wherein each characteristic member contained in the influence coefficient characteristics corresponding to each page optimization tag is respectively in one-to-one correspondence with the influence coefficient corresponding to each of the plurality of model processing units under the corresponding page optimization tag.

For example, taking 2 page optimization tags and 3 model processing units as examples, for the model processing unit 1, the model processing unit 2 and the model processing unit 3, the influence coefficients of the model processing unit 1 under the page optimization tag 1 and the page optimization tag 2 are respectively K1 and K1', the influence coefficients of the model processing unit 2 under the page optimization tag 1 and the page optimization tag 2 are respectively K2 and K2', the influence coefficients of the model processing unit 3 under the page optimization tag 1 and the page optimization tag 2 are respectively K3 and K3', the influence coefficients corresponding to the page optimization tag 1 are { K1, K2 and K3}, and the influence coefficients corresponding to the page optimization tag 2 are { K1', K2 'and K3' }.

The processing s10211_3 optimizes the tag for a plurality of pages, and performs the following steps: and determining the page optimization allocation support degree corresponding to the page optimization training data under one page optimization label based on the influence coefficient characteristics corresponding to the one page optimization label and the model generation information of each of the plurality of model processing units.

In some embodiments, the page optimization allocation model includes a plurality of allocation units, each allocation unit configured to obtain a page optimization allocation support of a page optimization tag;

based on the influence coefficient characteristics corresponding to one page optimization label and the model generation information of each of the plurality of model processing units, the processing s10211_3 determines the page optimization allocation support degree corresponding to the page optimization training data under the one page optimization label, and the processing s10211_3 can be realized by the following scheme:

a1, aggregating each characteristic member in the influence coefficient characteristics corresponding to one page optimization label with the model generation information of the corresponding model processing unit respectively to obtain final model generation information corresponding to one page optimization label.

A2, loading final model generation information corresponding to one page optimization label to a corresponding distribution unit, and determining page optimization distribution support degree corresponding to the page optimization training data under the one page optimization label.

For example, the page optimization assignment model makes confidence decisions for 2 page optimization tags, and for model processing unit 1, model processing unit 2, and model processing unit 3, it is assumed that the model coding feature generated by model processing unit 1 is D1, the model coding feature generated by model processing unit 2 is D2, the model coding feature generated by model processing unit 3 is D3, that is, the model coding features generated by model processing unit 1-model processing unit 3 are { D1, D2, D3}. The influence coefficient generated by the cyclic neural network 1 on the model processing unit 1 is K1, the influence coefficient generated by the cyclic neural network 2 on the model processing unit 2 is K2, the influence coefficient generated by the cyclic neural network 3 on the model processing unit 3 is K3, namely the influence coefficient generated by the cyclic neural network 1 on the model processing unit 1-model processing unit 3 is { K1, K2 and K3}, and similarly, the influence coefficient generated by the cyclic neural network 2 on the model processing unit 1-model processing unit 3 is { K1', K2' and K3 '.

Further, the model coding feature with weight obtained by aggregation of { D1, D2, D3} and { K1, K2, K3} by the aggregation unit 1 is { K1D1, K2D2, K3D3}, and is the loading to feature of the distribution unit 1; the model coding feature with weight obtained by the aggregation unit 2 for { D1, D2, D3} and { K1', K2', K3'} is { K1' D1, K2'D2, K3' D3}, which is the loading to feature of the allocation unit 2. And further, the page optimization distribution support degree corresponding to the page optimization training data under a plurality of page optimization labels is respectively determined.

And the processing S1022 determines a corresponding first training cost based on the decision page optimization allocation data and the actual page optimization allocation data corresponding to the page optimization training data.

When the decision page optimization allocation data includes page optimization allocation support for a plurality of page optimization tags, for example, page optimization tag 1, page optimization tag 2, page optimization tag 3, the first training cost includes a sum of the training cost under page optimization tag 1, the training cost under page optimization tag 2, and the training cost under page optimization tag 3.

Processing S1023, determining corresponding target training cost according to the first training cost and the first training optimization index; the first training optimization index is used for representing the global distinction of model generation information of each two model processing units determined according to corresponding page optimization training data, and the first training optimization index is inversely related to the target training cost.

In some possible embodiments, the first training optimization index may be determined based on model generation information of each of the plurality of model processing units determined in the above-mentioned process s 10211. Therefore, after executing the above-described process 10211, before executing the process 1023, the following steps may also be performed:

B. And respectively determining first distinguishing metric values between model generation information of each two model processing units in the plurality of model processing units, and determining a first training optimization index according to each determined first distinguishing metric value.

For example, the above step B may be implemented by the following scheme:

b1, respectively acquiring first feature deviation degrees between generated features corresponding to each two model processing units in the plurality of model processing units, and respectively taking the determined first feature deviation degrees as corresponding first distinguishing metric values.

In some possible embodiments, the first feature deviation degree between the generated features corresponding to each two model processing units may represent a distinction degree between the generated features corresponding to each two model processing units, where the larger the first feature deviation degree, the larger the description distinction degree, and thus the first feature deviation degree may be used as the first distinction degree value.

And B2, weighting the determined first difference metric values to determine a first training optimization index.

For example, for 3 model processing units, the first difference metric value of the generation feature of the model processing unit 1 and the generation feature of the model processing unit 2 is n1, the first difference metric value of the generation feature of the model processing unit 1 and the generation feature of the model processing unit 3 is n2, the first difference metric value of the generation feature of the model processing unit 2 and the generation feature of the model processing unit 3 is n3, and the first training optimization index m1=n1+n2+n3.

And processing S1024, performing traversal model weight parameter optimization on the page optimization allocation model based on the target training cost.

And the processing S103 determines a target page optimization allocation model when judging that the page optimization allocation model matches with the training termination condition.

For example, the training termination condition may be that the number of times the model weight parameter is optimized reaches a set number of times, or that the target training cost is less than the set training cost value.

In some possible embodiments, in the training process of the page optimization allocation model, after loading the page optimization training data each time, the first training cost may be obtained according to decision page optimization allocation data and actual page optimization allocation data associated with the corresponding page optimization training data, and then the first training cost is adjusted based on the first training optimization index, so as to determine the target training cost of the page optimization allocation model.

In some possible embodiments, the page optimization distribution model adopts a multi-model processing unit learning model, and when the multi-model processing unit learning model trains a plurality of page optimization labels, that is, trains a plurality of model processing units, influence coefficients of a plurality of model processing units corresponding to the page optimization labels are different, so that the decision accuracy of the page optimization labels can be improved. However, in the related art, when a plurality of model processing units for which model learning is performed by a multi-model processing unit, there is a problem in that influence coefficients of a plurality of model processing units corresponding to the respective plurality of model processing units are converged.

In some possible embodiments, after determining the influence coefficient features corresponding to each of the plurality of page optimization labels in the process 10211_2 and before determining the corresponding target training cost according to the first training cost and the first training optimization index in the process 1023, the method may further be implemented by the following scheme:

the processing S1023_0 respectively determines second distinguishing metric values between the influence coefficient characteristics corresponding to every two page optimizing labels in a plurality of preset page optimizing labels, and determines a second training optimizing index according to the determined second distinguishing metric values; the second training optimization index is used for representing the global distinction of the influence coefficient characteristics of the plurality of model processing units corresponding to each two page optimization labels, which are determined according to the corresponding page optimization training data, and the second training optimization index is inversely related to the target training cost;

For example, in the above-mentioned processing s1023, according to the first training cost and the first training optimization index, determining the corresponding target training cost may be implemented by the following scheme:

and the processing S10231 determines the target training cost according to the first training cost, the first training optimization index and the second training optimization index.

In some possible embodiments, when the page optimization distribution model trains for a plurality of page optimization labels, the degree of distinction of the influence coefficients of a plurality of model processing units corresponding to the page optimization labels can be ensured, so that the decision accuracy of the distribution support degree of the page optimization labels is improved.

In some possible embodiments, the processing s10231 determines second difference metric values between the influence coefficient features corresponding to each two page optimization tags in the plurality of page optimization tags, and determines a second training optimization index according to the determined second difference metric values, where the second training optimization index may be implemented by the following scheme:

and C1, respectively determining second characteristic deviation degrees between influence coefficient characteristics corresponding to each two page optimization labels in the plurality of page optimization labels, and taking each determined second characteristic deviation degree as a corresponding second distinguishing metric value.

In some possible embodiments, the second feature deviation degree between the influence coefficient features corresponding to each two page optimization tags may represent a degree of distinction between the influence coefficient features corresponding to each two page optimization tags, where the larger the second feature deviation degree, the larger the description distinction degree, and thus the second feature deviation degree may be taken as the second distinction degree value.

And C2, weighting the determined second distinguishing metric values to determine a second training optimization index.

For example, for 3 page-optimized tags, the second differential metric value of the influence coefficient feature corresponding to the page-optimized tag 1 and the influence coefficient feature corresponding to the page-optimized tag 2 is n1', the second differential metric value of the influence coefficient feature corresponding to the page-optimized tag 1 and the influence coefficient feature corresponding to the page-optimized tag 3 is n2', the second differential metric value of the influence coefficient feature corresponding to the page-optimized tag 2 and the influence coefficient feature corresponding to the page-optimized tag 3 is n3', and the second training optimization index m2=n1' +n2'+n3'.

An embodiment of a practical application method combined with the above training procedure is further described below with reference to fig. 1.

The method comprises the steps of processing 201, extracting characteristics of candidate page abnormal events of a cloud service online page, obtaining page abnormal path characteristics of the candidate page abnormal events, and respectively obtaining page optimization logic knowledge point characteristics of each page optimization scheme data to be distributed.

Reference may be made to the explanation of the previous embodiments with respect to page exception path features and page optimization logic knowledge point features. The cloud service online page may refer to an online image search cloud service page, and the like.

And the Process202 loads the determined page abnormal path characteristics and each page optimization logic knowledge point characteristic into a target page optimization distribution model meeting the model convergence condition respectively, and generates the page optimization distribution support degree of each page optimization scheme data to be distributed.

The target page optimization allocation model can be determined based on the method for training the page optimization allocation model, namely, the page optimization allocation model for initializing the model weight parameters is obtained by training according to the page optimization training data sequence; the target training cost determined after each time of loading the page optimization training data is determined at least based on the first training cost and a first training optimization index, wherein the first training cost is determined according to decision page optimization distribution data and actual page optimization distribution data associated with the corresponding page optimization training data; the page optimization distribution model comprises a plurality of model processing units, each model processing unit is used for extracting characteristics of page optimization training data loaded to the page optimization distribution model from one page optimization label, a first training optimization index is used for representing global distinction of model generation information of each two model processing units determined according to the corresponding page optimization training data, and the first training optimization index is inversely related to target training cost.

In some possible embodiments, the page optimization allocation support degree of each page optimization scheme data to be allocated in each page optimization scheme data to be allocated is: and optimizing page optimization allocation support of labels aiming at preset target pages.

The determined page abnormal path characteristics and the page optimization logic knowledge point characteristics are respectively loaded into a target page optimization distribution model meeting the model convergence condition in the Process of generating the page optimization distribution support degree of each page optimization scheme data to be distributed, and each page optimization distribution support degree of each page optimization scheme data to be distributed can be generated, the following steps can be executed:

and if the generated page optimization allocation support degree is larger than the first set support degree, taking the corresponding page optimization scheme data to be allocated as target page optimization scheme data.

The page optimization allocation support degree can represent the confidence degree of loading the page optimization scheme data to be allocated to the target cloud service page, the page optimization allocation support degree is in direct proportion to the confidence degree of loading the page optimization scheme data to be allocated to the target cloud service page, namely, when the page optimization allocation support degree is larger, the confidence degree of loading the page optimization scheme data to be allocated to the target cloud service page is larger, and when the page optimization allocation support degree is smaller, the confidence degree of loading the page optimization scheme data to be allocated to the target cloud service page is smaller.

For example, the first setting support degree may be set in advance, and the first setting support degree may be set based on the need, for example, may be 0.8. When the page optimization allocation support degree of the target cloud service page is greater than the first set support degree, the corresponding page optimization scheme data to be allocated can be used as target page optimization scheme data, the target page optimization scheme data are further input into the target cloud service page, or the target page optimization scheme data are used as candidate page optimization scheme data, and information for loading the target cloud service page is selected from the candidate page optimization scheme data.

Further, the determined target page optimization scheme data are sequentially sorted according to the respective page optimization allocation support degree, and then the page optimization scheme data input to the target cloud service page are selected based on the sequence sorting result.

For example, according to the descending order of page optimization allocation support, the data of each target page optimization scheme is ordered; or according to the order of page optimization allocation support degree from small to large, sequentially sorting the data of each target page optimization scheme.

In other embodiments, the page-optimal allocation support of each of the respective page-optimal allocation scheme data includes page-optimal allocation support for a preset plurality of page-optimal tags.

In some possible embodiments, the target page optimization allocation model may make decisions about allocation support for multiple page optimization tags at the same time, that is, make decisions about allocation support for multiple page optimization tags at the same time.

The determined page abnormal path characteristics and the page optimization logic knowledge point characteristics are respectively loaded into a target page optimization distribution model meeting model convergence conditions in the Process of generating the page optimization distribution support degree of each page optimization scheme data to be distributed, and each page optimization distribution support degree of each page optimization scheme data to be distributed can be generated, one of the following steps can be executed:

1. and if one of the page optimization allocation support degrees of the generated plurality of page optimization labels is larger than the corresponding second setting support degree, taking the corresponding page optimization scheme data to be allocated as target page optimization scheme data.

2. And if a plurality of page optimization allocation supporters of the generated plurality of page optimization labels are respectively larger than the respective third setting supporters, taking the corresponding page optimization scheme data to be allocated as target page optimization scheme data.

For example, the target page optimization allocation model simultaneously decides allocation support of 3 page optimization tags, as may be set to: the allocation support degree of any one page optimization label is larger than the corresponding set support degree, and the corresponding page optimization scheme data to be allocated can be used as target page optimization scheme data; as another example, it can be set as follows: the allocation support degree of any two page optimization labels is respectively larger than the corresponding setting support degree, and the corresponding page optimization scheme data to be allocated can be used as target page optimization scheme data; as another example, it can be set as follows: the allocation support degree of the two page optimization labels is designated to be respectively larger than the corresponding setting support degree, and the corresponding page optimization scheme data to be allocated can be used as target page optimization scheme data; as another example, it can be set as follows: the allocation support degree of the 3 page optimization tags is respectively larger than the corresponding setting support degree, and the corresponding page optimization scheme data to be allocated can be used as target page optimization scheme data.

And then selecting the page optimization scheme data input to the target cloud service page based on the order sorting result.

In some possible embodiments, the order may be sorted according to the page optimization allocation support of a specified page optimization tag, or the order may be sorted according to the average support of the page optimization allocation support of multiple page optimization tags.

For example, according to the descending order of page optimization allocation support degree of one page optimization label, performing order arrangement on each target page optimization scheme data, and then selecting the target page optimization scheme data ranked in the first N to input to the target cloud service page; or according to the order of the page optimization allocation support degree of one page optimization label from small to large, sequentially sorting the target page optimization scheme data, and then selecting the target page optimization scheme data arranged in the last M to input to the target cloud service page.

For another example, according to the descending order of the average support degree of the page optimization allocation support degrees of the plurality of page optimization tags, the data of each target page optimization scheme is ordered; and according to the descending order of the average support degree of the page optimization allocation support degrees of the plurality of page optimization labels, carrying out order arrangement on the data of each target page optimization scheme.

In some exemplary design ideas, feature extraction is performed on candidate page abnormal events of the cloud service online page, and page abnormal path features of the candidate page abnormal events are obtained, which can be achieved through the following exemplary steps.

The method comprises the steps of processing 2011, based on a target page anomaly analysis model, carrying out anomaly code analysis on the candidate page anomaly event to determine corresponding first anomaly code analysis information, and determining first anomaly confidence levels of the candidate page anomaly event in at least one derivative page running path and a plurality of initial page running paths respectively by combining the first anomaly code analysis information;

the Process2012 updates the first abnormal confidence level corresponding to the at least one derivative page running path respectively based on the target page abnormality analysis model and on abnormality prior information to determine the second abnormal confidence level corresponding to the at least one derivative page running path respectively, the plurality of initial page running paths and the at least one derivative page running path are generated respectively based on page running configuration data of different page service modes, the page service priority of the page running configuration data corresponding to the initial page running paths is higher than the page service priority of the page running configuration data corresponding to the derivative page running paths, and each initial page running path are different;

And a Process2013, combining the first abnormal confidence degrees respectively corresponding to the plurality of initial page running paths and the second abnormal confidence degrees respectively corresponding to the at least one derivative page running path, and fusing and outputting a target page running path corresponding to the candidate page abnormal event from the plurality of initial page running paths and the at least one derivative page running path, and determining the path characteristics reflected by the target page running path as the page abnormal path characteristics of the candidate page abnormal event.

(1) Inputting each template derived page abnormal event in the first template page abnormal event cluster into an initial page abnormal analysis model, further determining a corresponding second derived abnormal confidence level of the corresponding template derived page abnormal event in the at least one derived page running path based on the initial page abnormal analysis model, and determining corresponding second initial abnormal confidence levels of the corresponding template derived page abnormal event in the plurality of initial page running paths;

(2) Inputting each template initial page abnormal event in the second template page abnormal event cluster into the initial page abnormal analysis model, further determining a third derived abnormal confidence level of the corresponding template initial page abnormal event corresponding to the at least one derived page running path based on the initial page abnormal analysis model, and determining a third initial abnormal confidence level of the corresponding template initial page abnormal event corresponding to the plurality of initial page running paths;

(3) Determining a first dimension loss function value by combining the second derived anomaly confidence level, the second initial anomaly confidence level, the third derived anomaly confidence level and the third initial anomaly confidence level;

(4) Combining the second initial anomaly confidence level and the third initial anomaly confidence level to determine a second dimension loss function value;

(5) Performing weighted calculation on the first dimension loss function value and the second dimension loss function value, determining a corresponding target loss function value, performing model iterative updating on the initial page abnormal analysis model based on the target loss function value, and determining a fuzzy page abnormal analysis model corresponding to the initial page abnormal analysis model when the loss function value calculated in a model iterative updating flow is smaller than a set value, wherein the first template page abnormal event cluster corresponds to at least one derivative page running path, and the second template page abnormal event cluster corresponds to a plurality of initial page running paths;

(6) Collecting a plurality of updating page running configuration data, inputting each updating page running configuration data into the fuzzy page abnormality analysis model, further carrying out page abnormality path analysis based on the fuzzy page abnormality analysis model, determining a first derivative abnormality confidence level corresponding to the corresponding updating page running configuration data in the at least one derivative page running path, and determining a first initial abnormality confidence level corresponding to the corresponding updating page running configuration data in the plurality of initial page running paths, wherein each updating page running configuration data corresponds to one derivative page running path or one initial page running path;

(7) Determining derivative priori abnormal parameters corresponding to the at least one derivative page running path by combining the first derivative abnormal confidence coefficient;

(8) Determining initial priori abnormal parameters corresponding to the multiple initial page running paths by combining the first initial abnormal confidence degrees;

(9) Determining corresponding abnormality prior information based on the derived priori abnormality parameters and the initial priori abnormality parameters;

(10) And configuring the abnormality prior information in a full-connection unit of the fuzzy page abnormality analysis model, and outputting a corresponding target page abnormality analysis model, wherein the abnormality prior information is used for updating derivative abnormality confidence corresponding to the at least one derivative page running path.

For some possible implementations, big data system 100 may include a processor 110, a machine-readable storage medium 120, a bus 130, and a communication unit 140.

The processor 110 may perform various suitable actions and processes by programs stored in the machine-readable storage medium 120, such as program instructions associated with the artificial intelligence based cloud service online page optimization method described in the foregoing embodiments. The processor 110, the machine-readable storage medium 120, and the communication unit 140 communicate signals over the bus 130.

In particular, the processes described in the above exemplary flowcharts may be implemented as computer software programs, in accordance with embodiments of the present invention. For example, embodiments of the present invention include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via the communication unit 140, which, when executed by the processor 110, performs the above-described functions defined in the method of the embodiment of the invention.

Still another embodiment of the present invention provides a computer readable storage medium, where computer executable instructions are stored, where the computer executable instructions are used to implement the artificial intelligence based cloud service online page optimization method according to any one of the above embodiments when executed by a processor.

Still another embodiment of the present invention provides a computer program product, including a computer program, which when executed by a processor implements the artificial intelligence based cloud service online page optimization method according to any one of the above embodiments.

The foregoing is merely an optional implementation manner of some of the implementation scenarios of the present application, and it should be noted that, for those skilled in the art, other similar implementation manners based on the technical ideas of the present application are adopted without departing from the technical idea of the solution of the present application, which is also included in the protection scope of the embodiments of the present application.

Claims

1. An artificial intelligence based page anomaly analysis method, the method comprising:

2. The artificial intelligence based page fault analysis method of claim 1, further comprising:

based on the target page anomaly analysis model, carrying out anomaly code analysis on the candidate page anomaly event to determine corresponding first anomaly code analysis information, and determining first anomaly confidence degrees of the candidate page anomaly event corresponding to at least one derivative page running path and a plurality of initial page running paths respectively by combining the first anomaly code analysis information;

combining the first abnormal confidence degrees respectively corresponding to the plurality of initial page running paths and the second abnormal confidence degrees respectively corresponding to the at least one derivative page running path, fusing and outputting a target page running path corresponding to the candidate page abnormal event from the plurality of initial page running paths and the at least one derivative page running path, determining the path characteristics reflected by the target page running path as page abnormal path characteristics of the candidate page abnormal event, and respectively acquiring page optimization logic knowledge point characteristics of each page optimization scheme data to be distributed;

And respectively loading the determined page abnormal path characteristics and each page optimization logic knowledge point characteristic into a target page optimization distribution model meeting model convergence conditions, and generating respective page optimization distribution support degree of each page optimization scheme data to be distributed.

3. The artificial intelligence based page fault analysis method as claimed in claim 2, wherein the training step of the target page optimization allocation model comprises:

performing traversal model weight parameter optimization of a plurality of training stages on a page optimization distribution model of an initialization model weight parameter according to a page optimization training data sequence until the model weight parameter of the page optimization distribution model is not changed any more, and taking the page optimization distribution model generated in the last training stage as a target page optimization distribution model, wherein the page optimization distribution model comprises a plurality of model processing units, each model processing unit is used for extracting features of page optimization training data loaded into the page optimization distribution model from a page optimization tag, each page optimization training data in the page optimization training data sequence comprises a reference page abnormal path feature of a reference page abnormal event and a reference page optimization logic knowledge point feature of reference page optimization scheme data, wherein the reference page abnormal path feature comprises paths formed by running flow features of various page application programs when a page is abnormal, and the reference page optimization logic knowledge point feature comprises optimization logic knowledge points of optimization path features of the reference page optimization scheme data, and in one traversal training stage, the following steps are executed:

4. The artificial intelligence based page fault analysis method according to claim 2, wherein the page optimal allocation support degree of each page optimal allocation scheme data in the respective page optimal allocation scheme data is: page optimization allocation support of a target page optimization label is preset;

5. The artificial intelligence based page fault analysis method of claim 2, wherein the page optimal allocation support of each page optimal allocation scheme data in the respective page optimal allocation scheme data includes page optimal allocation support for a plurality of preset page optimal tags;

6. The artificial intelligence based page anomaly analysis method according to claim 2, wherein the loading the page optimization training data obtained from the page optimization training data sequence into the page optimization allocation model, determining decision page optimization allocation data corresponding to the page optimization training data, includes:

7. The artificial intelligence based page fault analysis method as claimed in claim 6, wherein the model generation information of each of the plurality of model processing units generates a feature; the determining, respectively, a first difference metric value between model generation information corresponding to each two model processing units in the plurality of model processing units, and determining, according to each determined first difference metric value, the first training optimization index includes:

8. The artificial intelligence based page fault analysis method of claim 6, wherein the decision page optimal allocation data comprises page optimal allocation support for a preset plurality of page optimal tags;

9. The artificial intelligence based page fault analysis method as claimed in claim 7, wherein the page optimization allocation model further comprises a plurality of allocation units, each allocation unit being used for obtaining a page optimization allocation support of a page optimization tag;

10. A big data system, characterized in that the big data system comprises a processor and a memory for storing a computer program capable of running on the processor, the processor being adapted to execute the artificial intelligence based page fault analysis method according to any of claims 1-9 when the computer program is run.