CN117574142A - Data processing method and related equipment - Google Patents
Data processing method and related equipment Download PDFInfo
- Publication number
- CN117574142A CN117574142A CN202210928247.4A CN202210928247A CN117574142A CN 117574142 A CN117574142 A CN 117574142A CN 202210928247 A CN202210928247 A CN 202210928247A CN 117574142 A CN117574142 A CN 117574142A
- Authority
- CN
- China
- Prior art keywords
- kernel
- level
- tensor
- model
- label
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title abstract description 5
- 238000012549 training Methods 0.000 claims abstract description 126
- 238000000034 method Methods 0.000 claims abstract description 72
- 238000013528 artificial neural network Methods 0.000 claims abstract description 37
- 238000012545 processing Methods 0.000 claims description 36
- 230000015654 memory Effects 0.000 claims description 29
- 238000004422 calculation algorithm Methods 0.000 claims description 28
- 230000015556 catabolic process Effects 0.000 claims description 28
- 238000006731 degradation reaction Methods 0.000 claims description 28
- 230000008569 process Effects 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 7
- 239000013598 vector Substances 0.000 description 10
- 206010047289 Ventricular extrasystoles Diseases 0.000 description 8
- 238000004891 communication Methods 0.000 description 8
- 238000005129 volume perturbation calorimetry Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 6
- 238000000605 extraction Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 230000008859 change Effects 0.000 description 4
- 238000004088 simulation Methods 0.000 description 3
- 239000007787 solid Substances 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000011478 gradient descent method Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 239000000725 suspension Substances 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012821 model calculation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000011524 similarity measure Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The embodiment of the application provides a data processing method and related equipment, which utilize a small number of kernel-level indexes with labels to complete the establishment of a model and reduce the labor and economic cost required for marking the kernel-level indexes. Comprising the following steps: the method comprises the steps of obtaining a first unlabeled kernel-level index, a second labeled kernel-level index and a first label corresponding to the first labeled kernel-level index, training a self-supervision learning training learner by using the first kernel-level index to obtain a first model for extracting features of the unlabeled kernel-level index, extracting features of the labeled kernel-level index by using the first model to obtain a second tensor, and training a fully-connected neural network learner by using the second tensor and the first label to obtain a second model, wherein labeled training data comprise the second kernel index and the first label.
Description
Technical Field
Embodiments of the present application relate to cloud computing, and in particular, to a method for data processing and related devices.
Background
With the rapid development of cloud computing technology, more and more enterprises and individuals begin to utilize cloud resources to perform some more complex computations to reduce the computing pressure of local devices. In the cloud computing scenario, a plurality of virtual machines are usually placed in the same host, which can improve the resource utilization of the host, and correspondingly, the possibility of degradation of service quality (quality of service, qoS) of services on the plurality of virtual machines is also improved.
Based on a protocol between the tenant and the service provider in the cloud scene, the service provider cannot obtain the service application type and key performance index of the tenant, and in order to better cope with the situation that QoS degradation occurs to the service, technicians propose kernel-level indexes when the virtual machine is operated, so as to evaluate the service application type and the service QoS degradation situation of the tenant. But since the kernel-level metrics obtained by the virtual machine are unlabeled data. To obtain the specified result by using the unlabeled data, the kernel-level index needs to be labeled to obtain a training sample, and how to reduce the labor and time cost for labeling the kernel-level index and obtain the QoS degradation condition with higher reliability becomes the first problem faced by the technicians.
Disclosure of Invention
The embodiment of the application provides a data processing method and related equipment, which are used for completing the establishment of an algorithm model of QoS degradation by using a small number of kernel-level indexes with labels so as to reduce the labor and economic cost required for marking the kernel-level indexes.
The first aspect of the present application provides a method for data processing, where the method is applied to cloud computing, and the cloud computing includes private cloud, public cloud and hybrid cloud, and the method includes: acquiring a first kernel level index, a second kernel level index and a first label, wherein the first label is a label corresponding to the second kernel level index; training according to a first kernel-level index and a self-supervision learning algorithm learner to obtain a first model, wherein the first model is used for obtaining a first tensor, and the first tensor is a tensor corresponding to the first kernel-level index; inputting kernel-level indexes in the labeled training data into a first model to obtain a second tensor, wherein the second tensor is a tensor corresponding to the kernel-level indexes in the labeled training data, and the labeled training data comprises the second kernel-level indexes and the first labels; training the fully-connected neural network learner according to the second tensor and the labels in the labeled training data to obtain a second model, wherein the second model is used for obtaining a second label corresponding to the first tensor.
The method comprises the steps of obtaining a first unlabeled kernel-level index, a second labeled kernel-level index and a first label corresponding to the first labeled kernel-level index, training a self-supervision learning training learner by using the first kernel-level index to obtain a first model capable of carrying out feature extraction on the first unlabeled kernel-level index to obtain a first tensor, carrying out feature extraction on the labeled kernel-level index by using the first model to obtain a second tensor, training a fully-connected neural network learner by using the second tensor and the first label, carrying out prediction error calculation on the model in the fully-connected neural network learner by using the first label in labeled training data, and continuously adjusting parameters of the model in the fully-connected application network learner until the prediction error of the model is smaller than a preset value, and considering that the second model training is completed, wherein the labeled training data comprises the second kernel index and the first label. The training of the model is completed by using a small number of kernel-level indexes with labels, so that the labor and economic cost required for marking the kernel-level indexes are reduced.
In a possible implementation manner of the first aspect, the labeled training data further includes a third kernel level index and a second label, where the second label is a label corresponding to the third kernel level index; the method further comprises the steps of: and acquiring a third kernel level index and a second label.
In the method, the labeled training data comprises a third kernel level index and a second label, wherein the second label is a label corresponding to the third kernel level index, and the method further comprises the step of acquiring the third kernel level index and the second label. The training label data is updated by acquiring the third kernel-level index and the second label corresponding to the third kernel-level index, and the full-connected neural network learner is trained by utilizing all the labeled training data to obtain the second model, so that the problem that catastrophic forgetting possibly occurs when the second model is obtained by only utilizing the third kernel-level index and the second label is avoided, namely that the content related to the second kernel-level index and the first label is forgotten when the second model is obtained based on new labeled training data.
In a possible implementation manner of the first aspect, the second kernel-level indicator includes a first number of kernel-level indicators, and the third kernel-level indicator includes a second number of kernel-level indicators; the sum of the first number and the second number is less than or equal to a preset value.
In the method, the problem that the training efficiency of the model is reduced along with the increase of the data quantity is avoided by limiting the number of kernel-level indexes in the label training data to be smaller than a preset value. The problem that the calculation power and the storage requirement of hardware are continuously improved along with the increase of the labeled training data is also reduced.
In a possible implementation manner of the first aspect, the self-supervised learning algorithm learner includes a self-attentive neural network transducer encoder and a contrast learner. In the method, the adoption of the transducer encoder can improve the training efficiency of the model by processing long sequence data and parallel data.
In a possible implementation manner of the first aspect, the first label includes at least one of a QoS degradation proportion and an application load type. In the method, the first tag includes at least one of QoS degradation ratio and application load type, and one or more tags may be output at the same time.
In a possible implementation manner of the first aspect, the first kernel level indicator includes at least one of a first central processing unit CPU utilization rate, a first process switching number, a first CPU instruction completion number, a first CPU instruction being a stall always cycle number, or a first CPU front end stall always cycle number.
A second aspect of the present application provides a computing device comprising:
the acquisition unit is used for acquiring a first kernel-level index, a second kernel-level index and a first label, wherein the first label is a label corresponding to the second kernel-level index;
the training unit is used for training according to the first kernel-level index and the self-supervision learning algorithm learner to obtain a first model, wherein the first model is used for obtaining a first tensor, and the first tensor is a tensor corresponding to the first kernel-level index;
The computing unit is used for inputting the kernel-level index in the labeled training data into the first model to obtain a second tensor, wherein the second tensor is a tensor corresponding to the kernel-level index in the labeled training data, and the labeled training data comprises a second kernel-level index and a first label;
the training unit is further configured to train the fully connected neural network learner according to the second tensor and the label in the labeled training data, to obtain a second model, where the second model is used to obtain a second label corresponding to the first tensor.
In a possible implementation manner of the second aspect, the labeled training data further includes a third kernel level index and a third label, where the third label is a label corresponding to the third kernel level index;
the obtaining unit is further configured to obtain a third kernel level indicator and a third tag.
In a possible implementation manner of the second aspect, the second kernel-level indicator includes a first number of kernel-level indicators, and the third kernel-level indicator includes a second number of kernel-level indicators; the sum of the first number and the second number is less than or equal to a preset value.
In a possible implementation manner of the second aspect, the self-supervised learning algorithm learner includes a self-attentive neural network transducer encoder and a contrast learner.
In a possible implementation manner of the second aspect, the first label includes at least one of QoS degradation proportion and application load type;
the second label includes at least one of a QoS degradation ratio and an application load type.
In a possible implementation manner of the second aspect, the first kernel level indicator includes at least one of a first central processing unit CPU utilization rate, a first process switching number, a first CPU instruction completion number, a first CPU instruction being a stall always cycle number, or a first CPU front end stall always cycle number.
A third aspect of the present application provides a computing device comprising: a processor and a memory;
the memory is used for storing computer execution instructions;
the processor is configured to execute computer-executable instructions stored in the memory to cause the electronic device to perform the method as in the first aspect of the present application or any of the possible implementations of the first aspect.
A fourth aspect of the present application provides a cluster of computing devices, comprising at least one computing device, each computing device comprising a processor and a memory;
the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform a method as in the first aspect or any of the possible implementations of the first aspect of the present application.
A fifth aspect of the application provides a computer program product comprising instructions which, when executed by a cluster of computing devices, cause the cluster of computing devices to perform a method as in the first aspect of the application or any of the possible implementations of the first aspect.
A sixth aspect of the application provides a computer readable storage medium comprising computer program instructions which, when executed by a cluster of computing devices, perform a method as in the first aspect of the application or any of the possible implementations of the first aspect.
Drawings
FIG. 1 is a schematic flow chart of data processing according to an embodiment of the present disclosure;
FIG. 2a is a schematic diagram of another flow chart of data processing according to an embodiment of the present application;
FIG. 2b is a schematic diagram of another flow chart of data processing according to an embodiment of the present disclosure;
FIG. 3 is a schematic flow chart of a self-supervised learning algorithm learner training method according to an embodiment of the present application;
FIG. 4 is a schematic flow chart of the operation of the encoder according to the embodiment of the present application;
FIG. 5 is a schematic diagram of a computing device according to an embodiment of the present application;
FIG. 6 is another structural schematic diagram of a computing device provided in an embodiment of the present application;
FIG. 7 is a schematic diagram of a computing cluster according to an embodiment of the present disclosure;
fig. 8 is another schematic structural diagram of a computing cluster according to an embodiment of the present application.
Detailed Description
Embodiments of the present application will now be described with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some, but not all embodiments of the present application. As a person of ordinary skill in the art can know, with the development of technology and the appearance of new scenes, the technical solutions provided in the embodiments of the present application are applicable to similar technical problems.
The terms first, second and the like in the description and in the claims of the present application and in the above-described figures, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
In the cloud computing field, to improve the resource utilization rate of a host, a plurality of virtual machines are usually disposed in the same host, so as to improve the resource utilization rate of the host, but increase the possibility of service QoS degradation of the virtual machines. In order to alleviate this problem, performance interference that may occur when different types of services are set in the same host is generally evaluated according to the service type of the tenant, so as to avoid situations where applications with relatively large interference to each other are deployed in the same host as much as possible. Or dynamically monitoring the service QoS degradation of the tenant, and when the QoS degradation occurs, expelling the low-priority virtual machine so as to ensure the service QoS performance of the tenant with high priority.
In public cloud and hybrid cloud scenarios, a cloud service provider cannot legally obtain service application types and key performance indexes of tenants, and further cannot utilize the information to adjust QoS degradation of services in cloud services. To solve this problem, technicians propose that the service type or service QoS degradation of the tenant can be evaluated using an Operating System (OS) kernel level index of the virtual machine runtime. And acquiring kernel-level indexes and corresponding service type information or QoS degradation information for a period of time, and training a machine learning algorithm by utilizing the data to obtain a prediction model, wherein the kernel-level indexes are input into the model when the prediction model is applied, and the prediction model outputs the service type information and the QoS degradation information according to the input kernel-level indexes so as to release host resources or transfer applications with larger interference to different hosts.
However, when the prediction model is trained, the collected massive data needs to be marked to obtain a training sample, and the labor and economic cost for marking the data are huge, so that the scheme cannot be widely applied, the labor and economic cost for marking the kernel-level index is reduced, and the primary problem faced by technicians is solved by obtaining the QoS degradation condition with higher reliability.
The method comprises the steps of obtaining a first model for extracting tensors by utilizing unlabeled kernel-level indexes and training a self-supervision algorithm learner, inputting kernel-level indexes corresponding to labels into the first model for calculation to obtain tensors corresponding to the labels, obtaining tensors corresponding to the labels by utilizing the first model calculation, and training a fully-connected neural network learner by utilizing the labels to obtain a second model, wherein the second model is used for obtaining labels corresponding to the input tensors. Therefore, the demand for labeled data is reduced when the model is trained, and the labor and economic cost for marking the kernel-level index is reduced.
Based on the above thought, the scheme provided by the embodiment of the application is described below with reference to the accompanying drawings. Referring to fig. 1, fig. 1 is a schematic flow chart of data processing according to an embodiment of the present application.
101. Acquiring a first kernel level index;
and acquiring a certain period of time by using a perf tool when a certain virtual machine runs from a host of public cloud, private cloud or mixed cloud scenes, wherein the kernel-level index of the virtual machine is used as a first kernel-level index.
Specifically, the kernel-level indicator may include at least one of a central processing unit (central processing unit, CPU) utilization, a process switching count, a CPU instruction completion count, a CPU instruction count of a stall cycle number or a CPU front-end stall cycle number.
It should be noted that, the manner of collecting the first kernel-level index from the host and the type of data included in the kernel-level index are both examples, and are not limited in practical application, and different collection manners of the first kernel-level index and the type of data of the kernel-level index may be set according to the needs.
102. Training according to the first kernel-level index and a self-supervision learning algorithm learner to obtain a first model;
after the first kernel-level index is obtained, the first kernel-level index is segmented by utilizing a self-supervision learning algorithm learner, similar data and non-similar data are distinguished through comparison learning, then self-supervision training is carried out according to similarity comparison information among the kernel-level indexes, whether the self-supervision training is finished or not is judged by using comparison loss, and when the comparison loss tends to be stable, the self-supervision training is finished, and a first model is obtained. The first model is used for obtaining a first tensor corresponding to the first kernel-level index.
Alternatively, the self-supervised learning algorithm learner may include a transducer encoder and a contrast learning learner.
The self-supervision learning algorithm learner can process a plurality of samples in parallel by using the transducer encoder, so that the training efficiency of the model is effectively improved.
It should be noted that the specific learning method included in the self-supervised learning algorithm learner is only an example, and in practical applications, other types of encoders and learners may also be used to obtain the first model using the self-supervised learning algorithm.
103. Acquiring a second kernel level index and a first label, wherein the first label is a label corresponding to the second kernel level index;
the first period collects kernel-level indexes with labels from private cloud or simulation environment to serve as second kernel-level indexes, and labels corresponding to the second kernel-level indexes serve as first labels. The first kernel level indicator contains the same data type as the second kernel level indicator.
Specifically, the first tag may be at least one of an application load type and a traffic QoS degradation ratio.
It should be noted that the description of the type of the first tag is merely an example, and in practical application, the specific type of the first tag may be adjusted according to the requirement, which is not limited herein.
It should be noted that the sequence of step 101, step 102 and step 103 is merely an example, and in practical application, step 103 may be performed before step 104, which is not limited herein.
104. Inputting kernel-level indexes in the labeled training data into a first model to obtain a second tensor;
after the second kernel-level index and the first label are acquired, the kernel-level index in the labeled training data is input into the first model, and a second tensor corresponding to the second kernel-level index is obtained. The tagged training data comprises a second kernel level indicator and a first tag.
105. Training the fully connected neural network learner according to the second tensor and the labeled training data to obtain a second model.
After the second tensor is obtained, training the fully-connected neural network learner according to the second tensor and the labels in the labeled training data, calculating the prediction error of the current model according to the labels in the labeled training data, and then adjusting the parameters in the neural network according to the back propagation algorithm until the prediction error of the current model is converged, so that the second model training is considered to be completed.
In particular, parameters in the neural network may be adjusted using a gradient descent method or a stochastic gradient method.
It should be noted that the method for adjusting the parameters in the neural network is merely exemplary and not limited to a specific adjusting manner, and in practical application, different methods may be used to adjust the parameters in the neural network according to practical requirements.
In the embodiment of the application, the self-supervision learning training learner is trained by acquiring the unlabeled kernel-level index and the labeled kernel-level index, a first model capable of extracting features of the unlabeled kernel-level index is obtained, the labeled kernel-level index is extracted by using the first model, a feature tensor is obtained, the fully-connected neural network learner is trained by using the feature tensor and the label corresponding to the labeled kernel-level index, the prediction error of the model in the fully-connected neural network learner is calculated by using the feature tensor and the corresponding label, and parameters of the model in the fully-connected neural network learner are continuously adjusted until the prediction error of the model is smaller than a preset value, and the second model training is considered to be completed. The training of the model is completed by using a small number of kernel-level indexes with labels, so that the labor and economic cost required for marking the kernel-level indexes are reduced.
When the tag of the tag data includes a service application type, the second model should correspondingly change along with the change of the service application type carried on the virtual machine, and the case that the second model is adjusted along with the change of the service application type carried on the virtual machine is described below with reference to fig. 2 a. Fig. 2a is a schematic flow chart of another data processing according to an embodiment of the present application.
201. Acquiring a first kernel level index;
202. training according to the first kernel-level index and a self-supervision learning algorithm learner to obtain a first model;
203. acquiring a second kernel level index and a first label, wherein the first label is a label corresponding to the second kernel level index;
the operations performed in steps 201 to 203 are similar to those performed in steps 101 to 103 in fig. 1, and are not repeated here.
204. Acquiring a third kernel level index and a second label, wherein the second label is a label corresponding to the third kernel level index;
after the second kernel-level index and the first label are obtained, if the type of the service application which can be borne on the virtual machine changes, the kernel-level index which can be borne and the type of the service application which can be borne correspondingly change is collected from the private cloud or the simulation environment in a second period of time to serve as a third kernel-level index, and labels corresponding to the kernel-level indexes serve as second labels.
205. Inputting kernel-level indexes in the labeled training data into a first model to obtain a second tensor;
after the second kernel-level index, the third kernel-level index, the first label and the second label are obtained, the kernel-level index in the labeled data is input into the first model to obtain a second tensor. The tagged data comprises a second kernel-level index, a third kernel-level index, a first tag and a second tag, wherein the first model is a feature extraction model.
As the number of times of acquiring the kernel-level index and the tag corresponding to the kernel-level index increases, the volume of the tagged training data is continuously increased, so as to avoid the occurrence of the situation that the training efficiency of the model is extremely reduced. The method can further judge whether the sum of the numbers of the second kernel-level index and the third kernel-level index is smaller than a preset value after the second kernel-level index, the third kernel-level index, the first label and the second label are obtained.
If the sum of the number of the second kernel-level indicators and the third kernel-level indicators is less than or equal to a preset value, the tagged training data includes the second kernel-level indicators, the third kernel-level indicators, the first tag and the second tag.
If the sum of the numbers of the second kernel-level indexes and the third kernel-level indexes is larger than a preset value, selecting the first number of the second kernel-level indexes and the second number of the third kernel-level indexes and the labels corresponding to the first number of the second kernel-level indexes and the second number of the third kernel-level indexes as the labeled training data, wherein the sum of the first number of the second kernel-level indexes and the second number of the third kernel-level indexes is smaller than or equal to the preset value.
The number of kernel-level indexes in the labeled training data is limited, so that the condition that the training efficiency of the model is continuously reduced along with the increase of the number of the labeled training data can be avoided. And the problem that the model is catastrophically forgotten is avoided by retaining a part of the second kernel-level index and the first label in the label data.
It should be noted that the process of controlling the number of kernel-level indexes in the labeled training data to be less than or equal to the preset value is merely an example, and in practical application, it may be determined whether to limit the number of kernel-level indexes in the labeled training data according to the requirement, which is not limited herein.
206. Training the fully connected neural network learner according to the second tensor and the labeled training data to obtain a second model.
After the second tensor is obtained, training the fully-connected neural network learner according to the second tensor and the labels in the labeled training data, calculating the prediction error of the current model according to the labels in the labeled training data, and then adjusting parameters in the neural network according to a reverse blogging algorithm and other methods until the prediction error of the current model converges, wherein the second model is considered to be trained, and the second model is a prediction model.
In particular, parameters in the neural network may be adjusted using a gradient descent method or a stochastic gradient method.
It should be noted that the method for adjusting the parameters in the neural network is merely exemplary and not limited to a specific adjusting manner, and in practical application, different methods may be used to adjust the parameters in the neural network according to practical requirements.
Optionally, before training the fully-connected neural network learner by using the label in the labeled training data, training the fully-connected neural network learner by using the second kernel level index and the first label, and after training, continuing to train the fully-connected neural network learner which completes one training by using the third kernel level index and the second label.
In the embodiment of the application, after the second kernel-level index and the first label are acquired, the third kernel-level index and the second label after the service application type which can be borne on the virtual machine is changed are acquired, and then the full-connection neural network learner is trained according to the kernel-level index in the labeled training data and the label corresponding to the kernel-level index to acquire the second model, so that the second model can be timely updated according to service adjustment in the virtual machine, the situation that the second model is not matched with an actual application scene is avoided, the quantity of data input into the full-connection neural network learner can be managed by updating the labeled training data, the cost and time of model training are saved while the new knowledge and the old knowledge are guaranteed, and the situation that the model is catastrophic and forgotten is avoided.
In the following, referring to fig. 2b, taking an example that the first label and the second label both include QoS degradation proportion and service application type as an example, a scheme proposed in the present application is introduced, and fig. 2b is another flow chart of data processing provided in an embodiment of the present application.
And collecting corresponding kernel-level index data X, service application type Y and QoS degradation proportion Z in the running process of part of service application types in private cloud and/or simulation environment as labeled training samples for parameter training of a prediction model.
The kernel-level index in the labeled training sample is input into a feature extraction model, a tensor corresponding to the kernel-level index in the labeled training sample is obtained through feature extraction, and the tensor is input into a prediction model facing a specific task. In fig. 2b, the prediction model is a prediction model formed by two fully connected neural networks, the prediction error of the current model is calculated according to the labels (service application type and QoS degradation proportion) in the labeled training samples, and the parameters in the model are adjusted according to the back propagation algorithm until the prediction error of the model converges, and the prediction model completes training.
Optionally, after the kernel-level index data, the service application type and the QoS degradation proportion corresponding to the service application type in the operation process are collected, a label training sample is maintained, and the collected kernel-level index data, the service application type and the QoS degradation proportion corresponding to the service application type in the operation process are stored in the label training sample for subsequent use. Specifically, when the corresponding kernel-level index data, the service application type and the QoS degradation proportion in the operation process of the service application type are acquired again later, judging whether the number of the kernel-level index and the re-acquired kernel-level index in the current labeled training sample reaches a preset value or not, and if the number of the kernel-level index and the re-acquired kernel-level index is smaller than the preset value, directly adding the re-acquired data into the labeled training sample; if the data is greater than the preset value, deleting a certain amount of data (the kernel-level index, the QoS degradation proportion corresponding to the kernel-level index and the service application type corresponding to the kernel-level index) from the current labeled training sample, so that after all or part of the re-acquired data is added to the labeled training sample, the number of the kernel-level indexes in the labeled training sample is smaller than the preset value.
As can be seen from fig. 1 and fig. 2a, after the first model is obtained through self-supervised learning, the second model is obtained through supervised learning by using tensor output by the first model and labeled training data. For an example, taking a self-supervised learning algorithm learner including contrast learning and a self-attentive neural network (transducer) as an example in conjunction with fig. 3, a first model obtained through self-supervised learning in the present solution is described, and referring to fig. 3, fig. 3 is a schematic flow chart of a training method of the self-supervised learning algorithm learner according to an embodiment of the present application.
301. Acquiring a first kernel level index;
and acquiring kernel-level indexes of the virtual machine in running from the public cloud and/or the mixed cloud, and taking the acquired kernel-level indexes as first kernel-level indexes.
For example, when the service application i is running in the virtual machine in 0s to 10s, the first kernel-level index is time series data formed by combining kernel-level indexes corresponding to each time of the virtual machine in 0s to 10 s.
It should be noted that the description of the first kernel-level indicator is merely an example, and in practical application, the meaning of the first kernel-level indicator may be interpreted according to practical situations, which is not limited herein.
302. Splitting the first kernel-level index according to time to obtain a first sample and a second sample;
splitting the first kernel-level index into at least two samples according to a certain time interval to obtain a first sample and a second sample.
For example, taking the first kernel-level index as the time series data formed by combining the kernel-level indexes corresponding to each time of the virtual machine between 0s and 10s by taking the operation of the business application i in the virtual machine as an example. Taking kernel-level indexes between 0s and 1s in the first kernel-level indexes as first samples; and taking kernel-level indexes between 2s and 3s in the first kernel-level indexes as second samples. The first sample is historical data relative to the second sample, denoted as H i The second sample is future data, denoted as F, compared to the first sample i 。
It should be noted that, the description of the first kernel-level indicator is only an example, and the description of the first sample and the second sample is only an example when the first kernel-level indicator is split into at least two samples, in practical application, the meaning of the first kernel-level indicator may be interpreted according to practical situations, and the first sample and the second sample may be divided in different manners, which is not limited herein.
303. Inputting a first sample and a second sample to an encoder, and obtaining a first characteristic tensor corresponding to the first sample and a second characteristic tensor corresponding to the second sample;
and inputting the first sample and the second sample obtained by splitting the first kernel-level index into an encoder of the self-supervision learning algorithm learner, and obtaining a first characteristic tensor corresponding to the first sample and a second characteristic tensor corresponding to the second sample.
Taking an encoder as a transducer encoder as an example, the working mode of the encoder is described as follows: referring to fig. 4, fig. 4 is a schematic flow chart of the operation of the encoder according to the embodiment of the present application.
After the first sample is used as input data to be input, the kernel-level indexes in the first sample are encoded, then the different types of kernel-level indexes of the first sample are subjected to position encoding, then the position-encoded first sample is input into a stackable processing module consisting of a multi-head attention mechanism and a feedforward network, the first sample is taken as a time sequence sample of the kernel-level indexes with the time length of 1s as an example, the first sample is a 1*d matrix, d identifies the number of the kernel-level indexes, the first sample is encoded and position-encoded, encoding information is combined into a matrix A, N groups of Q, K, V vectors are generated through N multi-head attention modules, Q is a query vector, K is a key vector, and V is a value vector. After calculating the score S for each group using Q and K, the output vector z for each group of vectors is calculated using a first formula. Wherein, the first formula is: z=softmax (S) ×v, combining the N output vectors z to obtain a first feature tensor corresponding to the first sample, denoted as h i 。
The manner of obtaining the second feature vector by using the second sample is similar to the manner of obtaining the first feature vector by using the editor of the transducer structure, and the second feature vector is denoted as f and will not be described here again i 。
It should be noted that, the description of the structure of the encoder of the self-supervised learning algorithm learner is merely an example, and in practical application, the encoder may be set according to a practical application scenario, which is not limited herein.
304. Judging whether the contrast loss of the first characteristic tensor and the second characteristic tensor tends to be stable or not;
after the first sample and the second sample are subjected to feature extraction through the encoder, whether the model is subjected to learning is judged by using a comparison learning method, a pair of samples from the same service application is called a positive sample, samples from different service applications are symmetrical to be negative samples, and the similarity between two samples in the positive sample is larger than the similarity between two samples in the negative sample.
Illustratively, business application i may be redis, business application j may be MySQL, first sample pair (H i ，F i ) Is a positive sample, the second sample pair (H i ，F j ) Is a negative example such that the similarity of two samples in a first sample pair is greater than the similarity of two samples in a second sample pair.
Based on the above thought, calculating the contrast loss L between samples by using the second formula and whether the samples are from the same business application, the second formula is:wherein sim () represents a similarity measure function, τ is a parameter whose specific value can be adjusted according to the problem scene, f j And representing the feature tensor corresponding to the future data obtained by splitting the business application j.
After the calculation is completed, it is determined whether the contrast loss tends to be stable. The mode of judging that the contrast loss tends to be stable may be to judge whether the contrast loss L is smaller than a preset value or whether the difference between the contrast losses L obtained by two adjacent calculations is smaller than a preset value.
If the contrast loss has stabilized, step 305 is performed.
If the contrast loss does not stabilize, step 306 is performed.
305. Outputting the first characteristic tensor and the second characteristic tensor as a first tensor;
and if the contrast loss between the samples tends to be stable, outputting the first eigenvector and the second eigenvector as a first tensor.
306. Parameters in the encoder are adjusted.
If the contrast loss does not stabilize, step 303 is performed after adjusting the parameters in the encoder.
The method provided by the embodiment of the application is introduced above, and the computing device cluster provided by the embodiment of the application are introduced below.
Referring to fig. 5, fig. 5 is a schematic structural diagram of a computing device according to an embodiment of the present application. Comprising the following steps:
an obtaining unit 501, configured to obtain a first kernel-level index, a second kernel-level index, and a first tag, where the first tag is a tag corresponding to the second kernel-level index;
the training unit 502 is configured to obtain a first model according to the first kernel-level index and the self-supervised learning algorithm learner, where the first model is used to obtain a first tensor, and the first tensor is a tensor corresponding to the first kernel-level index;
a calculating unit 503, configured to input a kernel-level index in the labeled training data into the first model, obtain a second tensor, where the second tensor is a tensor corresponding to the kernel-level index in the labeled training data, and the labeled training data includes a second kernel-level index and a first label;
the training unit 502 is further configured to train the fully-connected neural network learner according to the second tensor and the label in the labeled training data, to obtain a second model, where the second model is used to obtain a second label corresponding to the first tensor.
Optionally, the labeled training data further includes a third kernel level index and a second label, where the second label is a label corresponding to the third kernel level index;
The obtaining unit 501 is further configured to obtain a third kernel level indicator and a second tag.
Optionally, the second kernel-level indicator includes a first number of kernel-level indicators, and the third kernel-level indicator includes a second number of kernel-level indicators; the sum of the first number and the second number is less than or equal to a preset value.
Optionally, the self-supervised learning algorithm learner includes a self-attention neural network transducer encoder and a contrast learner.
Optionally, the first label includes at least one of QoS degradation ratio and application load type.
Optionally, the first kernel level indicator includes at least one of a first central processing unit CPU utilization, a first process switching number, a first CPU instruction completion number, a first CPU instruction being a number of cycles of suspension or a number of cycles of suspension at a front end of the first CPU.
The acquiring unit, the training unit and the calculating unit can be realized by software or hardware. By way of example, the implementation of the acquisition unit is described next by taking the acquisition unit as an example. Similarly, the implementation of the training unit and the computing unit may refer to the implementation of the acquisition unit.
Module as an example of a software functional unit, the acquisition unit may comprise code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container, among others. Further, the above-described computing examples may be one or more. For example, the acquisition unit may include code running on multiple hosts/virtual machines/containers. It should be noted that, multiple hosts/virtual machines/containers for running the code may be distributed in the same region (region), or may be distributed in different regions. Further, multiple hosts/virtual machines/containers for running the code may be distributed in the same availability zone (availability zone, AZ) or may be distributed in different AZs, each AZ comprising a data center or multiple geographically close data centers. Wherein typically a region may comprise a plurality of AZs.
Also, multiple hosts/virtual machines/containers for running the code may be distributed in the same virtual private cloud (virtual private cloud, VPC) or in multiple VPCs. In general, one VPC is disposed in one region, and a communication gateway is disposed in each VPC for implementing inter-connection between VPCs in the same region and between VPCs in different regions.
Unit as an example of a hardware functional unit, the acquisition unit may comprise at least one computing device, such as a server or the like. Alternatively, the acquisition unit may be a device or the like implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (programmable logic device, PLD). The PLD may be implemented as a complex program logic device (complex programmable logical device, CPLD), a field-programmable gate array (FPGA), a general-purpose array logic (generic array logic, GAL), or any combination thereof.
Multiple computing devices included in the acquisition unit may be distributed in the same region or may be distributed in different regions. The plurality of computing devices included in the acquisition unit may be distributed in the same AZ or may be distributed in different AZ. Also, the plurality of computing devices included in the acquisition unit may be distributed in the same VPC, or may be distributed in a plurality of VPCs. Wherein the plurality of computing devices may be any combination of computing devices such as servers, ASIC, PLD, CPLD, FPGA, and GAL.
It should be noted that, in other embodiments, the acquiring unit may be used to perform any step in the method of data processing, the training unit may be used to perform any step in the method of data processing, the calculating unit may be used to perform any step in the method of data processing, and the steps that the acquiring unit, the training unit, and the calculating unit are responsible for implementing may be specified as needed, and all functions of the computing device are implemented by different steps in the method that the acquiring unit, the training unit, and the calculating unit implement the data processing respectively.
The present application also provides a computing device 600. As shown in fig. 6, the computing device 600 includes: bus 602, processor 604, memory 606, and communication interface 608. The processor 604, the memory 606, and the communication interface 608 communicate via the bus 602. Computing device 600 may be a server or a terminal device. It should be understood that the present application is not limited to the number of processors, memories in computing device 600.
Bus 602 may be a peripheral component interconnect standard (peripheral component interconnect, PCI) bus or an extended industry standard architecture (extended industry standard architecture, EISA) bus, among others. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, only one line is shown in fig. 6, but not only one bus or one type of bus. Bus 604 may include a path to transfer information between various components of computing device 600 (e.g., memory 606, processor 604, communication interface 608).
The processor 604 may include any one or more of a central processing unit (central processing unit, CPU), a graphics processor (graphics processing unit, GPU), a Microprocessor (MP), or a digital signal processor (digital signal processor, DSP).
The memory 606 may include volatile memory (RAM), such as random access memory (random access memory). The processor 604 may also include non-volatile memory (ROM), such as read-only memory (ROM), flash memory, a mechanical hard disk (HDD), or a solid state disk (solid state drive, SSD).
The memory 606 has stored therein executable program code that the processor 604 executes to implement the functions of the aforementioned acquisition unit, training unit, and calculation unit, respectively, to implement the method of data processing. That is, the memory 606 has stored thereon instructions for performing the method of data processing.
The communication interface 603 enables communication between the computing device 600 and other devices or communication networks using a transceiver module such as, but not limited to, a network interface card, transceiver, or the like.
The embodiment of the application also provides a computing device cluster. The cluster of computing devices includes at least one computing device. The computing device may be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device may also be a terminal device such as a desktop, notebook, or smart phone.
As shown in fig. 7, the cluster of computing devices includes at least one computing device 600. The same instructions for performing the data processing method may be stored in memory 606 in one or more computing devices 600 in the computing device cluster.
In some possible implementations, portions of the instructions of the method for performing data processing may also be stored separately in the memory 606 of one or more computing devices 600 in the computing device cluster. In other words, a combination of one or more computing devices 600 may collectively execute instructions for performing a method of data processing.
It should be noted that the memory 606 in different computing devices 600 in the computing device cluster may store different instructions for performing part of the functions of the computing devices, respectively. That is, the instructions stored by the memory 606 in the different computing devices 600 may implement the functionality of one or more of the acquisition unit, training unit, and computing unit.
In some possible implementations, one or more computing devices in a cluster of computing devices may be connected through a network. Wherein the network may be a wide area network or a local area network, etc. Fig. 8 shows one possible implementation. As shown in fig. 8, two computing devices 600A and 600B are connected by a network. Specifically, the connection to the network is made through a communication interface in each computing device. In this type of possible implementation, instructions to perform the functions of the acquisition unit are stored in memory 606 in computing device 600A. Meanwhile, instructions to perform the functions of the training unit and the computing unit are stored in the memory 606 in the computing device 600B.
It should be appreciated that the functionality of computing device 600A shown in fig. 8 may also be performed by multiple computing devices 600. Likewise, the functionality of computing device 600B may also be performed by multiple computing devices 600.
The embodiment of the application also provides another computing device cluster. The connection between computing devices in the computing device cluster may be similar to the connection of the computing device cluster described with reference to fig. 7 and 8. In contrast, the memory 606 in one or more computing devices 600 in the computing device cluster may have stored therein the same instructions for performing the method of data processing.
In some possible implementations, portions of the instructions of the method for performing data processing may also be stored separately in the memory 606 of one or more computing devices 600 in the computing device cluster. In other words, a combination of one or more computing devices 600 may collectively execute instructions for performing a method of data processing.
Embodiments of the present application also provide a computer program product comprising instructions. The computer program product may be software or a program product containing instructions capable of running on a computing device or stored in any useful medium. The computer program product, when run on at least one computing device, causes the at least one computing device to perform a method of data processing.
Embodiments of the present application also provide a computer-readable storage medium. The computer readable storage medium may be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk), etc. The computer-readable storage medium includes instructions that direct a computing device to perform a method of data processing.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; these modifications or substitutions do not depart from the essence of the corresponding technical solutions from the protection scope of the technical solutions of the embodiments of the present invention.
Claims (16)
1. A method of data processing, the method being applied to cloud computing, the cloud computing comprising a private cloud, a public cloud, and a hybrid cloud, the method comprising:
acquiring a first kernel level index, a second kernel level index and a first label, wherein the first label is a label corresponding to the second kernel level index;
training according to the first kernel-level index and a self-supervision learning algorithm learner to obtain a first model, wherein the first model is used for obtaining a first tensor, and the first tensor is a tensor corresponding to the first kernel-level index;
inputting kernel-level indexes in the labeled training data into the first model to obtain a second tensor, wherein the second tensor is a tensor corresponding to the kernel-level indexes in the labeled training data, and the labeled training data comprises the second kernel-level indexes and the first labels;
Training the fully-connected neural network learner according to the second tensor and the labels in the labeled training data to obtain a second model, wherein the second model is used for obtaining a second label corresponding to the first tensor.
2. The method of claim 1, wherein the tagged training data further comprises a third kernel level indicator and a second tag, the second tag being a tag corresponding to the third kernel level indicator;
the method further comprises the steps of:
and acquiring the third kernel-level index and the second tag.
3. The method of claim 2, wherein the second kernel-level metrics comprise a first number of kernel-level metrics and the third kernel-level metrics comprise a second number of kernel-level metrics; the sum of the first number and the second number is less than or equal to a preset value.
4. A method according to any one of claims 1 to 3, wherein the self-supervised learning algorithm learner comprises a self-attention neural network transducer encoder and a contrast learner.
5. The method of any of claims 1 to 4, wherein the first label comprises at least one of QoS degradation ratio and application load type.
6. The method of any one of claims 1 to 5, wherein the first core level indicator comprises at least one of a first central processing unit CPU utilization, a first number of process switches, a first number of CPU instruction completions, a first number of CPU instruction cycles to stall, or a first number of CPU front end stall cycles to stall.
7. A computing device, comprising:
the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring a first kernel-level index, a second kernel-level index and a first label, and the first label is a label corresponding to the second kernel-level index;
the training unit is used for training to obtain a first model according to the first kernel-level index and a self-supervision learning algorithm learner, wherein the first model is used for obtaining a first tensor, and the first tensor is a tensor corresponding to the first kernel-level index;
the computing unit is used for inputting kernel-level indexes in the labeled training data into the first model to obtain a second tensor, wherein the second tensor is a tensor corresponding to the kernel-level indexes in the labeled training data, and the labeled training data comprises the second kernel-level indexes and the first labels;
the training unit is further configured to train the fully-connected neural network learner according to the second tensor and the label in the labeled training data, so as to obtain a second model, where the second model is used to obtain a second label corresponding to the first tensor.
8. The apparatus of claim 7, wherein the tagged training data further comprises a third kernel-level indicator and a second tag, the second tag being a tag corresponding to the third kernel-level indicator;
the obtaining unit is further configured to obtain the third kernel level indicator and the second tag.
9. The apparatus of claim 8, wherein the second kernel-level metrics comprise a first number of kernel-level metrics and the third kernel-level metrics comprise a second number of kernel-level metrics; the sum of the first number and the second number is less than or equal to a preset value.
10. The apparatus of any one of claims 7 to 9, wherein the self-supervised learning algorithm learner comprises a self-attention neural network transducer encoder and a contrast learner.
11. The apparatus according to any one of claims 7 to 10, wherein the first label includes at least one of a QoS degradation ratio and an application load type.
12. The apparatus of any of claims 7 to 11, wherein the first core level indicator comprises at least one of a first central processing unit CPU utilization, a first number of process switches, a first number of CPU instruction completions, a first CPU instruction being a number of stall cycles or a first CPU front end number of stall cycles.
13. A computing device, comprising: a processor and a memory;
the memory is used for storing computer execution instructions;
the processor is configured to execute computer-executable instructions stored in the memory to cause the electronic device to perform the method of any one of claims 1 to 6.
14. A cluster of computing devices, comprising at least one computing device, each computing device comprising a processor and a memory;
the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method of any one of claims 1 to 6.
15. A computer program product containing instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method of any of claims 1 to 6.
16. A computer readable storage medium comprising computer program instructions which, when executed by a cluster of computing devices, perform the method of any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210928247.4A CN117574142A (en) | 2022-08-03 | 2022-08-03 | Data processing method and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210928247.4A CN117574142A (en) | 2022-08-03 | 2022-08-03 | Data processing method and related equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117574142A true CN117574142A (en) | 2024-02-20 |
Family
ID=89883119
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210928247.4A Pending CN117574142A (en) | 2022-08-03 | 2022-08-03 | Data processing method and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117574142A (en) |
-
2022
- 2022-08-03 CN CN202210928247.4A patent/CN117574142A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111985229A (en) | Sequence labeling method and device and computer equipment | |
| AU2021309929B2 (en) | Anomaly detection in network topology | |
| CN104615730B (en) | A kind of multi-tag sorting technique and device | |
| CN114444619B (en) | Sample generation method, training method, data processing method and electronic device | |
| US11783221B2 (en) | Data exposure for transparency in artificial intelligence | |
| CN113255747B (en) | Quantum multichannel convolutional neural classification method, system, terminal and storage medium | |
| CN114610795A (en) | Digital service information processing method and server applied to big data mining | |
| WO2022001233A1 (en) | Pre-labeling method based on hierarchical transfer learning and related device | |
| CN116361567B (en) | Data processing method and system applied to cloud office | |
| CN116542673B (en) | Fraud identification method and system applied to machine learning | |
| CN111061763B (en) | Method and device for generating rule execution plan of rule engine | |
| CN117574142A (en) | Data processing method and related equipment | |
| CN116757476A (en) | Method and device for constructing risk prediction model and method and device for risk prevention and control | |
| CN115169489B (en) | Data retrieval method, device, equipment and storage medium | |
| CN115202879A (en) | Multi-type intelligent model-based cloud edge collaborative scheduling method and application | |
| CN109753708A (en) | A kind of payment amount prediction technique, device and readable storage medium storing program for executing | |
| CN115080745A (en) | Multi-scene text classification method, device, equipment and medium based on artificial intelligence | |
| US20210264313A1 (en) | Machine-learned data management between devices | |
| CN113627514A (en) | Data processing method and device of knowledge graph, electronic equipment and storage medium | |
| CN113537942A (en) | Method and system for increasing number of sample marks | |
| CN113837863B (en) | Business prediction model creation method and device and computer readable storage medium | |
| CN113240565B (en) | Target identification method, device, equipment and storage medium based on quantization model | |
| CN104834560B (en) | A kind of excavation moving method for obtaining cloud center and data centre information system | |
| CN112328960B (en) | Optimization method and device for data operation, electronic equipment and storage medium | |
| CN115062567B (en) | Condensation operation method and device for adjacent node set in graph data and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |