CN116401512A

CN116401512A - Data screening method, network index prediction method, device, equipment and medium

Info

Publication number: CN116401512A
Application number: CN202310362071.5A
Authority: CN
Inventors: 王梓旭; 李洋; 陆彦辉
Original assignee: Shenzhen Research Institute of Big Data SRIBD
Current assignee: Shenzhen Research Institute of Big Data SRIBD
Priority date: 2023-03-31
Filing date: 2023-03-31
Publication date: 2023-07-07

Abstract

The embodiment of the application provides a data screening method, a network index prediction method, a device, equipment and a medium, and belongs to the technical field of artificial intelligence and network communication. The method comprises the following steps: acquiring original prediction data and actual network index data of a network index to be predicted; performing dimension reduction processing on the original predicted data to obtain preliminary predicted data; screening from the original prediction data to obtain basic prediction data and data to be evaluated; inputting the basic prediction data and the data to be evaluated into a preset prediction model to perform network index prediction, so as to obtain predicted network index data; obtaining first error data of the data to be evaluated according to the predicted network index data and the actual network index data; sorting the data to be evaluated according to the first error data to obtain data to be excluded; and filtering the data to be excluded from the preliminary predicted data to obtain target predicted data. According to the method and the device, the calculated amount of the prediction model can be reduced, and therefore the prediction efficiency is improved.

Description

Data screening method, network index prediction method, device, equipment and medium

Technical Field

The present disclosure relates to the field of artificial intelligence technologies and network communication technologies, and in particular, to a data screening method, a network index prediction method, a device, equipment, and a medium.

Background

The user experience rate refers to a data rate obtained by a user in a unit time, and is used for representing a transmission rate obtained by the user in a real network environment. From this, it can be seen that the user experience rate can be used as a performance index for network optimization. In the related art, a machine learning method such as a deep neural network is used for predicting the user experience rate. Specifically, a prediction model is built, data collected from a real network environment is used as input of the prediction model, and output of the prediction model is the user experience rate. However, in the above method, data which is not actually related to the user experience rate is easy to exist in the input data of the model, so that the calculation amount of the prediction model is increased. Therefore, how to screen the input data to reduce the calculation amount of the prediction model, so as to improve the prediction efficiency is a technical problem to be solved urgently.

Disclosure of Invention

The main purpose of the embodiments of the present application is to provide a data screening method, a network index prediction method, a device, equipment and a medium, which aim to reduce the calculation amount of a prediction model, thereby improving the prediction efficiency.

To achieve the above object, a first aspect of an embodiment of the present application provides a data screening method, where the method includes:

acquiring original prediction data and actual network index data of a network index to be predicted;

performing dimension reduction processing on the original predicted data to obtain preliminary predicted data;

screening the original prediction data to obtain basic prediction data and data to be evaluated;

inputting the basic prediction data and the data to be evaluated into a preset prediction model to perform network index prediction, so as to obtain predicted network index data;

obtaining first error data of the data to be evaluated according to the predicted network index data and the actual network index data;

sorting the data to be evaluated according to the first error data to obtain data to be excluded;

and filtering the data to be excluded from the preliminary predicted data to obtain target predicted data.

In some embodiments, the performing the dimension reduction on the original predicted data to obtain preliminary predicted data includes:

performing data classification on the original predicted data to obtain a data set;

performing a loop operation until all of the data sets are traversed; wherein the cycling operation comprises: filtering the data set from the original predicted data to obtain first predicted data; inputting the first prediction data into the prediction model to perform network index prediction to obtain first network index data;

Obtaining second error data of the data set according to the first network index data and the actual network index data;

and screening the original prediction data according to the data set corresponding to the second error data to obtain the preliminary prediction data.

In some embodiments, the filtering the original prediction data according to the data set corresponding to the second error data to obtain the preliminary prediction data includes:

sorting the data sets according to the second error data to obtain sorting position data;

adding the data set in a preset original filtered data set according to the sequencing position data to obtain a preliminary filtered data set;

filtering the preliminary filtering data set from the original prediction data to obtain second prediction data;

inputting the second prediction data into the prediction model to perform network index prediction to obtain second network index data;

determining a target data set from the data sets according to the second network index data and the actual network index data;

and obtaining the preliminary prediction data according to the target data set.

In some embodiments, the obtaining the first error data of the data to be evaluated according to the predicted network indicator data and the actual network indicator data includes:

Performing difference calculation on the predicted network index data and the actual network index data to obtain a predicted difference value;

and calculating a ratio according to the predicted difference value and the actual network index data, wherein the first error data.

To achieve the above objective, a second aspect of the embodiments of the present application provides a network indicator prediction method, where a network indicator to be predicted includes a user experience rate, the method includes:

determining a preliminary data type of target prediction data, and acquiring actual prediction data of the user experience rate according to the preliminary data type; wherein the target prediction data is obtained according to the method of the first aspect;

inputting the actual prediction data into the prediction model to predict the user experience rate, so as to obtain an actual rate; wherein the target prediction data is obtained according to the method of the first aspect.

In some embodiments, the prediction model includes an intermediate layer, and the method further includes updating the prediction model prior to the determining the preliminary data type of the target prediction data, specifically including:

inputting target prediction data into the prediction model to perform network index prediction to obtain first network index data;

Increasing or decreasing the number of the intermediate layers to obtain a preliminary prediction model;

inputting the original prediction data into the preliminary prediction model to perform network index prediction to obtain second network index data;

inputting the target prediction data into the preliminary prediction model to perform network index prediction to obtain third network index data;

obtaining third error data according to the first network index data and the actual network index data, obtaining fourth error data according to the second network index data and the actual network index data, obtaining fifth error data according to the third network index data and the actual network index data, comparing the values of the third error data, the fourth error data and the fifth error data, and updating the prediction model according to the preliminary prediction model if the comparison result shows that the value of the third error data is minimum.

To achieve the above object, a third aspect of the embodiments of the present application provides a data screening apparatus, including:

the first data acquisition module is used for acquiring original prediction data and actual network index data of the network index to be predicted;

the dimension reduction module is used for carrying out dimension reduction processing on the original predicted data to obtain preliminary predicted data;

The screening module is used for screening the original prediction data to obtain basic prediction data and data to be evaluated;

the first prediction module is used for inputting the basic prediction data and the data to be evaluated into a preset prediction model to perform network index prediction to obtain predicted network index data;

the error data determining module is used for obtaining first error data of the data to be evaluated according to the predicted network index data and the actual network index data;

the sorting module is used for sorting the data to be evaluated according to the first error data to obtain data to be excluded;

and the filtering module is used for filtering the data to be excluded from the preliminary predicted data to obtain target predicted data.

To achieve the above object, a fourth aspect of the embodiments of the present application provides a network indicator prediction apparatus, including:

the second data acquisition module is used for determining a preliminary data type of target prediction data and acquiring actual prediction data of the user experience rate according to the preliminary data type; wherein the target prediction data is obtained according to the method of the first aspect;

the second prediction module is used for inputting the actual prediction data into the prediction model to predict the user experience rate so as to obtain the actual rate; wherein the target prediction data is obtained according to the method of the first aspect.

To achieve the above object, a fifth aspect of the embodiments of the present application proposes an electronic device, which includes a memory storing a computer program and a processor implementing the method according to the first aspect or the second aspect when the processor executes the computer program.

To achieve the above object, a sixth aspect of the embodiments of the present application proposes a computer readable storage medium storing a computer program, which when executed by a processor, implements the method of the first aspect or the second aspect.

The data screening method, the network index prediction device, the network index prediction equipment and the medium are used for obtaining preliminary prediction data with smaller data quantity by performing dimension reduction processing on original prediction data. On the basis, filtering the data to be eliminated in the preliminary data to obtain target prediction data with further reduced data quantity. Therefore, when the network index to be predicted is predicted according to the target prediction data and the prediction model, the data amount input into the prediction model can be reduced, so that the calculated amount of the prediction model is reduced, the performance requirement on the computing equipment loading the prediction model is reduced, and the prediction efficiency of the network index to be predicted is improved.

Drawings

Fig. 1 is a flowchart of a data screening method provided in an embodiment of the present application;

FIGS. 2A-2C are schematic diagrams of raw prediction data;

fig. 3A is a flowchart of step S120 in fig. 1;

fig. 3B is a flowchart of step S320 in fig. 3A;

fig. 4 is a flowchart of step S340 in fig. 3A;

FIG. 5 is a schematic diagram of ranking position data provided by an embodiment of the present application;

FIG. 6 is a flowchart of a first error data calculation method provided in an embodiment of the present application;

FIG. 7 is a flowchart of a network indicator prediction method according to an embodiment of the present disclosure;

FIG. 8 is another flowchart of a network indicator prediction method according to an embodiment of the present application

FIGS. 9A-9B are schematic diagrams of experimental results provided in the examples of the present application;

fig. 10 is a schematic structural diagram of a data screening device according to an embodiment of the present disclosure;

fig. 11 is a schematic structural diagram of a network indicator prediction apparatus according to an embodiment of the present application;

fig. 12 is a schematic hardware structure of an electronic device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.

It should be noted that although functional block division is performed in a device diagram and a logic sequence is shown in a flowchart, in some cases, the steps shown or described may be performed in a different order than the block division in the device, or in the flowchart. The terms first, second and the like in the description and in the claims and in the above-described figures, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the present application.

First, several nouns referred to in this application are parsed:

artificial intelligence (artificial intelligence, AI): is a new technical science for researching and developing theories, methods, technologies and application systems for simulating, extending and expanding the intelligence of people; artificial intelligence is a branch of computer science that attempts to understand the nature of intelligence and to produce a new intelligent machine that can react in a manner similar to human intelligence, research in this field including robotics, language recognition, image recognition, natural language processing, and expert systems. Artificial intelligence can simulate the information process of consciousness and thinking of people. Artificial intelligence is also a theory, method, technique, and application system that utilizes a digital computer or digital computer-controlled machine to simulate, extend, and expand human intelligence, sense the environment, acquire knowledge, and use knowledge to obtain optimal results.

Based on the foregoing, embodiments of the present application provide a data screening method, a network indicator prediction method, a device, equipment, and a medium, which aim to improve the calculation amount of a prediction model, thereby improving the prediction efficiency.

The data screening method, the network index prediction method, the device, the equipment and the medium provided by the embodiment of the application are specifically described through the following embodiments, and the data screening method in the embodiment of the application is described first.

The embodiment of the application can acquire and process the related data based on the artificial intelligence technology. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results.

Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.

The embodiment of the application provides a data screening method, which relates to the technical field of artificial intelligence and the technical field of network communication. The data screening method provided by the embodiment of the application can be applied to a terminal, a server side and software running in the terminal or the server side. In some embodiments, the terminal may be a smart phone, tablet, notebook, desktop, etc.; the server side can be configured as an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, and a cloud server for providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, basic cloud computing services such as big data and artificial intelligent platforms and the like; the software may be an application or the like that implements the data screening method, but is not limited to the above form.

The subject application is operational with numerous general purpose or special purpose computer system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

In the embodiments of the present application, when related processing is performed according to user information, user behavior data, data of a terminal used by a user, and the like, permission or consent of the user is obtained first, and the collection, use, processing, and the like of the data comply with related laws and regulations and standards of related countries and regions. In addition, when the embodiment of the application needs to acquire the sensitive personal information of the user, the independent permission or independent consent of the user is acquired through a popup window or a jump to a confirmation page or the like, and after the independent permission or independent consent of the user is explicitly acquired, necessary user related data for enabling the embodiment of the application to normally operate is acquired.

Fig. 1 is an optional flowchart of a data screening method according to an embodiment of the present application, where the method in fig. 1 may include, but is not limited to, steps S110 to S170.

Step S110, obtaining original prediction data and actual network index data of a network index to be predicted;

step S120, performing dimension reduction processing on the original predicted data to obtain preliminary predicted data;

step S130, screening from the original predicted data to obtain basic predicted data and data to be evaluated;

step S140, inputting the basic prediction data and the data to be evaluated into a preset prediction model for predicting network indexes to obtain predicted network index data;

step S150, obtaining first error data of the data to be evaluated according to the predicted network index data and the actual network index data;

step S160, sorting the data to be evaluated according to the first error data to obtain data to be excluded;

step S170, filtering the data to be excluded from the preliminary predicted data to obtain target predicted data.

In this embodiment of the present application, the network index to be predicted refers to a network index to be predicted, for example, may be an index related to network communication, such as a user experience rate, a delay, a regional transmission capacity, a peak data rate, and the like, which is not specifically limited. For convenience of explanation, however, the embodiment of the present application takes the network indicator to be predicted as the user experience rate as an example.

In step S110 of some embodiments, the actual network index data refers to actual data of the network index to be predicted in actual situations, and the original prediction data refers to data related to the network index to be predicted. When the network index to be predicted is taken as the user experience rate, the actual network index data is a true value corresponding to the user experience rate obtained in the actual network environment, and the original predicted data is data related to the user experience rate. It will be appreciated that the description of the embodiments herein as "related" refers to the following: when the B data can be predicted from the a data, it is indicated that the a data is related to the B data. As shown in fig. 2A-2C, the data related to the user experience rate (i.e., raw predicted data) includes 74-dimensional data. The 74-dimensional data may be acquired from a user's terminal, or a base station, and the 74-dimensional data is merely exemplary, i.e., data addition or subtraction may also be performed according to actual circumstances.

In step S120 of some embodiments, the dimension reduction process refers to an operation of converting high-dimensional data into low-dimensional data. The preliminary prediction data is data obtained by performing dimension reduction processing on the original prediction data, so that the data volume of the preliminary prediction data is smaller than that of the original prediction data. The dimension reduction processing can be any one of an empirical method, a measurement algorithm, a statistical analysis method (including variance filtering, chi-square filtering and the like) based on a statistical analysis method, a machine learning algorithm and the like. The specific method of the dimension reduction processing in the embodiment of the present application will be described below.

In step S130 of some embodiments, the basic prediction data refers to data that has weak relevance to the network indicator to be predicted in the original prediction data, and the data to be evaluated refers to all data except the basic prediction data in the original prediction data. For example, the base prediction data is 28-dimensional data, and the data to be evaluated is 46-dimensional data. It is to be understood that the method for determining the basic prediction data may be any of a historical empirical method, a machine learning algorithm, and the like, and the embodiment of the present application is not particularly limited.

In step S140 and step S150 of some embodiments, the prediction model is a pre-built model with network index prediction capabilities to be predicted. The model structure of the prediction model in the embodiment of the present application is not particularly limited. The basic prediction data and one piece of data to be evaluated are used as input data of a prediction model, so that the prediction model can predict and obtain a predicted value of a network index to be predicted (i.e. predicting network index data, such as predicted user experience rate) according to the basic prediction data and the data to be evaluated. Since the basic prediction data is data which has weak relativity with the network index to be predicted, the obtained prediction network index data is greatly influenced by the input data to be evaluated. Thus, in step S150, first error data representing a prediction error can be obtained from the predicted network index data and the actual network index data. A specific calculation method of the first error data will be described below. The operations described in step S140 and step S150 are performed on each data to be evaluated, thereby determining first error data of each data to be evaluated.

In step S160 and step S170 of some embodiments, the data to be evaluated is sorted according to the first error data, e.g. sorted according to the magnitude of the value, in a manner from big to small or from small to big. Taking the order of the values from large to small as an example, if the value of the first error data is small, the prediction error of the prediction model becomes small after the corresponding data to be evaluated and the basic prediction data with weaker relativity are input into the prediction model, namely, the corresponding data to be evaluated and the network index to be predicted are shown to have stronger relativity; if the first error data has a large value, the corresponding data to be evaluated and the basic prediction data with weaker relativity are input into the prediction model, and then the prediction error of the prediction model becomes larger, namely the corresponding data to be evaluated and the network index to be predicted are weaker relativity. Thus, the data to be evaluated which is ranked first is regarded as data to be excluded. Since the preliminary prediction data obtained in step S120 is the data obtained by the preliminary screening, there may be data with weak relevance in the preliminary prediction data, so in step S170, the data to be excluded in the preliminary prediction data is filtered, and the obtained remaining data (i.e., the target prediction data) are all data with strong relevance to the network index to be predicted. Therefore, the target prediction data is obtained by screening the original prediction data twice, and the data quantity of the target prediction data is reduced to a certain extent compared with that of the original prediction data.

In step S110 to step S170 illustrated in the embodiment of the present application, the initial prediction data with a smaller data size is obtained by performing the dimension reduction processing on the original prediction data. On the basis, filtering the data to be eliminated in the preliminary data to obtain target prediction data with further reduced data quantity. Therefore, when the network index to be predicted is predicted according to the target prediction data and the prediction model, the data amount input into the prediction model can be reduced, so that the calculated amount of the prediction model is reduced, the performance requirement on the computing equipment loading the prediction model is reduced, and the prediction efficiency of the network index to be predicted is improved.

The dimension reduction processing method will be described below. Referring to fig. 3A and 3B, in some embodiments, step S120 includes, but is not limited to including, step S310 through step S340.

Step S310, carrying out data classification on the original predicted data to obtain a data set;

step S320, executing a circulation operation until all data sets are traversed;

step S330, obtaining second error data of the data set according to the first network index data and the actual network index data;

and step S340, screening the original prediction data according to the data group corresponding to the second error data to obtain preliminary prediction data.

The loop operation in step S320 includes, but is not limited to, steps S321 to S322.

Step S321, filtering a data set from original predicted data to obtain first predicted data;

step S322, inputting the first prediction data into the prediction model for network index prediction to obtain first network index data.

In step S310 of some embodiments, the original prediction data is data classified to obtain a plurality of data sets. The data classification basis may be a preset number, an acquisition source, a data type, and the like. The preset number refers to equally dividing the original predicted data, so that each data group includes the same preset number of original predicted data. Acquisition source refers to classifying the original predicted data of the same data source into a set of data sets. The data type refers to classifying the original predicted data of the same data type into a group of data sets. Because the correlation difference between the original predicted data with the same data type and the network index to be predicted is smaller, the data classification is carried out according to the data type, so that the correlation between a plurality of original predicted data with the same data type and the network index to be predicted can be judged at the same time.

In particular, the method of determining the data type may include an identification symbol method, a clustering method, or the like. Wherein, the identification symbol method refers to: since it is known in advance which original predicted data is to be acquired, it is unknown which value the original predicted data corresponds to, it is possible to set a flag (including a character, a color, a label, etc.) to the original predicted data in advance, and set different flags to the original predicted data of different data types, and set the same flag to the original predicted data of the same data type, so that data classification can be performed according to the flag. The clustering method is to classify the original predicted data according to a clustering algorithm. Taking the identifier as an example, referring to fig. 2A to 2C, different identifiers are set for each piece of original prediction data, for example, the 38 th dimension indicates that the identifier of the original prediction data corresponding to the average value of the downlink Rank is AveDLRank, and the 55 th dimension indicates that the identifier of the original prediction data corresponding to the average value of the modulation coding scheme is MIMOAveMCS. The identification characters of the original prediction data with the same data type have the same character prefix, and the character prefixes of the identification characters of the original prediction data with different data types are different. For example, 74-dimensional raw prediction data shown in fig. 2A to 2C may be divided into fifteenth classes, the first class including 1-8-th dimensional data, the second class including 9-16-th dimensional data, the third class including 17-24-th dimensional data, the fourth class including 25-27-th dimensional data, the fifth class including 28-37-th dimensional data, the sixth class including 38-40-th dimensional data, the seventh class including 41-45-th dimensional data, the eighth class including 46-52-th dimensional data, the ninth class including 53-59-th dimensional data, the tenth class including 60-62-th dimensional data, the eleventh class including 63-65-th dimensional data, the twelfth class including 66-68-th dimensional data, the thirteenth class including 69-71-th dimensional data, the fourteenth class including 72-74-th dimensional data, and the fifteenth class including 9-24-th dimensional data. That is, the original prediction data can be divided into fifteen sets of data by the above method.

In steps S321 to S322 of some embodiments, the first prediction data refers to data remaining after filtering any one data set from the original prediction data. And taking the first prediction data as input data of a prediction model, so that the prediction model predicts the network index according to the first prediction data, and obtaining the first network index prediction data. The above operation is performed on each data set until all data sets are traversed, that is, the above operation is repeated for fifteen times.

In step S330 of some embodiments, second error data of the corresponding data set is calculated according to the first network indicator data and the actual network indicator data, where the second error data is used to represent an error of the prediction model for performing network indicator prediction according to the first prediction data. The specific calculation method of the second error data may refer to the following calculation method of the first error data. For example, still taking fig. 2A to 2C as an example, the second error data corresponding to each data set is shown in table 1 below.

TABLE 1

In table 1, "deleted feature class" refers to the data group corresponding to the deleted feature class, and "remaining data dimension" refers to the data amount of the first predicted data. For example, deleting the first class refers to deleting a data group consisting of 1 st-8 th-dimensional data from 74-dimensional original predicted data, and the remaining first predicted data has 66 dimensions in total. The 66-dimension data are input into a prediction model for network index prediction, and second error data obtained by calculation according to the first network index data obtained by prediction and the actual network index data are 35.79%. It can be understood that the deleted feature class of "0" means that the 74-dimensional original prediction data is input into the prediction model for network index prediction without deletion, and 34.98% of error data is calculated according to the output data of the prediction model and the actual network index data. The other data in Table 1 were obtained in the same manner as described above. The 34.98% error data may be used as reference error data, which serves to constrain the data screening operation, i.e. it should be ensured that the data screening is performed with small variations in the error data, i.e. the data screening operation cannot affect the accuracy of the prediction.

In step S340 of some embodiments, after each data set is deleted from the original predicted data, if the value of the second error data obtained according to the corresponding first predicted data is larger, it indicates that the deleted data set has stronger correlation with the network indicator to be predicted; and if the value of the second error data obtained according to the corresponding first prediction data is smaller, the deleted data set is weaker in relation with the network index to be predicted. Therefore, the data group with strong relativity of the data group and the network index to be predicted can be determined according to the second error data, so that preliminary prediction data can be obtained by screening the original prediction data according to the corresponding data group. Specific screening methods for preliminary prediction data will be described below.

The benefit of step S310 to step S340 is that the primary screening of the original predicted data can be performed, and the screening can be performed in a data set manner, so that the efficiency of the primary screening and the accuracy of the primary screening can be improved to a certain extent.

The following describes a screening method of preliminary prediction data. Referring to fig. 4, in some embodiments, step S340 includes, but is not limited to including, steps S410 through S460.

Step S410, sorting the data sets according to the second error data to obtain sorting position data;

Step S420, adding a data set in a preset original filtered data set according to the ordering position data to obtain a preliminary filtered data set;

step S430, filtering the preliminary filtering data set from the original prediction data to obtain second prediction data;

step S440, inputting the second prediction data into the prediction model for network index prediction to obtain second network index data;

step S450, determining a target data set from the data sets according to the second network index data and the actual network index data;

step S460, preliminary prediction data is obtained according to the target data set.

In step S410 of some embodiments, the data sets are sorted from large to small or from small to large according to the second error data, resulting in sorted position data. The ranking position data refers to the position data of a certain data group in all the ranked data groups, for example, referring to fig. 5, the ranking position data of the data group a is 1, the ranking position data of the data group B is 2, and so on. Since a total of 15 data sets are included, the maximum value of the sort position data is 15.

In step S420 of some embodiments, the original filtered data set is a preset data set, and the original filtered data set is a null data set or a data set including specific data in an initial state. And adding the data sets into the original filtered data sets in turn according to the ordering position data to obtain a plurality of preliminary filtered data sets with different data volumes. Wherein the order of adding the data sets is determined according to the relatedness of the data sets to the network indicator to be predicted. Specifically, the data set with weak relevance is added first, and then the data set with strong relevance is added, so that deviation of judgment of the data set with weak relevance is avoided when the data set with strong relevance is added first. Therefore, when sorting from large to small, the data set with the first sorting is added to the original filtered data set, that is, the data set with the small sorting position data value is added first. When sorting is performed from small to large, the data set with the last sorting is added to the original filtered data set, namely the data set with the large sorting position data value is added. The specific operation of the sequential addition is as follows, in the first addition operation, the data set (such as the data set a) determined according to the above method is added to the original filtered data set in the initial state, so as to obtain a preliminary filtered data set. In the second addition operation, one data set (data set B) is determined again from the remaining data sets using the above-described method, and the data set (data set B) is added to the preliminary filtered data set obtained in the first addition operation to update the preliminary filtered data set. And so on until after a certain addition operation, the updated preliminary filtered data set includes fifteen data sets. Therefore, the sequential addition operation is to continuously add new data sets to update the number of the data sets contained in the preliminary filtering data sets, and finally obtain a plurality of preliminary filtering data sets with different data amounts.

In steps S430 to S440 of some embodiments, one preliminary filtered data set is filtered out from the original predicted data, and the remaining data is used as the second predicted data. And taking the second prediction data as input data of a prediction model, so that the prediction model predicts the network index to be predicted according to the second prediction data, and second network index data is obtained. Error data is calculated according to the second network index data and the actual network index data, and the calculation method of the error data can refer to the calculation method of the first error data. And carrying out the operation on each primary filtered data set to obtain a plurality of error data. For example, error data shown in table 2 below is obtained.

TABLE 2

Because the fifteenth class is a special class consisting of the second class and the third class, the fifteenth class is filtered from the original predicted data first, and an error value of 34.79% is calculated according to the filtered residual data and the predicted model, which is different from the method for determining the preliminary filtered data set. That is, the data set corresponding to the fifteenth class is taken as the specific data included in the original filtered data set, the data set corresponding to the fourteenth class is added, the original filtered data set is updated according to the data set corresponding to the fourteenth class, and the preliminary filtered data set (including the data set corresponding to the fifteenth class and the data set corresponding to the fourteenth class) is obtained, and at this time, an error value of 34.89% is obtained. And adding a data set corresponding to the twelfth class on the basis of the operation, updating the preliminary filtering data set (comprising the data set corresponding to the fifteenth class, the data set corresponding to the fourteenth class and the data set corresponding to the twelfth class) according to the data set corresponding to the twelfth class, and calculating to obtain a 34.71% error value, and the like.

In step S450 of some embodiments, when the error value calculated according to the second network index data and the actual network index data obtained from a certain updated preliminary filtered data set is larger, it indicates that the data set triggering the preliminary filtered data set update has stronger correlation with the network index to be predicted, and the data set is used as the target data set. It will be appreciated that the "trigger" described in the embodiments of the present application is used to represent the following: when the data set H is added to the preliminary filtering data set obtained in the previous operation, the preliminary filtering data set is updated according to the data set H, and the data set H can trigger the preliminary filtering data set to be updated.

It can be understood that the number of the target data sets can be adaptively set according to practical situations, and the examples of the application are not particularly limited. However, for convenience of explanation, in table 2, since the error value corresponding to the addition of the seventh class, the addition of the fifth class, the addition of the first class, and the addition of the sixth class is the largest, the following four data sets are used as the target data sets in the embodiment of the present application: a data set corresponding to the seventh class, a data set corresponding to the fifth class, a data set corresponding to the first class, and a data set corresponding to the sixth class.

In step S460 of some embodiments, the determined plurality of target data sets are combined to obtain preliminary prediction data. For example, the four obtained target data sets are combined to obtain 28-dimensional preliminary prediction data.

The benefit of step S410 to step S460 is that, by sequentially adding the data sets to the sorted position data to update the method of primarily filtering the data sets, the strength of each data set relative to the data to be predicted can be accurately determined, and the influence of the data set with strong relativity on the data set with weak relativity can be avoided, so that the accuracy of determining the target data set can be improved.

Next, in connection with the examples of the above embodiments, the steps S130 to S170 will be explained.

Referring to fig. 2A to 2C, the 1 st to 27 th dimensional data, and the 60 th dimensional data, 28 th dimensional raw prediction data in total are taken as basic prediction data, and the remaining raw prediction data are taken as data to be evaluated. It will be appreciated that the 1 st to 8 th dimensions are generally used in combination due to the specificity of wave velocity. In one operation, the basic prediction data and a piece of data to be evaluated are taken as input data of a prediction model, and first error data is obtained. The above operation is performed on all the data to be evaluated, and the first error data shown in table 3 below is obtained.

/>

TABLE 3 Table 3

In table 3, adding "0" to the data to be evaluated indicates that only the basic prediction data is taken as the input data of the prediction model. The change condition of the first error data after the data to be evaluated is added can be determined according to the table 3, and then the strength of the correlation between each data to be evaluated and the network index to be predicted can be determined. For example, after adding a certain data to be evaluated, if the first error data is greater than 62.99%, it indicates that the data to be evaluated has weaker relevance to the network index to be evaluated; if the first error data is smaller than 62.99%, the data to be evaluated is more relevant to the network index to be evaluated.

In some embodiments, referring to fig. 6, the method for calculating the first error data includes steps S610 to S620.

Step S610, performing difference calculation on the predicted network index data and the actual network index data to obtain a predicted difference;

step S620, calculating the ratio according to the predicted difference and the actual network index data, and obtaining first error data.

As can be seen from table 3, the first error data substantially refers to the absolute percentage value of the predicted network index data and the actual network index data. Therefore, in step S610, the difference is first obtained between the predicted network index data and the actual network index data, and the absolute value of the difference is used as the predicted difference. In step S620, a ratio of the predicted difference to the actual network index data is calculated to obtain first error data.

After the first error value of each data to be evaluated is obtained according to the method, the data to be evaluated can be ranked according to the first error data, and preliminary excluded data can be determined according to the ranking result. It is understood that the preliminary exclusion data refers to data having weak correlation with the predicted network index among the plurality of data to be evaluated. The preliminary exclusion data is then combined to obtain combined data, which may be any combination, empirically, according to a data type, according to a relativity, etc., and the embodiment of the present application is not particularly limited. And combining to obtain various combined data, filtering the combined data in the preliminary predicted data, and taking the rest data as input data of a prediction model to obtain predicted data and further obtain error data. And finally, taking the combined data corresponding to the error data with smaller difference value of the reference error data as data to be excluded, and filtering the data to be excluded from the preliminary predicted data, wherein the data quantity (namely the data dimension) of the residual data is smaller. The method has the advantage that the data quantity can be further reduced on the basis of preliminary prediction data under the condition that the prediction accuracy is basically unchanged. Table 4 below is an illustration of filtering out combined data in preliminary prediction data.

TABLE 4 Table 4

From the error data in table 4, it can be determined that the combined data in table 4, in which the data dimension is 23 and the error data is 34.92%, is the data to be excluded. Filtering the data to be excluded from the 28-dimensional preliminary prediction data obtained according to steps S410 to S460 to obtain 23-dimensional target prediction data as shown in table 5 below:

top0-7BeamRsrp
	AveDLRank、AveDLMCS、AveDLUEPRBUsed
MIMOAveDLSchRankRb
	MIMOAveDLSchRb
MIMOAveMCS
	MIMOAveDLMuSchRankRb
MIMOAveDLMuSchRb
	MIMOAveMuMCS
MIMOAveBeamPair
	UEMeasInfo.AccumMeasItems.DLPrbUsedNum
UEMeasInfo.AccumMeasItems.DlIniTranNackNumCode0
	UEMeasInfo.AccumMeasItems.DlIniTranTotalNumCode0
UEMeasInfo.AccumMeasItems.DlRetranNumCode0
	UEMeasInfo.AccumMeasItems.DlMCSCode0

TABLE 5

Referring to fig. 7, the embodiment of the present application further provides a network indicator prediction method, where the network indicator to be predicted is a user experience rate, and the network indicator prediction method includes, but is not limited to, steps S710 to S720.

Step 710, determining a preliminary data type of the target prediction data, and obtaining actual prediction data of the user experience rate according to the preliminary data type;

and step 720, inputting the actual prediction data into a prediction model to perform user experience rate prediction, so as to obtain an actual rate.

In step S710 of some embodiments, a preliminary data type of the target prediction data determined according to the above embodiments is acquired, the preliminary data type being used to represent a data type of each target prediction data. In practical application, corresponding data in a real network is obtained according to the data type, and practical prediction data is obtained. Taking table 5 as an example, 23-dimensional actual prediction data will be obtained.

In step S710 of some embodiments, the actual prediction data is used as input data of the prediction model obtained according to the above embodiments, so as to predict the user experience rate, and obtain the actual rate.

The benefit of steps S710 to S720 is that the amount of data of the obtained actual predicted data can be reduced, thereby improving the prediction efficiency of the user experience rate.

In some embodiments, before step S710 is performed, that is, before the actual prediction is performed, experimental comparison may be performed on the prediction model, the target prediction data, and the original prediction data, so as to determine feasibility of the data screening method described in the foregoing embodiments, thereby improving accuracy of prediction in the actual prediction.

The test comparison method will be described below. Referring to fig. 8, the network index prediction method further includes, but is not limited to, including steps S810 to S850 before step S710.

Step S810, inputting target prediction data into a prediction model to perform network index prediction to obtain first network index data;

step S820, increasing or decreasing the number of intermediate layers to obtain a preliminary prediction model;

step S830, inputting the original prediction data into the preliminary prediction model for network index prediction to obtain second network index data;

Step S840, inputting the target prediction data into the preliminary prediction model for network index prediction to obtain third network index data;

step S850, obtaining third error data according to the first network index data and the actual network index data, obtaining fourth error data according to the second network index data and the actual network index data, obtaining fifth error data according to the third network index data and the actual network index data, comparing the values of the third error data, the fourth error data and the fifth error data, and updating the prediction model according to the preliminary prediction model if the comparison result shows that the value of the third error data is minimum.

In some embodiments, a model structure of a predictive model includes an input layer, an intermediate layer, and an output layer, wherein the intermediate layer includes a hidden layer. The hidden layer is used for abstracting the characteristics of the input data to another dimension space so as to show the more abstract characteristics of the input data, thereby realizing better linear division. When the hidden layer comprises multiple layers, the hidden layer is an abstraction of multiple layers of features of the input data. It follows that the number of hidden layers is related to the output of the predictive model. Therefore, in the embodiment of the application, the number of intermediate layers is changed on the basis of an original prediction model, and the dimension of input data is changed to determine the influence of input data with different layers and different dimensions on a prediction result.

Specifically, in step S810 of some embodiments, the target prediction data is input to the prediction model to perform network index prediction, so as to obtain first network index data, where the first network index data is a prediction result obtained according to the number of intermediate layers of the original prediction model and the screened target prediction data.

In step S820 of some embodiments, the number of intermediate layers of the original prediction model is increased or decreased, for example, the number of layers of the prediction model in step S810 is 6, and this step adds 1 layer to the intermediate layers, resulting in a preliminary prediction model with the number of intermediate layers being 7. It will be appreciated that the number of intermediate layers added each time may be the same or different, for example 1 layer is added to this step, 2 layers, 3 layers, etc. may be added subsequently. The operation of reducing the number of intermediate layers is similar to the operation of increasing the number of intermediate layers described above, and will not be described again.

In steps S830 to S840 of some embodiments, the original prediction data and the target prediction data are input to the preliminary prediction model to perform network index prediction, so as to determine the difference of the prediction results obtained by the same prediction model according to the input data with different dimensions.

In step S850 of some embodiments, performing error calculation on the first network index data and the actual network index data to obtain third error data; performing error calculation on the second network index data and the actual network index data to obtain fourth error data; and carrying out error calculation on the third network index data and the actual network index data to obtain fifth error data. And comparing the reference error data, the third error data, the fourth error data and the fifth error data in numerical value, and updating the prediction model according to the number of intermediate layers corresponding to the error data with the minimum numerical value. For example, as shown in table 6 below.

TABLE 6

In table 6, the number of intermediate layers of the original prediction model is 6, and 34.98% of error data (reference error data) is obtained when the original prediction data is used as input data; when the target prediction data is taken as input data, 34.92% error data is obtained. Increasing the number of intermediate layers of the prediction model, and increasing the intermediate layers from 6 layers to 7 layers, wherein when the original prediction data is taken as input data, 35.09% of error data is obtained; when the target prediction data is taken as input data, 35.03% error data is obtained. Reducing the number of intermediate layers of the prediction model, and reducing the intermediate layers from 6 layers to 5 layers, wherein 34.57% of error data are obtained when the original prediction data are taken as input data; when the target prediction data is taken as input data, 34.94% error data is obtained. Comparing the six error data, determining that the number of intermediate layers is 6 layers according to the comparison result, and when the dimension of input data is 23 dimensions, the corresponding error data is the smallest, so that the number of layers of the prediction model is not updated in the embodiment of the application. This has the advantage that it is possible to determine whether the target prediction data can replace the original prediction data.

Assuming that the number of intermediate layers is 7, when the dimension of the input data is 23, the corresponding error data is minimum, and the intermediate layers of the prediction model are updated to 7 layers. At this time, in steps S710 to S720, actual application is performed using the prediction model with 7 layers as intermediate layers.

Taking experimental data with 6 middle layers as an example, assume that each middle layer uses Relu as the activation function and dropout=0.2 is chosen to prevent model overfitting. The iteration number of the prediction model is selected to be 25, and the learning rate is set to be 10 ^-4 . When training the prediction model, the training set and the test set are divided according to the ratio of 8:2. Referring to fig. 9A, a diagram of error data corresponding to the 74-dimensional data (i.e., original prediction data) as input data of a prediction model and performing iterative training on the prediction model for 25 times according to the prediction result and actual network index data is shown. Referring to fig. 9B, a diagram of corresponding error data is shown when 23-dimensional data (i.e., target prediction data) is used as input data of a prediction model, and the prediction model is trained for 25 iterations according to the prediction result and actual network index data. By comparing fig. 9A and 9B, it is found that 23-dimensional data does not negatively affect the prediction result, and thus the user experience rate can be predicted using 23-dimensional data instead of 74-dimensional data. It will be appreciated that the error data in tables 1 to 4 and 6 are all obtained according to the converged prediction model, and the convergence condition includes reaching the preset iteration number (25 times), the loss function decreasing by not more than the preset loss function threshold, and the like as described above.

Referring to fig. 10, an embodiment of the present application further provides a data screening device, which may implement the data screening method, where the device includes:

a first data obtaining module 1010, configured to obtain original prediction data and actual network index data of a network index to be predicted;

the dimension reduction module 1020 is configured to perform dimension reduction processing on the original predicted data to obtain preliminary predicted data;

the screening module 1030 is configured to screen out basic prediction data and data to be evaluated from the original prediction data;

the first prediction module 1040 is configured to input the basic prediction data and the data to be evaluated into a preset prediction model to perform network index prediction, so as to obtain predicted network index data;

the error data determining module 1050 is configured to obtain first error data of the data to be evaluated according to the predicted network index data and the actual network index data;

the sorting module 1060 is configured to sort the data to be evaluated according to the first error data, so as to obtain data to be excluded;

and a filtering module 1070, configured to filter the data to be excluded from the preliminary predicted data to obtain target predicted data.

The specific implementation of the data screening device is basically the same as the specific embodiment of the data screening method, and will not be described herein.

Referring to fig. 11, the embodiment of the present application further provides a network index prediction device, which may implement the network index prediction method, where the device includes:

a second data obtaining module 1110, configured to determine a preliminary data type of the target predicted data, and obtain actual predicted data of the user experience rate according to the preliminary data type;

the second prediction module 1120 is configured to input actual prediction data into the prediction model to perform user experience rate prediction, so as to obtain an actual rate;

the specific implementation of the network index prediction device is basically the same as the specific embodiment of the network index prediction method, and will not be described herein.

The embodiment of the application also provides electronic equipment, which comprises a memory and a processor, wherein the memory stores a computer program, and the processor realizes the data screening method or the network index prediction method when executing the computer program. The electronic equipment can be any intelligent terminal including a tablet personal computer, a vehicle-mounted computer and the like.

Referring to fig. 12, fig. 12 illustrates a hardware structure of an electronic device according to another embodiment, the electronic device includes:

the processor 1210 may be implemented by a general-purpose CPU (central processing unit), a microprocessor, an application-specific integrated circuit (ApplicationSpecificIntegratedCircuit, ASIC), or one or more integrated circuits, etc. for executing related programs to implement the technical solutions provided by the embodiments of the present application;

Memory 1220 may be implemented in the form of read-only memory (ReadOnlyMemory, ROM), static storage, dynamic storage, or random access memory (RandomAccessMemory, RAM). The memory 1220 may store an operating system and other application programs, and when the technical solutions provided in the embodiments of the present application are implemented by software or firmware, relevant program codes are stored in the memory 1220, and the processor 1210 invokes the data screening method or the network index prediction method to execute the embodiments of the present application;

an input/output interface 1230 for implementing information input and output;

the communication interface 1240 is configured to implement communication interaction between the device and other devices, and may implement communication in a wired manner (e.g. USB, network cable, etc.), or may implement communication in a wireless manner (e.g. mobile network, WIFI, bluetooth, etc.);

bus 1250 for transferring information between the various components of the device (e.g., processor 1210, memory 1220, input/output interface 1230, and communication interface 1240);

wherein processor 1210, memory 1220, input/output interface 1230 and communication interface 1240 are communicatively coupled to each other within the device via bus 1250.

The embodiment of the application also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program realizes the data screening method or the network index prediction method when being executed by a processor.

The memory, as a non-transitory computer readable storage medium, may be used to store non-transitory software programs as well as non-transitory computer executable programs. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory remotely located relative to the processor, the remote memory being connectable to the processor through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The embodiments described in the embodiments of the present application are for more clearly describing the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application, and as those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

It will be appreciated by those skilled in the art that the technical solutions shown in the figures do not constitute limitations of the embodiments of the present application, and may include more or fewer steps than shown, or may combine certain steps, or different steps.

The above described apparatus embodiments are merely illustrative, wherein the units illustrated as separate components may or may not be physically separate, i.e. may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

Those of ordinary skill in the art will appreciate that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof.

The terms "first," "second," "third," "fourth," and the like in the description of the present application and in the above-described figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that embodiments of the present application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

It should be understood that in this application, "at least one" means one or more, and "a plurality" means two or more. "and/or" for describing the association relationship of the association object, the representation may have three relationships, for example, "a and/or B" may represent: only a, only B and both a and B are present, wherein a, B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b or c may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the above-described division of units is merely a logical function division, and there may be another division manner in actual implementation, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described above as separate components may or may not be physically separate, and components shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including multiple instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of the various embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing a program.

Preferred embodiments of the present application are described above with reference to the accompanying drawings, and thus do not limit the scope of the claims of the embodiments of the present application. Any modifications, equivalent substitutions and improvements made by those skilled in the art without departing from the scope and spirit of the embodiments of the present application shall fall within the scope of the claims of the embodiments of the present application.

Claims

1. A method of data screening, the method comprising:

2. The method of claim 1, wherein performing the dimension reduction on the raw prediction data to obtain preliminary prediction data comprises:

3. The method according to claim 2, wherein the screening the original prediction data according to the data set corresponding to the second error data to obtain the preliminary prediction data includes:

and obtaining the preliminary prediction data according to the target data set.

4. A method according to any one of claims 1 to 3, wherein said deriving first error data of said data to be evaluated from said predicted network metric data and said actual network metric data comprises:

5. A network indicator prediction method, wherein a network indicator to be predicted includes a user experience rate, the method comprising:

Determining a preliminary data type of target prediction data, and acquiring actual prediction data of the user experience rate according to the preliminary data type; wherein the target prediction data is obtained according to the method of any one of claims 1 to 4;

inputting the actual prediction data into the prediction model to predict the user experience rate, so as to obtain an actual rate; wherein the target prediction data is obtained according to the method of any one of claims 1 to 4.

6. The method according to claim 5, wherein the predictive model comprises an intermediate layer, the method further comprising updating the predictive model prior to said determining the preliminary data type of the target predictive data, in particular comprising:

7. A data screening apparatus, the apparatus comprising:

8. A network indicator prediction apparatus, the apparatus comprising:

the second data acquisition module is used for determining a preliminary data type of target prediction data and acquiring actual prediction data of the user experience rate according to the preliminary data type; wherein the target prediction data is obtained according to the method of any one of claims 1 to 4;

the second prediction module is used for inputting the actual prediction data into the prediction model to predict the user experience rate so as to obtain the actual rate; wherein the target prediction data is obtained according to the method of any one of claims 1 to 4.

9. An electronic device comprising a memory storing a computer program and a processor implementing the method of any one of claims 1 to 4 or the method of claim 5 or 6 when the computer program is executed by the processor.

10. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the method of any one of claims 1 to 4 or implements the method of claim 5 or 6.