CN116303100A

CN116303100A - File integration test method and system based on big data platform

Info

Publication number: CN116303100A
Application number: CN202310560435.0A
Authority: CN
Inventors: 柳宁波; 安迪; 宋少鸿
Original assignee: Suzhou Yingtiandi Information Technology Co ltd
Current assignee: Suzhou Yingtiandi Information Technology Co ltd
Priority date: 2023-05-18
Filing date: 2023-05-18
Publication date: 2023-06-23
Anticipated expiration: 2043-05-18
Also published as: CN116303100B

Abstract

The embodiment of the application provides a file integration test method and system based on a big data platform, which are used for analyzing different file integration stability test data, so that stability test cases corresponding to different abnormal load state information are weakened, the stability test cases which can not analyze an abnormal load path of a labeling test node from target file integration stability test data are weakened, the stability test cases which can analyze the abnormal load path of the labeling test node from the target file integration stability test data are strengthened, a description vector set which is more convenient for extracting the abnormal load path of the labeling test node from the target file integration stability test data is generated, and the labeling test node can extract a specific abnormal load path from the target file integration stability test data more accurately under the condition of strengthening a loss characteristic value between the labeling test node and a test scheduling node.

Description

File integration test method and system based on big data platform

Technical Field

The application relates to the technical field of computers, in particular to a file integration test method and system based on a big data platform.

Background

Currently, in the analysis and integration flow of special dat compressed large files by docking ftp and sftp based on a large data platform, the integrated file system corresponding to the cloud end is required to be processed, so that the stability of the integrated file system is also related to the analysis and integration reliability of the subsequent large data platform. Based on the above, stability test needs to be performed on the integrated file system so as to timely discover possible abnormal load paths, thereby providing basis for subsequent system optimization. However, it is difficult for the existing scheme to accurately extract a specific abnormal load path from the target file integration stability test data.

Disclosure of Invention

In view of the foregoing, an object of the present application is to provide a file integration testing method and system based on a big data platform.

According to a first aspect of the present application, a file integration test method based on a big data platform is provided, and the method is applied to a cloud computing system, and includes:

acquiring target file integration stability test data of an integrated file system aiming at a big data platform, wherein the target file integration stability test data comprises a labeling test node;

carrying out abnormal load state prediction on the test feedback vector in the target file integration stability test data to generate abnormal load state information in the target file integration stability test data;

Obtaining target test running situation information corresponding to the stability test instance according to an abnormal load characteristic value of test execution operation in the stability test instance corresponding to the abnormal load state information and a loss characteristic value between the set abnormal load characteristic value, wherein corresponding relation information exists between the stability test instance and the target test running situation information;

according to the corresponding relation information between the stability test instance and the target test running situation information, vector description is carried out on the stability test instance by the target test running situation information, and a description vector set corresponding to the integrated stability test data of the target file is generated;

and according to the description vector set, carrying out abnormal load path analysis on the labeling test node from the target file integration stability test data, and generating an abnormal load path of the labeling test node in the target file integration stability test data.

For some exemplary design ideas, the obtaining the target test running situation information corresponding to the stability test case according to the abnormal load characteristic value of the test execution operation in the stability test case corresponding to the abnormal load state information and the loss characteristic value between the set abnormal load characteristic value includes:

Acquiring initial test running situation information, wherein the initial test running situation information is predetermined test running situation information;

and adjusting the situation coefficient of the initial test running situation information according to the abnormal load characteristic value of the test execution operation in the stability test instance corresponding to the abnormal load state information and the loss characteristic value between the set abnormal load characteristic value, and generating target test running situation information corresponding to the stability test instance.

For some exemplary design ideas, the adjusting the situation coefficient of the initial test running situation information according to the abnormal load characteristic value of the test execution operation in the stability test instance corresponding to the abnormal load state information and the loss characteristic value between the set abnormal load characteristic values includes:

determining a plurality of test events in a stability test case corresponding to the abnormal load state information, wherein the plurality of test events comprise test events corresponding to the abnormal load state information, and the plurality of test events respectively correspond to test execution operations;

determining an abnormal load characteristic value corresponding to the stability test instance according to the access resource histogram data among the test execution operations corresponding to the plurality of test events;

Assuming that the abnormal load characteristic value corresponding to the stability test case is smaller than the set abnormal load characteristic value, reducing the situation coefficient of the test operation situation information by a first appointed updating coefficient; or, assuming that the abnormal load characteristic value corresponding to the stability test case is greater than the set abnormal load characteristic value, and increasing the situation coefficient of the test operation situation information by using a second specified updating coefficient.

For some exemplary design considerations, the method further comprises:

acquiring test execution operations respectively corresponding to a plurality of resource scheduling components in the integrated stability test data of the target file;

acquiring component abnormal load characteristic values corresponding to the resource scheduling components according to the plurality of resource scheduling components and test execution operations corresponding to the plurality of resource scheduling components respectively;

calculating component abnormal load characteristic values corresponding to the resource scheduling components respectively, and determining an abnormal load average characteristic value of the target file integration stability test data;

and determining the set abnormal load characteristic value corresponding to the target file integration stability test data according to the abnormal load average characteristic value.

For some exemplary design ideas, the obtaining, according to the plurality of resource scheduling components and the test execution operations corresponding to the plurality of resource scheduling components, component abnormal load feature values corresponding to the plurality of resource scheduling components respectively includes:

determining an attention resource scheduling component and N resource scheduling components in a target scheduling interval of the attention resource scheduling component when acquiring component abnormal load characteristic values of the attention resource scheduling components in the plurality of resource scheduling components;

determining an abnormal component load characteristic value corresponding to the attention resource scheduling component according to the test execution operation corresponding to the N resource scheduling components and the access resource histogram data of the test execution operation corresponding to the attention resource scheduling component;

and acquiring component abnormal load characteristic values respectively corresponding to the plurality of resource scheduling components.

For some exemplary design ideas, the generating a description vector set corresponding to the integrated stability test data of the target file according to the correspondence information between the stability test instance and the target test operation situation information, and using the target test operation situation information to perform vector description on the stability test instance includes:

Acquiring at least two pieces of abnormal load state information, wherein the at least two pieces of abnormal load state information comprise first abnormal load state information and second abnormal load state information;

acquiring a first stability test case corresponding to the first abnormal load state information and a second stability test case corresponding to the second abnormal load state information, wherein the first stability test case and the second stability test case are different;

vector description is carried out on the first stability test instance according to first target test running situation information corresponding to the first stability test instance, and a first description vector set corresponding to the first stability test instance is generated;

vector description is carried out on the second stability test instance according to second target test running situation information corresponding to the second stability test instance, and a second description vector set corresponding to the second stability test instance is generated; and generating the description vector set corresponding to the target file integrated stability test data according to the first description vector set and the second description vector set.

For some exemplary design considerations, the generating the description vector set corresponding to the target file integrated stability test data according to the first description vector set and the second description vector set includes:

Acquiring a priori stability test case, wherein the priori stability test case represents a stability test case in which the abnormal load state information does not exist in the integrated stability test data of the target file;

vector description is carried out on the prior stability test instance by using initial test running situation information, and a resource scheduling component description vector set corresponding to the prior stability test instance is generated;

and fusing the description vector set of the resource scheduling component with the first description vector set and the second description vector set to generate the description vector set corresponding to the target file integrated stability test data.

For some exemplary design ideas, the analyzing the abnormal load path of the labeling test node from the target file integrated stability test data according to the description vector set, generating the abnormal load path of the labeling test node in the target file integrated stability test data includes:

inputting the description vector set into a target training neural network meeting model convergence conditions;

clustering the description vector set by using the target training neural network, and determining a clustering result, wherein the clustering is used for clustering test fragments with the same abnormal description category in the target file integrated stability test data, and the abnormal description category characterizes a preset test execution operation interval;

According to the clustering result, carrying out abnormal load path extraction on the labeling test nodes in the target file integration stability test data, and outputting the abnormal load paths of the labeling test nodes in the target file integration stability test data from the target training neural network;

the step of extracting the abnormal load path of the labeling test node in the target file integration stability test data according to the clustering result, and outputting the abnormal load path of the labeling test node in the target file integration stability test data from the target training neural network comprises the following steps:

extracting abnormal load paths from marked test nodes and test scheduling nodes in the target file integration stability test data by using the test execution operation state in the clustering result, wherein the test scheduling nodes represent scheduling nodes except the marked test nodes in the target file integration stability test data;

and determining an abnormal load path of the labeling test node in the target file integrated stability test data according to the abnormal load path extraction result of the labeling test node.

For some exemplary design considerations, the method further comprises:

acquiring file integration stability test data, wherein the file integration stability test data comprises test data acquired from a target test log;

performing relevance fusion on the file integration stability test data to generate target file integration stability test data;

segmenting the target file integration stability test data to generate a plurality of target member file integration stability test data in the target file integration stability test data, wherein the plurality of target member file integration stability test data comprises N stability test instances with abnormal load state information;

the obtaining the target test running situation information corresponding to the stability test case according to the abnormal load characteristic value of the test execution operation in the stability test case corresponding to the abnormal load state information and the loss characteristic value between the set abnormal load characteristic value comprises the following steps:

acquiring the integrated stability test data of the target member files with the abnormal load state information from the integrated stability test data of the plurality of target member files as a stability test instance corresponding to the abnormal load state information;

And acquiring target test running situation information corresponding to the stability test case according to the abnormal load characteristic value of the test execution operation in the stability test case corresponding to the abnormal load state information and the loss characteristic value between the set abnormal load characteristic value.

According to a second aspect of the present application, a cloud computing system is provided, where the integrated file system includes a machine-readable storage medium and a processor, the machine-readable storage medium stores machine-executable instructions, and the processor implements the file integration test method based on the big data platform when executing the machine-executable instructions.

According to a third aspect of the present application, there is provided a computer-readable storage medium having stored therein computer-executable instructions that, when executed, implement the foregoing large data platform-based file integration test method.

According to any one of the aspects, in the application, analysis is performed on the stability test data of different file integration, abnormal load estimation is avoided depending on the relation between the stability test data of multiple file integration, so that stability test cases corresponding to different abnormal load state information are weakened, stability test cases which cannot analyze an abnormal load path of a labeling test node from the stability test data of the target file integration are weakened, stability test cases which can analyze the abnormal load path of the labeling test node from the stability test data of the target file integration are strengthened, a description vector set which is more convenient for extracting the abnormal load path of the labeling test node from the stability test data of the target file integration is generated, and a specific abnormal load path of the labeling test node can be extracted from the stability test data of the target file integration more accurately under the condition that loss characteristic values between the labeling test node and a test scheduling node are strengthened.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered limiting in scope, and that other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic flow chart of a file integration test method based on a big data platform according to an embodiment of the present application;

fig. 2 is a schematic component structure of a cloud computing system for implementing the file integration test method based on the big data platform according to the embodiment of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions of the embodiments of the present application will be clearly and completely described below according to the drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are only for the purpose of illustration and description, and are not intended to limit the protection scope of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. A flowchart, as used in this application, illustrates operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow diagrams may be implemented out of order and that steps without logical context may be performed in reverse order or concurrently. Moreover, one or more other operations may be added to the flow chart or one or more operations may be destroyed from the flow chart as directed by those skilled in the art in light of the present disclosure.

In addition, the described embodiments are only some, but not all, of the embodiments of the present application. The components of the embodiments of the present application, which are generally described and illustrated in the figures herein, may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, as provided in the accompanying drawings, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, which can be made by those skilled in the art, in light of the embodiments of the present application without undue burden, are within the scope of the present application.

Fig. 1 shows a flow chart of a file integration test method based on a large data platform according to an embodiment of the present application, and it should be understood that, in other embodiments, the order of part of the steps in the file integration test method based on a large data platform according to the present embodiment may be shared with each other according to actual needs, or part of the steps may be omitted or maintained. The file integration test method based on the big data platform comprises the following steps of:

step110, obtain the integrated stability test data of the target file of the integrated file system for the big data platform.

For some exemplary design considerations, the target file integration stability test data is test data obtained from a target test log.

The target file integration stability test data comprises labeling test nodes.

The labeling test node characterizes the test node which needs to be subjected to abnormal load estimation after the characteristic analysis of the file integration stability test data.

For some exemplary design considerations, the target file integration stability test data includes N annotated test nodes.

For some exemplary design ideas, when acquiring target file integration stability test data of an integrated file system for a big data platform, taking a labeling test node as a target positioning position, and acquiring target file integration stability test data comprising the labeling test node based on an abnormal load path of the labeling test node.

For some exemplary design ideas, after target file integration stability test data of an integrated file system for a big data platform is obtained, key test nodes in the target file integration stability test data are used as labeling test nodes.

step120, predicting the abnormal load state of the test feedback vector in the integrated stability test data of the target file, and generating the abnormal load state information in the integrated stability test data of the target file.

For some exemplary design considerations, abnormal load state predictions are used to obtain a sequence of test events for which there is a significant abnormality in load states in the target file integration stability test data.

step130, obtaining the target test running situation information corresponding to the stability test case according to the abnormal load characteristic value of the test execution operation in the stability test case corresponding to the abnormal load state information and the loss characteristic value between the set abnormal load characteristic values corresponding to the target file integrated stability test data.

The abnormal load characteristic value is set as a pre-configured threshold characteristic value. For example, setting the abnormal load feature value is determined based on a plurality of test execution operations in the target file integrated stability test data; alternatively, the abnormal load characteristic value is set to a preset fixed value or the like.

For some exemplary design ideas, after test execution operations corresponding to a plurality of resource scheduling components in target file integration stability test data of an integrated file system of a big data platform are acquired, N rest resource scheduling components associated with the resource scheduling components are determined by taking each resource scheduling component as a reference, and component abnormal load characteristic values between the test execution operations corresponding to the N resource scheduling components and the test execution operations corresponding to the resource scheduling components are analyzed, so that component abnormal load characteristic values corresponding to the plurality of resource scheduling components are obtained. For example, according to the component abnormal load characteristic values respectively corresponding to the plurality of resource scheduling components, the set abnormal load characteristic value corresponding to the target file integration stability test data is determined.

For some exemplary design ideas, taking an abnormal load average characteristic value of component abnormal load characteristic values corresponding to a plurality of resource scheduling components respectively as a set abnormal load characteristic value corresponding to the target file integration stability test data; or determining the median of the component abnormal load characteristic values from the component abnormal load characteristic values respectively corresponding to the resource scheduling components, and taking the median as the set abnormal load characteristic value corresponding to the target file integrated stability test data; or, taking the specified coefficient product of the abnormal load average characteristic value of the component abnormal load characteristic values corresponding to the resource scheduling components as the set abnormal load characteristic value corresponding to the target file integration stability test data.

For example, the stability test case corresponding to the abnormal load state information includes a test event corresponding to the abnormal load state information and test events corresponding to other N resource scheduling components. For some exemplary design ideas, the test events corresponding to the abnormal load state information and the test events corresponding to the associated N resource scheduling components form a stability test case. And when the stability test case corresponding to the abnormal load state information is obtained, the stability test case corresponding to the abnormal load state information is formed through the abnormal load state information.

Or splitting the target file integrated stability test data to generate a plurality of target member file integrated stability test data obtained after splitting, wherein the plurality of target member file integrated stability test data are respectively used as a stability test case, the stability test case comprises at least two test events, when the stability test case corresponding to the abnormal load state information is obtained, the target member file integrated stability test data corresponding to the abnormal load state information is determined through the abnormal load state information, and the target member file integrated stability test data are used as the stability test case corresponding to the abnormal load state information.

For some exemplary design ideas, after obtaining the abnormal load state information, determining N resource scheduling components in the stability test case corresponding to the abnormal load state information, obtaining test execution operations corresponding to the abnormal load state information, and the test execution operations corresponding to the N resource scheduling components, and determining abnormal load characteristic values of the test execution operations between the test execution operations corresponding to the abnormal load state information and the test execution operations corresponding to the N resource scheduling components based on the test execution operations corresponding to the abnormal load state information and the test execution operations corresponding to the N resource scheduling components.

For example, based on comparison data between the abnormal load characteristic value of the test execution operation and the set abnormal load characteristic value in the stability test case corresponding to the abnormal load state information, the situation coefficient of the initial test operation situation information is adjusted, so that the target test operation situation information corresponding to the stability test case is obtained.

For some exemplary design ideas, the situation coefficient of the initial test running situation information is updated differently through comparison data between the abnormal load characteristic value and the set abnormal load characteristic value of the test execution operation in the stability test instance corresponding to different abnormal load state information, so as to obtain target test running situation information corresponding to different stability test instances respectively. Namely: and corresponding relation information exists between the stability test instance and the target test running situation information.

For example: after obtaining comparison data 1 between the abnormal load characteristic value of the test execution operation in the stability test example 1 corresponding to the abnormal load state information 1 and the set abnormal load characteristic value and comparison data 2 between the abnormal load characteristic value of the test execution operation in the stability test example 2 corresponding to the abnormal load state information 2 and the set abnormal load characteristic value; carrying out first updating on situation coefficients of the initial test running situation information based on the comparison data 1 to generate target test running situation information 1 corresponding to the stability test example 1; and carrying out second updating on situation coefficients of the initial test running situation information based on the comparison data 2, and generating target test running situation information 2 and the like corresponding to the stability test example 2.

For example, the stability test case representation is used for obtaining the integrated stability test data of the target member file after the integrated stability test data of the target file is segmented, and different abnormal load state information may correspond to different stability test cases or may correspond to the same stability test case; abnormal load state information may or may not exist in the different target member file integration stability test data.

For some exemplary design considerations, when multiple abnormal load state information are included in the same stability test case, when target test running situation information corresponding to the stability test case is determined according to the abnormal load state information, an abnormal load feature value corresponding to the stability test case is determined based on component abnormal load feature values between the multiple abnormal load state information and multiple resource scheduling components in the stability test case.

step140, according to the corresponding relation information between the stability test instance and the target test running situation information, vector description is carried out on the stability test instance by the target test running situation information, and a description vector set corresponding to the integrated stability test data of the target file is generated.

For some exemplary design ideas, after the target test operation situation information corresponding to different stability test cases is obtained, vector description is carried out on the corresponding stability test cases through the different target test operation situation information, so that vector description results corresponding to the different stability test cases are obtained.

For example: after the target test running situation information 1 corresponding to the stability test example 1 and the target test running situation information 2 corresponding to the stability test example 2 are obtained, vector description is carried out on the stability test example 1 by the target test running situation information 1, and a vector description result 1 corresponding to the stability test example 1 is generated; and carrying out vector description on the stability test case 2 by using the target test running situation information 2, and generating a vector description result 2 corresponding to the stability test case 2.

For example, according to the vector description results respectively corresponding to the different stability test cases, a description vector set corresponding to the integrated stability test data of the target file is generated. For some exemplary design ideas, the description vector sets are fused with the vector description results respectively corresponding to different stability test cases, so that the stability test data corresponding to the integration of the target file is obtained.

step150, according to the description vector set, analyzing the abnormal load path of the labeling test node from the target file integration stability test data, and generating the abnormal load path of the labeling test node in the target file integration stability test data.

Based on the above steps, the method analyzes the stability test data of different file integration, avoids carrying out abnormal load estimation only depending on the relation between the stability test data of multiple file integration, weakens the stability test cases which cannot analyze the abnormal load path of the labeling test node from the stability test data of the target file integration for the stability test cases corresponding to different abnormal load state information, strengthens the stability test cases which can analyze the abnormal load path of the labeling test node from the stability test data of the target file integration, and generates a description vector set which is more convenient for extracting the abnormal load path of the labeling test node from the stability test data of the target file integration. Therefore, when the labeling test node analyzes the abnormal load path from the target file integrated stability test data through the description vector set, the loss characteristic value between the labeling test node and the test scheduling node is enhanced, and the labeling test node can extract the specific abnormal load path from the target file integrated stability test data more accurately.

For some exemplary design ideas, segmenting the integrated stability test data of the target file to generate at least two stability test cases, and after determining the stability test cases corresponding to the abnormal load state information, acquiring the target test operation state information corresponding to the stability test cases by a method for adjusting the state coefficients of the initial test operation state information. For some exemplary design considerations, step130 may also be implemented as steps 210 through 270 as follows.

step210, obtain initial test running situation information.

The initial test running situation information is predetermined test running situation information.

step220, obtaining test execution operations corresponding to a plurality of resource scheduling components in the target file integration stability test data of the integrated file system of the big data platform.

For example, each test event in the integrated stability test data of the target file is taken as a basis, a plurality of resource scheduling components in the integrated stability test data of the target file are determined, and test execution operations respectively corresponding to the plurality of resource scheduling components are determined, so that the test execution operations respectively corresponding to the plurality of resource scheduling components in the integrated stability test data of the target file are obtained.

step230, obtaining component abnormal load characteristic values corresponding to the resource scheduling components according to the test execution operations corresponding to the resource scheduling components and the resource scheduling components respectively.

For some exemplary design ideas, the component abnormal load feature value characterizes a load state of test execution operations between the test execution operations corresponding to the resource scheduling component and the test execution operations corresponding to the N resource scheduling components in the target scheduling interval. That is, for any one of the plurality of resource scheduling components, after the test execution operation of the resource scheduling component and the test execution operation corresponding to N remaining resource scheduling components in the target scheduling interval of the resource scheduling component are obtained, the load state of the test execution operation between the test execution operation of the resource scheduling component and the test execution operation corresponding to N remaining resource scheduling components is analyzed, so that the component abnormal load characteristic value corresponding to the resource scheduling component can be known.

For some exemplary design considerations, upon acquisition of component abnormal load feature values for an attention resource scheduling component of a plurality of resource scheduling components, an attention resource scheduling component and N resource scheduling components within a target scheduling interval of the attention resource scheduling component are determined.

For some exemplary design ideas, after obtaining a plurality of resource scheduling components in the integrated stability test data of the target file, one resource scheduling component is arbitrarily selected as an attention resource scheduling component, for example: the resource scheduling component Z1 is arbitrarily selected as an attention resource scheduling component, and N resource scheduling components of the attention resource scheduling component in a target scheduling interval are determined, that is: n remaining resource scheduling components other than the attention resource scheduling component are determined within the target scheduling interval. For example: the 8 resource scheduling components associated with the resource scheduling component Z1 are taken as N resource scheduling components selected in the target scheduling interval.

For example, after determining the attention resource scheduling component and N resource scheduling components within the target scheduling interval of the attention resource scheduling component, a test execution operation corresponding to the attention resource scheduling component and a test execution operation corresponding to the N resource scheduling components respectively are determined.

For some exemplary design ideas, determining component abnormal load characteristic values corresponding to the attention resource scheduling components according to the test execution operations corresponding to the N resource scheduling components and the access resource histogram data of the test execution operations corresponding to the attention resource scheduling components.

step240, calculating component abnormal load characteristic values corresponding to the resource scheduling components respectively, and determining an abnormal load average characteristic value of the integrated stability test data of the target file.

For some exemplary design ideas, after obtaining component abnormal load feature values corresponding to the plurality of resource scheduling components respectively through the step230, component abnormal load feature values corresponding to the plurality of resource scheduling components in the target file integration stability test data are calculated, so as to obtain an abnormal load average feature value corresponding to the target file integration stability test data.

step250, determining a set abnormal load characteristic value corresponding to the integrated stability test data of the target file according to the abnormal load average characteristic value.

For some exemplary design ideas, after determining the abnormal load average characteristic value corresponding to the integrated stability test data of the target file, determining the set abnormal load characteristic value corresponding to the integrated stability test data of the target file according to the abnormal load average characteristic value.

step260, slicing the integrated stability test data of the target file to generate a plurality of stability test cases in the integrated stability test data of the target file.

step270 adjusts the situation coefficient of the initial test running situation information according to the abnormal load characteristic value of the test execution operation in the stability test instance corresponding to the abnormal load state information and the loss characteristic value between the set abnormal load characteristic values corresponding to the integrated stability test data of the target file, and generates the target test running situation information corresponding to the stability test instance.

For some exemplary design ideas, after obtaining a plurality of abnormal load state information, determining stability test cases corresponding to the plurality of abnormal load state information respectively, and determining abnormal load characteristic values of test execution operations in the stability test cases.

For example: after obtaining a plurality of abnormal load state information in the integrated stability test data of the target file, determining a stability test instance corresponding to each abnormal load state information, namely: and determining which area of the segmented target file integrated stability test data each piece of abnormal load state information is located. For example, after obtaining the plurality of abnormal load state information, stability test cases corresponding to the plurality of abnormal load state information are additionally determined, and based on the resource scheduling component and the abnormal load state information in the stability test cases, an abnormal load characteristic value of the test execution operation of the abnormal load state information is determined.

For example, the stability test cases for which different abnormal load state information corresponds respectively may be the same, for example: the stability test cases corresponding to the abnormal load state information 1 and the abnormal load state information 2 are all realized as a stability test case 1; alternatively, the stability test cases respectively corresponding to different abnormal load state information may be different, for example: abnormal load state information 1 corresponds to stability test case 1, abnormal load state information 2 corresponds to stability test case 2, etc.

When determining the abnormal load characteristic value of the test execution operation in the stability test case corresponding to each abnormal load state information, determining the test execution operation corresponding to the abnormal load state information through the test execution operation corresponding to the resource scheduling component in the stability test case, for example: determining a gradient value between a test execution operation corresponding to the abnormal load state information and a test execution operation corresponding to a resource scheduling component in the stability test case, and taking the gradient value as an abnormal load characteristic value of the test execution operation corresponding to the stability test case of the abnormal load state information; or determining the cost of the test execution operation between the test execution operation corresponding to the abnormal load state information and the test execution operation corresponding to the resource scheduling component in the stability test case, and taking the cost of the test execution operation as the abnormal load characteristic value of the test execution operation corresponding to the stability test case of the abnormal load state information.

For some exemplary design ideas, the implementation of the set abnormal load characteristic value corresponding to the integrated stability test data of the target file is taken as the test execution operation cost as an example. When the abnormal load characteristic value of the test execution operation in the stability test case corresponding to the abnormal load state information is compared with the set abnormal load characteristic value corresponding to the target file integrated stability test data, the abnormal load characteristic value of the test execution operation in the stability test case corresponding to the abnormal load state information is also realized to be the test execution operation cost, the test execution operation cost obtained by the solution corresponding to the stability test case is compared with the set abnormal load characteristic value realized to be the test execution operation cost, and therefore comparison data between the abnormal load characteristic value of the test execution operation in the stability test case corresponding to the abnormal load state information and the set abnormal load characteristic value are determined.

For example, a plurality of test events within a stability test case corresponding to abnormal load state information is determined. The plurality of test events comprise test events corresponding to abnormal load state information, and the plurality of test events respectively correspond to test execution operations.

For some exemplary design ideas, determining an abnormal load characteristic value corresponding to a stability test case according to access resource histogram data among test execution operations corresponding to a plurality of test events; if the abnormal load characteristic value corresponding to the stability test case is smaller than the set abnormal load characteristic value, the situation coefficient of the test operation situation information is reduced by the first appointed updating coefficient.

For example, based on the load states among the test execution operations corresponding to the plurality of test events in the stability test case corresponding to the abnormal load state information, the abnormal load characteristic value corresponding to the stability test case is determined. For example: and determining an abnormal load characteristic value corresponding to the stability test instance based on the access resource histogram data among the test execution operations corresponding to the test events.

For some exemplary design ideas, determining an abnormal load characteristic value corresponding to a stability test case according to access resource histogram data among test execution operations corresponding to a plurality of test events; if the abnormal load characteristic value corresponding to the stability test case is larger than the set abnormal load characteristic value, the situation coefficient of the test operation situation information is increased by the second specified updating coefficient.

For some exemplary design ideas, an abnormal load average characteristic value, which is set to 1.2 times the abnormal load characteristic value, is described as an example. After the abnormal load state information 1 and the abnormal load state information 2 are obtained, the abnormal load characteristic value of the test execution operation in the stability test case 1 corresponding to the abnormal load state information 1 and the abnormal load characteristic value of the test execution operation in the stability test case 2 corresponding to the abnormal load state information 2 are respectively compared with the set abnormal load characteristic value.

For example: when the abnormal load characteristic value of the test execution operation in the stability test case 1 corresponding to the abnormal load state information 1 is smaller than the set abnormal load characteristic value corresponding to the integrated stability test data of the target file, reducing the situation coefficient of the initial test operation situation information to realize the first update of the initial test operation situation information, thereby obtaining the target test operation situation information 1 corresponding to the stability test case 1, wherein the situation coefficient value in the target test operation situation information is smaller than 1; when the abnormal load characteristic value of the test execution operation in the stability test case 2 corresponding to the abnormal load state information 2 is larger than the set abnormal load characteristic value corresponding to the integrated stability test data of the target file, the situation coefficient of the initial test operation situation information is increased to realize the second updating of the initial test operation situation information, so that the target test operation situation information 2 corresponding to the stability test case 2 is obtained, and the situation coefficient in the target test operation situation information is larger than 1.

Therefore, based on different abnormal load state information, situation coefficients of the initial test operation situation information are updated differently, so that target test operation situation information corresponding to the stability test cases one by one is obtained, vector description is carried out on the stability test cases by adopting the target test operation situation information, loss characteristic values between the labeling test nodes and the test scheduling nodes are enhanced, and the labeling test nodes can extract specific abnormal load paths from the stability test data of the target file integration more accurately.

For some exemplary design ideas, vector description is carried out on the stability test case by the target test running situation information based on the corresponding relation information of the one-to-one correspondence between the stability test case and the target test running situation information, and a description vector set corresponding to the integrated stability test data of the target file is obtained according to the results of the vector description. For some exemplary design considerations, the above embodiments may be implemented by steps 310 through 380 as follows.

step310 obtains target file integration stability test data for an integrated file system of a large data platform.

The target file integration stability test data comprises labeling test nodes.

step320, performing abnormal load state prediction on the test feedback vector in the target file integration stability test data, and generating abnormal load state information in the target file integration stability test data.

step330 obtains the target test running situation information corresponding to the stability test case according to the abnormal load characteristic value of the test execution operation in the stability test case corresponding to the abnormal load state information and the loss characteristic value between the set abnormal load characteristic value.

And the stability test case and the target test running situation information are in corresponding relation information.

For example, after obtaining the plurality of abnormal load state information, determining test execution operations corresponding to the plurality of abnormal load state information respectively and test execution operations corresponding to the resource scheduling components in the stability test case of the abnormal load state information, thereby determining abnormal load characteristic values of the test execution operations of the abnormal load state information in the stability test case. And comparing the abnormal load characteristic value of the test execution operation corresponding to each abnormal load state information with the set abnormal load characteristic value, and updating the initial test running situation information based on the comparison data so as to obtain target test running situation information corresponding to each stability test instance.

For some exemplary design considerations, the abnormal load feature value is set to a value that analyzes the integrated stability test data for the target file. If the abnormal load characteristic value of the test execution operation corresponding to the abnormal load state information in the stability test instance is smaller than the set abnormal load characteristic value corresponding to the integrated stability test data of the target file, the situation coefficient of the initial test operation situation information is reduced; if the abnormal load characteristic value of the test execution operation corresponding to the abnormal load state information in the stability test instance is larger than the set abnormal load characteristic value corresponding to the integrated stability test data of the target file, the situation coefficient of the test operation situation information is increased, and the like.

step340, obtaining at least two abnormal load state information corresponding to the labeling test node, and stability test cases corresponding to the at least two abnormal load state information respectively.

The at least two abnormal load state information comprises first abnormal load state information and second abnormal load state information, the stability test instance corresponding to the first abnormal load state information is a first stability test instance, the stability test instance corresponding to the second abnormal load state information is a second stability test instance, and the first stability test instance and the second stability test instance are different.

step350, vector description is carried out on the first stability test case according to the first target test running situation information corresponding to the first stability test case, and a first description vector set corresponding to the first stability test case is generated.

For some exemplary design ideas, the first target test running situation information is target test running situation information obtained according to comparison data between an abnormal load characteristic value of a test execution operation corresponding to the first abnormal load state information in the stability test case and a set abnormal load characteristic value, and the first target test running situation information corresponds to the first stability test case and can be used for updating the first stability test case in a targeted manner. For example: and substituting the resource scheduling component in the first stability test instance into the first target test running situation information to generate a first description vector set corresponding to the first stability test instance, wherein the resource scheduling component in the first stability test instance comprises first abnormal load state information.

step360, vector description is carried out on the second stability test case according to second target test running situation information corresponding to the second stability test case, and a second description vector set corresponding to the second stability test case is generated.

For some exemplary design ideas, the second target test running situation information is target test running situation information obtained according to comparison data between an abnormal load characteristic value of a test execution operation corresponding to the second abnormal load state information in the stability test case and a set abnormal load characteristic value, and the second target test running situation information corresponds to the second stability test case and can be used for updating the second stability test case in a targeted manner. For example: and substituting the second stability test instance into the second target test running situation information to generate a second description vector set corresponding to the second stability test instance.

step370 generates a description vector set corresponding to the integrated stability test data of the target file according to the first description vector set and the second description vector set.

For some exemplary design ideas, after the first description vector set and the second description vector set are obtained, the description vector set is fused with the first description vector set and the second description vector set, and then the description vector set corresponding to the integrated stability test data of the target file is generated.

For some exemplary design ideas, a priori stability test case is obtained, and the priori stability test case characterizes a stability test case without abnormal load state information in the integrated stability test data of the target file.

For example, vector description is performed on the prior stability test case by using initial test running situation information, and a resource scheduling component description vector set corresponding to the prior stability test case is generated.

For example, when the situation coefficient corresponding to the initial test operation situation information is adjusted according to the integrated stability test data of the target member files, vector description is performed on the integrated stability test data of the N target member files without abnormal load state information through the initial test operation situation information.

For some exemplary design considerations, a resource scheduling component description vector set is fused with a first description vector set and a second description vector set to generate a description vector set corresponding to the target file integration stability test data.

For some exemplary design ideas, after the description vector sets of the resource scheduling components corresponding to the N resource scheduling components are obtained, the description vector sets of the N resource scheduling component description vector sets are fused with the first description vector set and the second description vector set, so that the description vector sets corresponding to the integrated stability test data of the target file are obtained.

step380, according to the description vector set, analyzing the abnormal load path of the labeling test node from the target file integration stability test data, and generating the abnormal load path of the labeling test node in the target file integration stability test data.

In the description vector set, according to the abnormal load characteristic value of the test execution operation in the stability test instance of the abnormal load state information, the abnormal load state information is subjected to differential transformation, so that the description vector set can better embody the loss characteristic value of the labeling test node and the test scheduling node, and when the description vector set is analyzed, the analysis of an abnormal load path from the target file integrated stability test data is more convenient to obtain the labeling test node.

For some exemplary design considerations, the set of description vectors is input into a target training neural network that satisfies model convergence conditions.

The target training neural network is used for identifying the labeling test nodes.

For example, the target training neural network analyzes the description vector set based on the pixel loss characteristic value of the description vector set, so that an abnormal load path of the labeling test node in the target file integrated stability test data is extracted, and the abnormal load path of the labeling test node in the target file integrated stability test data is output from the target training neural network.

For example, the target training neural network clusters the description vector set to determine a clustering result. The clustering is used for clustering the test fragments with the same abnormal description category in the target file integrated stability test data.

For example, according to the clustering result, the abnormal load path extraction is performed on the labeling test nodes in the target file integration stability test data, and the abnormal load paths of the labeling test nodes in the target file integration stability test data are output from the target training neural network.

For some exemplary design ideas, based on the test execution operation states in the description vector set, the abnormal load path extraction is performed on the labeling test nodes and the test scheduling nodes in the target file integrated stability test data. The test scheduling nodes represent scheduling nodes except the labeling test nodes in the target file integrated stability test data.

For example, according to the extraction result of the abnormal load path of the labeling test node, determining the abnormal load path of the labeling test node in the target file integrated stability test data.

After vector description is carried out on the corresponding stability test cases by using different target test running situation information, the description vector sets corresponding to the stability test cases are fused, so that the description vector sets reflecting the overall characteristics of the integrated stability test data of the target file are obtained, and when the labeling test nodes analyze abnormal load paths from the integrated stability test data of the target file through the description vector sets, the loss characteristic values between the labeling test nodes and the test scheduling nodes are enhanced, so that the labeling test nodes can extract specific abnormal load paths from the integrated stability test data of the target file more accurately.

Fig. 2 schematically illustrates a cloud computing system 100 that may be used to implement various embodiments described herein.

For one embodiment, fig. 2 illustrates a cloud computing system 100, the cloud computing system 100 having one or more processors 102, a control module (chipset) 104 coupled to one or more of the processor(s) 102, a memory 106 coupled to the control module 104, a non-volatile memory (NVM)/storage 108 coupled to the control module 104, one or more input/output devices 110 coupled to the control module 104, and a network interface 112 coupled to the control module 106.

The processor 102 may include one or more single-core or multi-core processors, and the processor 102 may include any combination of general-purpose or special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In some exemplary design concepts, the cloud computing system 100 can be used as a cloud computing system device such as a gateway in the embodiments of the present application.

In some example design considerations, the cloud computing system 100 may include one or more computer-readable media (e.g., memory 106 or NVM/storage 108) having instructions 114 and one or more processors 102, in fusion with the one or more computer-readable media, configured to execute the instructions 114 to implement the modules to perform the actions described in this disclosure.

For one embodiment, the control module 104 may include any suitable interface controller to provide any suitable interface to one or more of the processor(s) 102 and/or any suitable device or component in communication with the control module 104.

The control module 104 may include a memory controller module to provide an interface to the memory 106. The memory controller modules may be hardware modules, software modules, and/or firmware modules.

Memory 106 may be used, for example, to load and store data and/or instructions 114 for cloud computing system 100. For one embodiment, memory 106 may comprise any suitable volatile memory, such as, for example, a suitable DRAM. In some exemplary design considerations, memory 106 may include a double data rate type four synchronous dynamic random access memory (DDR 4 SDRAM).

For one embodiment, control module 104 may include one or more input/output controllers to provide interfaces to NVM/storage 108 and input/output device(s) 110.

For example, NVM/storage 108 may be used to store data and/or instructions 114. NVM/storage 108 may include any suitable nonvolatile memory (e.g., flash memory) and/or may include any suitable nonvolatile storage(s) (e.g., one or more Hard Disk Drives (HDDs), one or more Compact Disc (CD) drives, and/or one or more Digital Versatile Disc (DVD) drives).

NVM/storage 108 may include storage resources that are physically part of the device on which cloud computing system 100 is installed, or may be accessible by the device without necessarily being part of the device. For example, NVM/storage 108 may be accessed via input/output device(s) 110 according to a network.

Input/output device(s) 110 may provide an interface for cloud computing system 100 to communicate with any other suitable device, input/output device 110 may include a communication component, pinyin component, sensor component, and the like. The network interface 112 may provide an interface for the cloud computing system 100 to communicate in accordance with one or more networks, and the cloud computing system 100 may communicate wirelessly with one or more components of a wireless network in accordance with any of one or more wireless network standards and/or protocols, such as accessing a wireless network in accordance with a communication standard, such as WwFw, 2G, 3G, 4G, 5G, etc., or a combination thereof.

For one embodiment, one or more of the processor(s) 102 may be loaded with logic of one or more controllers (e.g., memory controller modules) of the control module 104. For one embodiment, one or more of the processor(s) 102 may be loaded together with logic of one or more controllers of the control module 104 to form a system level load (SwP). For one embodiment, one or more of the processor(s) 102 may be integrated on the same mold as logic of one or more controllers of the control module 104. For one embodiment, one or more of the processor(s) 102 may be integrated on the same die with logic of one or more controllers of the control module 104 to form a system on chip (SoC).

In various embodiments, the cloud computing system 100 may be, but is not limited to being: cloud computing systems, desktop computing devices, or mobile computing devices (e.g., laptop computing devices, handheld computing devices, tablet computers, netbooks, etc.). In various embodiments, the cloud computing system 100 may have more or fewer components and/or different architectures. For example, in some exemplary design considerations, cloud computing system 100 includes one or more cameras, keyboards, liquid Crystal Display (LCD) screens (including touch screen displays), non-volatile memory ports, multiple antennas, graphics chips, application Specific Integrated Circuits (ASICs), and speakers.

The foregoing has outlined rather broadly the more detailed description of embodiments of the present application, wherein specific examples are provided herein to illustrate the principles and embodiments of the present application, the above examples being provided solely to assist in the understanding of the methods of the present application and the core ideas thereof; meanwhile, as those skilled in the art will have modifications in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.

Claims

1. The file integration test method based on the big data platform is characterized by being applied to the cloud computing system, and comprises the following steps:

2. The file integration test method based on a big data platform according to claim 1, wherein the obtaining the target test running situation information corresponding to the stability test case according to the abnormal load characteristic value of the test execution operation in the stability test case corresponding to the abnormal load state information and the loss characteristic value between the set abnormal load characteristic value includes:

3. The file integration test method based on the big data platform according to claim 2, wherein the adjusting the situation coefficient of the initial test running situation information according to the abnormal load characteristic value of the test execution operation in the stability test instance corresponding to the abnormal load state information and the loss characteristic value between the set abnormal load characteristic values includes:

4. A big data platform based file integration test method according to any of claims 1-3, wherein the method further comprises:

5. The method for testing file integration based on a big data platform according to claim 4, wherein the obtaining the component abnormal load characteristic values respectively corresponding to the plurality of resource scheduling components according to the test execution operations respectively corresponding to the plurality of resource scheduling components comprises:

6. The big data platform based file integration test method according to any one of claims 1-3, wherein the generating a description vector set corresponding to the target file integration stability test data according to the correspondence information between the stability test instance and the target test operation situation information and performing vector description on the stability test instance by using the target test operation situation information includes:

vector description is carried out on the second stability test instance according to second target test running situation information corresponding to the second stability test instance, and a second description vector set corresponding to the second stability test instance is generated;

and generating the description vector set corresponding to the target file integrated stability test data according to the first description vector set and the second description vector set.

7. The large data platform based file integration test method of claim 6, wherein the generating the description vector set corresponding to the target file integration stability test data from the first description vector set and the second description vector set comprises:

8. The big data platform based file integration test method according to any one of claims 1-3, wherein the performing, according to the description vector set, the abnormal load path analysis on the labeling test node from the target file integration stability test data, and generating the abnormal load path of the labeling test node in the target file integration stability test data, includes:

9. A big data platform based file integration test method according to any of claims 1-3, wherein the method further comprises:

10. The file integration test system based on the big data platform is characterized by comprising a cloud computing system and an integrated file system in communication connection with the cloud computing system, wherein the cloud computing system is specifically used for: