CN115982646B - Management method and system for multisource test data based on cloud platform - Google Patents
Management method and system for multisource test data based on cloud platform Download PDFInfo
- Publication number
- CN115982646B CN115982646B CN202310269927.4A CN202310269927A CN115982646B CN 115982646 B CN115982646 B CN 115982646B CN 202310269927 A CN202310269927 A CN 202310269927A CN 115982646 B CN115982646 B CN 115982646B
- Authority
- CN
- China
- Prior art keywords
- test
- data
- index
- test data
- nodes
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a management method and a system of multisource test data based on a cloud platform, wherein the management method comprises the following steps: collecting test data of a tested piece, matching the test label with the test data, sending the test label to a cloud platform for storage and data cleaning, and obtaining index interpretation data by using the test label to match the evaluation standard of the test index in the test data set; the index interpretation data and the multi-source test data sequence are imported into a low-dimensional vector space, and representation learning is carried out through an undirected heterogeneous graph; and constructing a test data comprehensive analysis model, carrying out data fusion on the multi-source test data to obtain a comprehensive analysis result, and generating an analysis report of the to-be-tested piece according to a preset test item of the to-be-tested piece. According to the invention, the multisource test data of the to-be-tested piece in different test systems are reorganized and fused in a multidimensional manner, and the comprehensive intelligent analysis of the multisource test data and the comprehensive macroscopic data is directly carried out in real time, so that the high performance requirement of the intelligent analysis of the data is met while the test data is acquired.
Description
Technical Field
The invention relates to the technical field of data management, in particular to a method and a system for managing multisource test data based on a cloud platform.
Background
With the rapid development of computer technology, computer automatic test systems have been widely used in various industries, and as a core function data management in computer automatic test systems, how to manage test data more simply and effectively is a problem and trouble to be solved by each system test developer, how to uniformly manage multi-source test projects is also more and more paid attention to users and breakthrough in development of various technologies. For the enterprise level, how to effectively manage the original data, index data and the like of each test stage becomes a key for the enterprise to shorten the research and development period of new products and ensure the dominant position of the new products in the market.
At present, a large number of automatic test systems generate massive test data, which cover the original data, interpretation result data, test environment data and index interpretation data of each test stage of a tested product, the data are stored in each test system in a scattered manner, manual management is adopted for most of the test data at present, so that the data are in an island phenomenon, and cannot be comprehensively analyzed, and in order to optimize the management of the test data by related units, the test data can be well assisted in decision, so that a management method of the test data is needed to uniformly manage the data in the test system and perform later data processing, and various intelligent analyses of the basic data and comprehensive macroscopic data are performed on line in real time. In the management of test data, how to perform comprehensive intelligent analysis according to multi-source test data is one of the problems to be solved at present.
Disclosure of Invention
In order to solve the technical problems, the invention provides a management method and system of multi-source test data based on a cloud platform.
The first aspect of the invention provides a management method of multi-source test data based on a cloud platform, which comprises the following steps:
collecting test data of a tested piece, matching the test label, and sending the test data to a cloud platform for storage, wherein the cloud platform carries out data cleaning on the multi-source test data to remove obvious abnormal test data in each multi-source test data sequence;
classifying the multi-source test data after data cleaning according to the test tags, obtaining test data sets under each test tag, and obtaining index interpretation data by matching test standards of test indexes in the test data sets with the test tags;
the index interpretation data and the multi-source test data sequence are imported into a low-dimensional vector space, and multi-source test data and index interpretation data corresponding to the test indexes under each label are represented and learned through undirected heterograms in the low-dimensional vector space;
and constructing a comprehensive analysis model of the test data, carrying out data fusion on the multi-source test data, acquiring a comprehensive analysis result, matching multiple report forms according to preset test items of the to-be-tested piece, and generating an analysis report of the to-be-tested piece based on the test report.
In the scheme, multi-source test data after data cleaning are classified according to test tags, test data sets under the test tags are obtained, the test tags are used for matching with the judgment standards of test indexes in the test data sets, and index interpretation data are obtained, specifically:
acquiring a multi-source test data sequence after data cleaning, clustering multi-source test data under the same test label, forming test data sets under each test label according to acquisition time stamps, and obtaining test indexes according to the test data sets;
establishing a search task by using test labels of each test data set, and obtaining test indexes and corresponding judgment standards with similarity meeting preset standards by using similarity calculation in a cloud platform data search space;
performing data matching on the retrieved test indexes and the test indexes in each test data set, and after the matching of all the test indexes in each test data set is finished, distributing corresponding judgment standards to the test indexes as the judgment standards of the current test indexes in each test data set;
and judging the multi-source test data of the tested piece according to the judging standard of the current test index, and obtaining index judging data.
In this scheme, import the said index interpretation data and multi-source test data sequence into the low-dimensional vector space, in the low-dimensional vector space will test the multi-source test data and index interpretation data that index correspond under each label through the undirected different composition to show and learn, specifically:
acquiring common test contents and test items through cloud platform data statistics, performing data retrieval according to the common test contents and the test items to acquire corresponding analysis reports, and reading historical test indexes in the analysis reports and comprehensive analysis results of the historical test indexes;
importing current multi-source test data and index interpretation data of the to-be-tested piece into a low-dimensional vector space, and judging whether a historical interaction relation exists in the test index under the current test item according to a comprehensive analysis result of the historical test index;
constructing undirected heterograph of test data by utilizing multi-source test data and index interpretation data of a piece to be tested in a low-dimensional vector space, taking the test data and index interpretation data of each test index as nodes of the undirected heterograph, marking the nodes as test index nodes, and taking historical interaction relations among the test indexes as edge structures among the nodes.
In the scheme, a comprehensive analysis model of test data is constructed, multi-source test data are subjected to data fusion, and comprehensive analysis results are obtained, specifically:
Constructing a test data comprehensive analysis model based on deep learning, performing representation learning on undirected heterograms of the test data through a graph convolutional neural network, acquiring preset test items of a piece to be tested, and calculating information contribution rates of all test indexes according to the preset test items;
selecting a test index with the maximum information contribution rate, marking corresponding test index nodes, obtaining nodes with connection relation with the marked nodes in the undirected heterogram as associated nodes, and generating a characteristic test index node set by the marked nodes and the associated nodes;
acquiring the mahalanobis distance between each characteristic test index node in the characteristic test index node set and other test index nodes, presetting a mahalanobis distance threshold range, judging whether the mahalanobis distance is in the preset mahalanobis distance threshold range, and if so, taking the test index node as an adjacent node of the characteristic test index node closest to the test index node;
heterogeneous fusion is carried out on the adjacent nodes and the characteristic test index nodes through a message transmission mechanism and a neighbor aggregation mechanism of the graph convolutional neural network, and embedded representation of the characteristic test index nodes is generated;
acquiring relevant test data and prediction data according to test items of a piece to be tested, generating a training data set, training a gated circulating neural network through the training data, and inputting embedded representation of characteristic test index nodes into the gated circulating neural network for prediction analysis;
And matching the predictive analysis with a preset report form, and outputting a predictive analysis result.
In the scheme, an attention mechanism is introduced into a neighbor aggregation mechanism of the graph convolution neural network, and the attention mechanism is specifically as follows:
the information contribution rate of other test indexes is obtained, and the information contribution rate is used as attention weight to carry out weight distribution on the characteristic test index nodes;
and carrying out feature aggregation according to the attention weight in combination with other test index nodes, updating the self-representation of the feature index nodes, and generating embedded vector representations with other test index data features.
In the scheme, the method further comprises the step of customizing the test data analysis template according to the test data comprehensive analysis model, wherein the method specifically comprises the following steps:
acquiring the position of the basic data corresponding to the fusion test data in the low-dimensional vector space according to the test data comprehensive analysis model, training the comprehensive analysis model according to the basic data, selecting a report pattern after the comprehensive analysis model accords with the verification standard, and matching with the test item label to generate a data analysis template;
carrying out structural processing on the data analysis templates of different test items, storing the data analysis templates into a cloud platform database, and calling the data analysis templates according to the test requirements of the to-be-tested piece;
Acquiring historical analysis reports of different tested pieces in the preset time of the same enterprise user, extracting test tag information, judgment index and judgment standard information and report information of the enterprise user in the historical analysis reports, and acquiring preference characteristics of the enterprise user according to the extracted information;
and customizing the enterprise user portrait through the preference characteristics, configuring a testing system of a current piece to be tested and a multi-source testing data management analysis environment in advance according to the enterprise user portrait, and updating the user portrait according to the change of the testing requirements of the enterprise user.
The second aspect of the present invention also provides a management system for multi-source test data based on a cloud platform, the system comprising: the system comprises a memory and a processor, wherein the memory comprises a management method program of multi-source test data based on a cloud platform, and the management method program of the multi-source test data based on the cloud platform realizes the following steps when being executed by the processor:
collecting test data of a tested piece, matching the test label, and sending the test data to a cloud platform for storage, wherein the cloud platform carries out data cleaning on the multi-source test data to remove obvious abnormal test data in each multi-source test data sequence;
Classifying the multi-source test data after data cleaning according to the test tags, obtaining test data sets under each test tag, and obtaining index interpretation data by matching test standards of test indexes in the test data sets with the test tags;
the index interpretation data and the multi-source test data sequence are imported into a low-dimensional vector space, and multi-source test data and index interpretation data corresponding to the test indexes under each label are represented and learned through undirected heterograms in the low-dimensional vector space;
and constructing a comprehensive analysis model of the test data, carrying out data fusion on the multi-source test data, acquiring a comprehensive analysis result, matching multiple report forms according to preset test items of the to-be-tested piece, and generating an analysis report of the to-be-tested piece based on the test report.
The invention discloses a management method and a system of multisource test data based on a cloud platform, wherein the management method comprises the following steps: collecting test data of a tested piece, matching the test label with the test data, sending the test label to a cloud platform for storage and data cleaning, and obtaining index interpretation data by using the test label to match the evaluation standard of the test index in the test data set; the index interpretation data and the multi-source test data sequence are imported into a low-dimensional vector space, and representation learning is carried out through an undirected heterogeneous graph; and constructing a test data comprehensive analysis model, carrying out data fusion on the multi-source test data to obtain a comprehensive analysis result, and generating an analysis report of the to-be-tested piece according to a preset test item of the to-be-tested piece. According to the invention, the multisource test data of the to-be-tested piece in different test systems are reorganized and fused in a multidimensional manner, and the comprehensive intelligent analysis of the multisource test data and the comprehensive macroscopic data is directly carried out in real time, so that the high performance requirement of the intelligent analysis of the data is met while the test data is acquired.
Drawings
FIG. 1 shows a flow chart of a method for managing multi-source test data based on a cloud platform of the present invention;
FIG. 2 shows a flow chart of a method for representing and learning multi-source test data and index interpretation data through undirected heterograms;
FIG. 3 is a flow chart showing a method for constructing a test data comprehensive analysis model to obtain a comprehensive analysis result;
fig. 4 shows a block diagram of a management system for multi-source test data based on a cloud platform according to the present invention.
Detailed Description
In order that the above-recited objects, features and advantages of the present invention will be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description. It should be noted that, in the case of no conflict, the embodiments of the present application and the features in the embodiments may be combined with each other.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those described herein, and therefore the scope of the present invention is not limited to the specific embodiments disclosed below.
Fig. 1 shows a flowchart of a method for managing multi-source test data based on a cloud platform according to the present invention.
As shown in fig. 1, a first aspect of the present invention provides a method for managing multi-source test data based on a cloud platform, including:
s102, collecting test data of a tested piece, matching test labels, and sending the test data to a cloud platform for storage, wherein the cloud platform carries out data cleaning on multi-source test data, and obvious abnormal test data in each multi-source test data sequence are removed;
s104, classifying the multi-source test data after data cleaning according to the test tags, obtaining test data sets under each test tag, and obtaining index interpretation data by utilizing the test tags to match the evaluation standards of the test indexes in the test data sets;
s106, importing the index interpretation data and the multi-source test data sequence into a low-dimensional vector space, and performing representation learning on the multi-source test data and the index interpretation data corresponding to the test indexes under each label through undirected heterograms in the low-dimensional vector space;
s108, constructing a comprehensive analysis model of the test data, carrying out data fusion on the multi-source test data, obtaining a comprehensive analysis result, matching multiple report forms according to preset test items of the to-be-tested piece, and generating an analysis report of the to-be-tested piece based on the test report.
It should be noted that, the automatic test system acquires the multi-source test data of the part to be tested, and invokes the communication DLL program to communicate with the data management system in the cloud platform, and the data management system in the cloud platform monitors the data file, analyzes the check data and stores the data through the file monitoring program.
The method comprises the steps of acquiring a multi-source test data sequence after data cleaning, clustering multi-source test data under the same test label, forming test data sets under each test label according to acquisition time stamps, and obtaining test indexes according to the test data sets, wherein the test labels comprise test content, test environment, test time and other information; establishing a search task by using test labels of each test data set, and obtaining test indexes and corresponding judgment standards with similarity meeting preset standards by using similarity calculation in a cloud platform data search space; performing data matching on the retrieved test indexes and the test indexes in each test data set, and after the matching of all the test indexes in each test data set is finished, distributing corresponding judgment standards to the test indexes as the judgment standards of the current test indexes in each test data set; and judging the multi-source test data of the tested piece according to the judging standard of the current test index, and obtaining index judging data.
FIG. 2 shows a flow chart of a method for representing and learning multi-source test data and index interpretation data through undirected heterograms.
According to the embodiment of the invention, the index interpretation data and the multi-source test data sequence are imported into a low-dimensional vector space, and the multi-source test data and the index interpretation data corresponding to the test indexes under each label are represented and learned through undirected heterograph in the low-dimensional vector space, specifically:
S202, acquiring common test contents and test items through cloud platform data statistics, performing data retrieval according to the common test contents and the test items to acquire corresponding analysis reports, and reading historical test indexes and comprehensive analysis results of the historical test indexes in the analysis reports;
s204, importing current multi-source test data and index interpretation data of the to-be-tested piece into a low-dimensional vector space, and judging whether a historical interaction relation exists in the test index under the current test item according to a comprehensive analysis result of the historical test index;
s206, constructing undirected heterograms of the test data by utilizing the multi-source test data and the index interpretation data of the to-be-tested piece in the low-dimensional vector space, taking the test data and the index interpretation data of each test index as nodes of the undirected heterograms, marking the nodes as test index nodes, and taking the historical interaction relationship between the test indexes as edge structures between the nodes.
The undirected heterograph ,/>Representing a test index node set, wherein the nodes comprise multisource test data set index interpretation data under test indexes,/I->And (3) representing a set of interaction relations among the test index nodes, and if the historical test comprehensive analysis result contains the original basic data of the two test indexes at the same time, proving that the two test indexes participate in comprehensive analysis together, wherein the interaction relations exist among the two test indexes.
FIG. 3 shows a flow chart of a method for constructing a test data comprehensive analysis model to obtain a comprehensive analysis result.
According to the embodiment of the invention, a test data comprehensive analysis model is constructed, the multi-source test data is subjected to data fusion, and a comprehensive analysis result is obtained, specifically:
s302, constructing a test data comprehensive analysis model based on deep learning, performing representation learning on undirected heterograms of test data through a graph convolutional neural network, acquiring preset test items of a piece to be tested, and calculating information contribution rates of all test indexes according to the preset test items;
s304, selecting a test index with the maximum information contribution rate, marking a corresponding test index node, acquiring a node which has a connection relation with the marked node in the undirected heterograph as an associated node, and generating a characteristic test index node set by the marked node and the associated node;
s306, acquiring the mahalanobis distance between each characteristic test index node in the characteristic test index node set and other test index nodes, presetting a mahalanobis distance threshold range, judging whether the mahalanobis distance is in the preset mahalanobis distance threshold range, and if so, taking the test index node as an adjacent node of the characteristic test index node closest to the test index node;
S308, carrying out heterogeneous fusion on adjacent nodes and characteristic test index nodes through a message transmission mechanism and a neighbor aggregation mechanism of the graph convolutional neural network to generate embedded representation of the characteristic test index nodes;
s310, acquiring relevant test data and prediction data according to test items of a piece to be tested, generating a training data set, training a gated circulating neural network through the training data, and inputting embedded representation of characteristic test index nodes into the gated circulating neural network for prediction analysis;
s312, matching the predictive analysis with a preset report form, and outputting a predictive analysis result.
The method includes the steps that information contribution rates of other test indexes are obtained, and weight distribution is carried out on characteristic test index nodes by taking the information contribution rates as attention weights; performing feature aggregation according to the attention weight in combination with other test index nodes, updating self-representation of the feature index nodes, and generating embedded vector representation with other test index data features;
calculating the contribution degree of the test index information, namely marking m test fingers asNormalizing the test index, and calculating a correlation coefficient matrix based on the normalized test index >Characteristic value of +.>，，/>Representing an m-order identity matrix; obtaining test factors of test indexes, supposing that the test items comprise information of all the test indexes, judging the capability of the test factors to interpret the test items through variance contribution of the test factors, comparing and judging according to a preset variance contribution threshold, retaining important test factors of all the test indexes, and adding the proportion of the information of each important test factor in the test indexes to the information of the test items to obtain the index->Information contribution rate of (a) test indexCan be expressed as several test factors +.>And factor load->The products are added, and the load factor matrix is +.>M represents the total number of test indexes, i represents the number of test index items, n represents the total number of factor loads in the test index i, and j represents the factor load item in the test index iA number;
the information contribution rateThe calculation formula of (2) is as follows: />The method comprises the steps of carrying out a first treatment on the surface of the The information contribution rate is used as the attention weight, the attention weight is combined with other test index nodes to perform feature aggregation, the self representation of the feature index nodes is updated, the embedded representation of the feature test index nodes is obtained according to a neighbor aggregation mechanism, and the formula for aggregation of adjacent nodes is as follows: />Wherein->Embedded representation of a representation feature test indicator node, +. >Representing an activation function->Representing an initial representation of the characteristic test indicator node, +.>Representing adjacent nodes->Attention weight, weight->Representing a matrix of feature transformation parameters.
Generally, comprehensive analysis of a piece to be tested comprises life prediction, degradation prediction, performance prediction and the like, a gated circulating neural network is selected as a second half data analysis part of a test data comprehensive analysis model, the part can carry out adaptive setting of deep learning networks such as CNN, LSTM and the like according to test items, embedded representation of characteristic test index nodes is imported into the gated circulating neural network, and the gated circulating neural network comprisesReset gate and update gate two gating structures. The final output state of the gated recurrent neural network is the preamble stateCandidate state->Is added by weight, and the weight of the two is obtained by updating a gateControl, candidate state is reset gate +.>Control, obtaining final estimated state +.>，。
It should be noted that, according to the test data comprehensive analysis model, the test data analysis template is customized, specifically: acquiring the position of the basic data corresponding to the fusion test data in the low-dimensional vector space according to the test data comprehensive analysis model, training the comprehensive analysis model according to the basic data, selecting a report pattern after the comprehensive analysis model accords with the verification standard, and matching with the test item label to generate a data analysis template; carrying out structural processing on the data analysis templates of different test items, storing the data analysis templates into a cloud platform database, and calling the data analysis templates according to the test requirements of the to-be-tested piece; acquiring historical analysis reports of different tested pieces in the preset time of the same enterprise user, extracting test tag information, judgment index and judgment standard information and report information of the enterprise user in the historical analysis reports, and acquiring preference characteristics of the enterprise user according to the extracted information; and customizing the enterprise user portrait through the preference characteristics, configuring a testing system of a current piece to be tested and a multi-source testing data management analysis environment in advance according to the enterprise user portrait, and updating the user portrait according to the change of the testing requirements of the enterprise user.
If the current test item does not have the existing test data analysis template after being matched by the test data analysis template, calculating the information contribution rate of the test index of the current test item, and screening the test index meeting the preset requirement according to the information contribution rate; sequencing the screened test indexes according to the information contribution rate, and sequentially calculating the pearson correlation coefficient with the test indexes corresponding to the test data analysis templates in the cloud platform database according to the sequencing result; acquiring a test data analysis template corresponding to test data with the pearson correlation coefficient meeting the requirements, and setting the priority of the acquired test data analysis template according to the sorting result of the test indexes obtained by screening; and selecting the test data analysis template with the highest priority to perform data migration training, and performing comprehensive analysis on the current test data by using the test data analysis template after the migration training.
Fig. 4 shows a block diagram of a management system for multi-source test data based on a cloud platform according to the present invention.
The second aspect of the present invention also provides a management system 4 for multi-source test data based on a cloud platform, the system comprising: the memory 41 and the processor 42, wherein the memory comprises a management method program of the multi-source test data based on the cloud platform, and the management method program of the multi-source test data based on the cloud platform realizes the following steps when being executed by the processor:
Collecting test data of a tested piece, matching the test label, and sending the test data to a cloud platform for storage, wherein the cloud platform carries out data cleaning on the multi-source test data to remove obvious abnormal test data in each multi-source test data sequence;
classifying the multi-source test data after data cleaning according to the test tags, obtaining test data sets under each test tag, and obtaining index interpretation data by matching test standards of test indexes in the test data sets with the test tags;
the index interpretation data and the multi-source test data sequence are imported into a low-dimensional vector space, and multi-source test data and index interpretation data corresponding to the test indexes under each label are represented and learned through undirected heterograms in the low-dimensional vector space;
and constructing a comprehensive analysis model of the test data, carrying out data fusion on the multi-source test data, acquiring a comprehensive analysis result, matching multiple report forms according to preset test items of the to-be-tested piece, and generating an analysis report of the to-be-tested piece based on the test report.
It should be noted that, the automatic test system acquires the multi-source test data of the part to be tested, and invokes the communication DLL program to communicate with the data management system in the cloud platform, and the data management system in the cloud platform monitors the data file, analyzes the check data and stores the data through the file monitoring program.
The method comprises the steps of acquiring a multi-source test data sequence after data cleaning, clustering multi-source test data under the same test label, forming test data sets under each test label according to acquisition time stamps, and obtaining test indexes according to the test data sets, wherein the test labels comprise test content, test environment, test time and other information; establishing a search task by using test labels of each test data set, and obtaining test indexes and corresponding judgment standards with similarity meeting preset standards by using similarity calculation in a cloud platform data search space; performing data matching on the retrieved test indexes and the test indexes in each test data set, and after the matching of all the test indexes in each test data set is finished, distributing corresponding judgment standards to the test indexes as the judgment standards of the current test indexes in each test data set; and judging the multi-source test data of the tested piece according to the judging standard of the current test index, and obtaining index judging data.
According to the embodiment of the invention, the index interpretation data and the multi-source test data sequence are imported into a low-dimensional vector space, and the multi-source test data and the index interpretation data corresponding to the test indexes under each label are represented and learned through undirected heterograph in the low-dimensional vector space, specifically:
Acquiring common test contents and test items through cloud platform data statistics, performing data retrieval according to the common test contents and the test items to acquire corresponding analysis reports, and reading historical test indexes in the analysis reports and comprehensive analysis results of the historical test indexes;
importing current multi-source test data and index interpretation data of the to-be-tested piece into a low-dimensional vector space, and judging whether a historical interaction relation exists in the test index under the current test item according to a comprehensive analysis result of the historical test index;
constructing undirected heterograph of test data by utilizing multi-source test data and index interpretation data of a piece to be tested in a low-dimensional vector space, taking the test data and index interpretation data of each test index as nodes of the undirected heterograph, marking the nodes as test index nodes, and taking historical interaction relations among the test indexes as edge structures among the nodes.
The undirected heterograph ,/>Representing a test index node set, wherein the nodes comprise multisource test data set index interpretation data under test indexes,/I->And (3) representing a set of interaction relations among the test index nodes, and if the historical test comprehensive analysis result contains the original basic data of the two test indexes at the same time, proving that the two test indexes participate in comprehensive analysis together, wherein the interaction relations exist among the two test indexes.
According to the embodiment of the invention, a test data comprehensive analysis model is constructed, the multi-source test data is subjected to data fusion, and a comprehensive analysis result is obtained, specifically:
constructing a test data comprehensive analysis model based on deep learning, performing representation learning on undirected heterograms of the test data through a graph convolutional neural network, acquiring preset test items of a piece to be tested, and calculating information contribution rates of all test indexes according to the preset test items;
selecting a test index with the maximum information contribution rate, marking corresponding test index nodes, obtaining nodes with connection relation with the marked nodes in the undirected heterogram as associated nodes, and generating a characteristic test index node set by the marked nodes and the associated nodes;
acquiring the mahalanobis distance between each characteristic test index node in the characteristic test index node set and other test index nodes, presetting a mahalanobis distance threshold range, judging whether the mahalanobis distance is in the preset mahalanobis distance threshold range, and if so, taking the test index node as an adjacent node of the characteristic test index node closest to the test index node;
heterogeneous fusion is carried out on the adjacent nodes and the characteristic test index nodes through a message transmission mechanism and a neighbor aggregation mechanism of the graph convolutional neural network, and embedded representation of the characteristic test index nodes is generated;
Acquiring relevant test data and prediction data according to test items of a piece to be tested, generating a training data set, training a gated circulating neural network through the training data, and inputting embedded representation of characteristic test index nodes into the gated circulating neural network for prediction analysis;
and matching the predictive analysis with a preset report form, and outputting a predictive analysis result.
The method includes the steps that information contribution rates of other test indexes are obtained, and weight distribution is carried out on characteristic test index nodes by taking the information contribution rates as attention weights; performing feature aggregation according to the attention weight in combination with other test index nodes, updating self-representation of the feature index nodes, and generating embedded vector representation with other test index data features;
calculating the contribution degree of the test index information, namely marking m test fingers asNormalizing the test index, and calculating a correlation coefficient matrix based on the normalized test index>Characteristic value of +.>，，/>Representing an m-order identity matrix; obtaining test factors of test indexes, supposing that the test items comprise information of all the test indexes, judging the capability of the test factors to interpret the test items through variance contribution of the test factors, comparing and judging according to a preset variance contribution threshold, retaining important test factors of all the test indexes, and adding the proportion of the information of each important test factor in the test indexes to the information of the test items to obtain the index- >Information contribution rate of (a) test indexCan be expressed as several test factors +.>And factor load->The products are added, and the load factor matrix is +.>M represents the total number of test indexes, i represents the number of test index items, n represents the total number of factor loads in the test index i, and j represents the number of factor loads in the test index i;
the information contribution rateThe calculation formula of (2) is as follows: />The method comprises the steps of carrying out a first treatment on the surface of the The information contribution rate is used as attention weight, the attention weight is combined with other test index nodes to perform feature aggregation, the self representation of the feature index nodes is updated, and the feature index nodes are aggregated according to neighborsThe synthesis mechanism obtains an embedded representation of the feature test index node.
Generally, comprehensive analysis of a piece to be tested comprises life prediction, degradation prediction, performance prediction and the like, a gating circulating neural network is selected as a second half data analysis part of a test data comprehensive analysis model, the part can carry out adaptive setting of deep learning networks such as CNN, LSTM and the like according to test items, embedded representations of characteristic test index nodes are led into the gating circulating neural network, and the gating circulating neural network comprises two gating structures of a reset gate and an update gate. The final output state of the gated recurrent neural network is the preamble state Candidate state->Is added by weight, and the weight of the two is obtained by updating a gateControl, candidate state is reset gate +.>Control, obtaining final estimated state +.>，。
It should be noted that, according to the test data comprehensive analysis model, the test data analysis template is customized, specifically: acquiring the position of the basic data corresponding to the fusion test data in the low-dimensional vector space according to the test data comprehensive analysis model, training the comprehensive analysis model according to the basic data, selecting a report pattern after the comprehensive analysis model accords with the verification standard, and matching with the test item label to generate a data analysis template; carrying out structural processing on the data analysis templates of different test items, storing the data analysis templates into a cloud platform database, and calling the data analysis templates according to the test requirements of the to-be-tested piece; acquiring historical analysis reports of different tested pieces in the preset time of the same enterprise user, extracting test tag information, judgment index and judgment standard information and report information of the enterprise user in the historical analysis reports, and acquiring preference characteristics of the enterprise user according to the extracted information; and customizing the enterprise user portrait through the preference characteristics, configuring a testing system of a current piece to be tested and a multi-source testing data management analysis environment in advance according to the enterprise user portrait, and updating the user portrait according to the change of the testing requirements of the enterprise user.
The third aspect of the present invention also provides a computer readable storage medium, where the computer readable storage medium includes a method program for managing multi-source test data based on a cloud platform, where the method program for managing multi-source test data based on a cloud platform implements the steps of the method for managing multi-source test data based on a cloud platform according to any one of the above.
In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above described device embodiments are only illustrative, e.g. the division of the units is only one logical function division, and there may be other divisions in practice, such as: multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. In addition, the various components shown or discussed may be coupled or directly coupled or communicatively coupled to each other via some interface, whether indirectly coupled or communicatively coupled to devices or units, whether electrically, mechanically, or otherwise.
The units described above as separate components may or may not be physically separate, and components shown as units may or may not be physical units; can be located in one place or distributed to a plurality of network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present invention may be integrated in one processing unit, or each unit may be separately used as one unit, or two or more units may be integrated in one unit; the integrated units may be implemented in hardware or in hardware plus software functional units.
Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the above method embodiments may be implemented by hardware related to program instructions, and the foregoing program may be stored in a computer readable storage medium, where the program, when executed, performs steps including the above method embodiments; and the aforementioned storage medium includes: a mobile storage device, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk or an optical disk, or the like, which can store program codes.
Alternatively, the above-described integrated units of the present invention may be stored in a computer-readable storage medium if implemented in the form of software functional modules and sold or used as separate products. Based on such understanding, the technical solutions of the embodiments of the present invention may be embodied in essence or a part contributing to the prior art in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, ROM, RAM, magnetic or optical disk, or other medium capable of storing program code.
The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (5)
1. The management method of the multi-source test data based on the cloud platform is characterized by comprising the following steps of:
collecting test data of a tested piece, matching the test label, and sending the test data to a cloud platform for storage, wherein the cloud platform carries out data cleaning on the multi-source test data to remove obvious abnormal test data in each multi-source test data sequence;
classifying the multi-source test data after data cleaning according to the test tags, obtaining test data sets under each test tag, and obtaining index interpretation data by matching test standards of test indexes in the test data sets with the test tags;
the index interpretation data and the multi-source test data sequence are imported into a low-dimensional vector space, and multi-source test data and index interpretation data corresponding to the test indexes under each label are represented and learned through undirected heterograms in the low-dimensional vector space;
constructing a comprehensive analysis model of test data, carrying out data fusion on multi-source test data, acquiring a comprehensive analysis result, matching multiple report forms according to preset test items of the to-be-tested piece, and generating an analysis report of the to-be-tested piece based on the test report;
classifying the multi-source test data after data cleaning according to the test tags, obtaining test data sets under each test tag, and utilizing the test tags to match the judgment standards of the test indexes in the test data sets to obtain index judgment data, wherein the method specifically comprises the following steps:
Acquiring a multi-source test data sequence after data cleaning, clustering multi-source test data under the same test label, forming test data sets under each test label according to acquisition time stamps, and obtaining test indexes according to the test data sets;
establishing a search task by using test labels of each test data set, and obtaining test indexes and corresponding judgment standards with similarity meeting preset standards by using similarity calculation in a cloud platform data search space;
performing data matching on the retrieved test indexes and the test indexes in each test data set, and after the matching of all the test indexes in each test data set is finished, distributing corresponding judgment standards to the test indexes as the judgment standards of the current test indexes in each test data set;
judging the multi-source test data of the tested piece according to the judging standard of the current test index to obtain index judging data;
the index interpretation data and the multi-source test data sequence are imported into a low-dimensional vector space, and the multi-source test data and the index interpretation data corresponding to the test indexes under each label are represented and learned through undirected heterograph in the low-dimensional vector space, specifically:
Acquiring common test contents and test items through cloud platform data statistics, performing data retrieval according to the common test contents and the test items to acquire corresponding analysis reports, and reading historical test indexes in the analysis reports and comprehensive analysis results of the historical test indexes;
importing current multi-source test data and index interpretation data of the to-be-tested piece into a low-dimensional vector space, and judging whether a historical interaction relation exists in the test index under the current test item according to a comprehensive analysis result of the historical test index;
constructing undirected heterograms of the test data by utilizing the multi-source test data and the index interpretation data of the to-be-tested piece in the low-dimensional vector space, taking the test data and the index interpretation data of each test index as nodes of the undirected heterograms, marking the nodes as test index nodes, and taking the historical interaction relations among the test indexes as edge structures among the nodes;
constructing a comprehensive analysis model of test data, carrying out data fusion on multi-source test data, and acquiring a comprehensive analysis result, wherein the comprehensive analysis result comprises the following specific steps:
constructing a test data comprehensive analysis model based on deep learning, performing representation learning on undirected heterograms of the test data through a graph convolutional neural network, acquiring preset test items of a piece to be tested, and calculating information contribution rates of all test indexes according to the preset test items;
Selecting a test index with the maximum information contribution rate, marking corresponding test index nodes, obtaining nodes with connection relation with the marked nodes in the undirected heterogram as associated nodes, and generating a characteristic test index node set by the marked nodes and the associated nodes;
acquiring the mahalanobis distance between each characteristic test index node in the characteristic test index node set and other test index nodes, presetting a mahalanobis distance threshold range, judging whether the mahalanobis distance is in the preset mahalanobis distance threshold range, and if so, taking the test index node as an adjacent node of the characteristic test index node closest to the test index node;
heterogeneous fusion is carried out on the adjacent nodes and the characteristic test index nodes through a message transmission mechanism and a neighbor aggregation mechanism of the graph convolutional neural network, and embedded representation of the characteristic test index nodes is generated;
acquiring relevant test data and prediction data according to test items of a piece to be tested, generating a training data set, training a gated circulating neural network through the training data, and inputting embedded representation of characteristic test index nodes into the gated circulating neural network for prediction analysis;
and matching the predictive analysis with a preset report form, and outputting a predictive analysis result.
2. The method for managing multi-source test data based on the cloud platform according to claim 1, wherein the generating of the embedded representation of the feature test index node is specifically:
the information contribution rate of other test indexes is obtained, and the information contribution rate is used as attention weight to carry out weight distribution on the characteristic test index nodes;
and carrying out feature aggregation according to the attention weight in combination with other test index nodes, updating the self-representation of the feature index nodes, and generating embedded vector representations with other test index data features.
3. The method for managing multi-source test data based on a cloud platform according to claim 1, further comprising:
acquiring the position of the basic data corresponding to the fusion test data in the low-dimensional vector space according to the test data comprehensive analysis model, training the comprehensive analysis model according to the basic data, selecting a report pattern after the comprehensive analysis model accords with the verification standard, and matching with the test item label to generate a data analysis template;
carrying out structural processing on the data analysis templates of different test items, storing the data analysis templates into a cloud platform database, and calling the data analysis templates according to the test requirements of the to-be-tested piece;
Acquiring historical analysis reports of different tested pieces in the preset time of the same enterprise user, extracting test tag information, judgment index and judgment standard information and report information of the enterprise user in the historical analysis reports, and acquiring preference characteristics of the enterprise user according to the extracted information;
and customizing the enterprise user portrait through the preference characteristics, configuring a testing system of a current piece to be tested and a multi-source testing data management analysis environment in advance according to the enterprise user portrait, and updating the user portrait according to the change of the testing requirements of the enterprise user.
4. A management system for multisource test data based on a cloud platform, the system comprising: the system comprises a memory and a processor, wherein the memory comprises a management method program of multi-source test data based on a cloud platform, and the management method program of the multi-source test data based on the cloud platform realizes the following steps when being executed by the processor:
collecting test data of a tested piece, matching the test label, and sending the test data to a cloud platform for storage, wherein the cloud platform carries out data cleaning on the multi-source test data to remove obvious abnormal test data in each multi-source test data sequence;
Classifying the multi-source test data after data cleaning according to the test tags, obtaining test data sets under each test tag, and obtaining index interpretation data by matching test standards of test indexes in the test data sets with the test tags;
the index interpretation data and the multi-source test data sequence are imported into a low-dimensional vector space, and multi-source test data and index interpretation data corresponding to the test indexes under each label are represented and learned through undirected heterograms in the low-dimensional vector space;
constructing a comprehensive analysis model of test data, carrying out data fusion on multi-source test data, acquiring a comprehensive analysis result, matching multiple report forms according to preset test items of the to-be-tested piece, and generating an analysis report of the to-be-tested piece based on the test report;
classifying the multi-source test data after data cleaning according to the test tags, obtaining test data sets under each test tag, and utilizing the test tags to match the judgment standards of the test indexes in the test data sets to obtain index judgment data, wherein the method specifically comprises the following steps:
acquiring a multi-source test data sequence after data cleaning, clustering multi-source test data under the same test label, forming test data sets under each test label according to acquisition time stamps, and obtaining test indexes according to the test data sets;
Establishing a search task by using test labels of each test data set, and obtaining test indexes and corresponding judgment standards with similarity meeting preset standards by using similarity calculation in a cloud platform data search space;
performing data matching on the retrieved test indexes and the test indexes in each test data set, and after the matching of all the test indexes in each test data set is finished, distributing corresponding judgment standards to the test indexes as the judgment standards of the current test indexes in each test data set;
judging the multi-source test data of the tested piece according to the judging standard of the current test index to obtain index judging data;
the index interpretation data and the multi-source test data sequence are imported into a low-dimensional vector space, and the multi-source test data and the index interpretation data corresponding to the test indexes under each label are represented and learned through undirected heterograph in the low-dimensional vector space, specifically:
acquiring common test contents and test items through cloud platform data statistics, performing data retrieval according to the common test contents and the test items to acquire corresponding analysis reports, and reading historical test indexes in the analysis reports and comprehensive analysis results of the historical test indexes;
Importing current multi-source test data and index interpretation data of the to-be-tested piece into a low-dimensional vector space, and judging whether a historical interaction relation exists in the test index under the current test item according to a comprehensive analysis result of the historical test index;
constructing undirected heterograms of the test data by utilizing the multi-source test data and the index interpretation data of the to-be-tested piece in the low-dimensional vector space, taking the test data and the index interpretation data of each test index as nodes of the undirected heterograms, marking the nodes as test index nodes, and taking the historical interaction relations among the test indexes as edge structures among the nodes;
constructing a comprehensive analysis model of test data, carrying out data fusion on multi-source test data, and acquiring a comprehensive analysis result, wherein the comprehensive analysis result comprises the following specific steps:
constructing a test data comprehensive analysis model based on deep learning, performing representation learning on undirected heterograms of the test data through a graph convolutional neural network, acquiring preset test items of a piece to be tested, and calculating information contribution rates of all test indexes according to the preset test items;
selecting a test index with the maximum information contribution rate, marking corresponding test index nodes, obtaining nodes with connection relation with the marked nodes in the undirected heterogram as associated nodes, and generating a characteristic test index node set by the marked nodes and the associated nodes;
Acquiring the mahalanobis distance between each characteristic test index node in the characteristic test index node set and other test index nodes, presetting a mahalanobis distance threshold range, judging whether the mahalanobis distance is in the preset mahalanobis distance threshold range, and if so, taking the test index node as an adjacent node of the characteristic test index node closest to the test index node;
heterogeneous fusion is carried out on the adjacent nodes and the characteristic test index nodes through a message transmission mechanism and a neighbor aggregation mechanism of the graph convolutional neural network, and embedded representation of the characteristic test index nodes is generated;
acquiring relevant test data and prediction data according to test items of a piece to be tested, generating a training data set, training a gated circulating neural network through the training data, and inputting embedded representation of characteristic test index nodes into the gated circulating neural network for prediction analysis;
and matching the predictive analysis with a preset report form, and outputting a predictive analysis result.
5. The cloud platform-based multi-source test data management system of claim 4, wherein the generating of the embedded representation of the feature test index node is specifically:
the information contribution rate of other test indexes is obtained, and the information contribution rate is used as attention weight to carry out weight distribution on the characteristic test index nodes;
And carrying out feature aggregation according to the attention weight in combination with other test index nodes, updating the self-representation of the feature index nodes, and generating embedded vector representations with other test index data features.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310269927.4A CN115982646B (en) | 2023-03-20 | 2023-03-20 | Management method and system for multisource test data based on cloud platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310269927.4A CN115982646B (en) | 2023-03-20 | 2023-03-20 | Management method and system for multisource test data based on cloud platform |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115982646A CN115982646A (en) | 2023-04-18 |
| CN115982646B true CN115982646B (en) | 2023-07-18 |
Family
ID=85958579
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310269927.4A Active CN115982646B (en) | 2023-03-20 | 2023-03-20 | Management method and system for multisource test data based on cloud platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115982646B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118035334B (en) * | 2024-04-12 | 2024-06-11 | 山东棱镜智能科技有限公司 | Data element analysis and arrangement method and system based on artificial intelligence |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112199421A (en) * | 2020-12-04 | 2021-01-08 | 中国电力科学研究院有限公司 | Multi-source heterogeneous data fusion and measurement data multi-source mutual verification method and system |
| CN113392894A (en) * | 2021-06-09 | 2021-09-14 | 瓴域影诺(北京)科技有限公司 | Cluster analysis method and system for multi-group mathematical data |
Family Cites Families (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR3013834B1 (en) * | 2013-11-28 | 2015-12-25 | Airbus Operations Sas | METHOD FOR MERGING SENSOR DATA USING A COHERENCE CRITERION |
| CN106651188A (en) * | 2016-12-27 | 2017-05-10 | 贵州电网有限责任公司贵阳供电局 | Electric transmission and transformation device multi-source state assessment data processing method and application thereof |
| CN110958220B (en) * | 2019-10-24 | 2020-12-29 | 中国科学院信息工程研究所 | Network space security threat detection method and system based on heterogeneous graph embedding |
| CN111259133B (en) * | 2020-01-17 | 2021-02-19 | 成都信息工程大学 | Personalized recommendation method integrating multiple information |
| US11461367B2 (en) * | 2020-04-13 | 2022-10-04 | Singapore University Of Technology And Design | Multi-source data management mechanism and platform |
| CN111678709B (en) * | 2020-06-16 | 2021-04-30 | 山东大学 | Electric vehicle distributed networked test system and method based on cloud platform |
| CN112199510A (en) * | 2020-09-17 | 2021-01-08 | 中国建设银行股份有限公司 | Fraud probability determination method and device, electronic equipment and storage medium |
| CN112561165A (en) * | 2020-12-16 | 2021-03-26 | 南京航空航天大学 | Multidimensional time series data prediction method based on combined model |
| CN114218850A (en) * | 2021-11-09 | 2022-03-22 | 大连理工大学 | Heterogeneous multi-relation graph representation learning method |
| CN114596061B (en) * | 2022-03-02 | 2023-03-24 | 穗保(广州)科技有限公司 | Project data management method and system based on big data |
| CN114928548B (en) * | 2022-04-26 | 2023-07-28 | 苏州大学 | Social network information propagation scale prediction method and device |
| CN114997546B (en) * | 2022-08-05 | 2022-10-25 | 天津市地质研究和海洋地质中心 | Comprehensive survey evaluation method, system and medium for monitoring well based on multi-source data |
| CN115080574B (en) * | 2022-08-22 | 2022-11-18 | 西安弘捷电子技术有限公司 | Method and device for processing electronic product test data, computer equipment and storage medium |
| CN115641097B (en) * | 2022-12-23 | 2023-03-28 | 深圳市仕瑞达自动化设备有限公司 | Non-standard machining collaborative manufacturing management method and system based on cloud platform |
-
2023
- 2023-03-20 CN CN202310269927.4A patent/CN115982646B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112199421A (en) * | 2020-12-04 | 2021-01-08 | 中国电力科学研究院有限公司 | Multi-source heterogeneous data fusion and measurement data multi-source mutual verification method and system |
| CN113392894A (en) * | 2021-06-09 | 2021-09-14 | 瓴域影诺(北京)科技有限公司 | Cluster analysis method and system for multi-group mathematical data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115982646A (en) | 2023-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110163647B (en) | Data processing method and device | |
| CN109409561B (en) | Construction method of multi-time scale time sequence collaborative prediction model | |
| CN115878904B (en) | Intellectual property personalized recommendation method, system and medium based on deep learning | |
| CN113626607B (en) | Abnormal work order identification method and device, electronic equipment and readable storage medium | |
| CN117271767B (en) | Operation and maintenance knowledge base establishing method based on multiple intelligent agents | |
| CN115982646B (en) | Management method and system for multisource test data based on cloud platform | |
| CN115564071A (en) | Method and system for generating data labels of power Internet of things equipment | |
| CN111210332A (en) | Method and device for generating post-loan management strategy and electronic equipment | |
| CN115936748A (en) | Business big data analysis method and system | |
| CN115660695A (en) | Customer service personnel label portrait construction method and device, electronic equipment and storage medium | |
| CN115600818A (en) | Multi-dimensional scoring method and device, electronic equipment and storage medium | |
| CN112712270B (en) | Information processing method, device, equipment and storage medium | |
| CN114580955A (en) | Policy recommendation method, system, device and storage medium | |
| CN114240102A (en) | Line loss abnormal data identification method and device, electronic equipment and storage medium | |
| CN113313615A (en) | Method and device for quantitatively grading and grading enterprise judicial risks | |
| CN117668236B (en) | Analysis method, system and storage medium of patent standard fusion system | |
| CN117217710B (en) | Intelligent management method and system for virtual commodity and shortcut service | |
| CN116823069B (en) | Intelligent customer service quality inspection method based on text analysis and related equipment | |
| CN116341990B (en) | Knowledge management evaluation method and system for infrastructure engineering | |
| CN118211064A (en) | Training method and device of visit rate prediction model | |
| CN116719888A (en) | Cloud computing-based enterprise group service method, system and storage medium | |
| CN112819545A (en) | IP (Internet protocol) valuation method and system applying machine learning | |
| CN117217812A (en) | User behavior prediction method and device, computer equipment and storage medium | |
| CN117787512A (en) | Enterprise judicial management method, system, equipment and storage medium | |
| CN115577291A (en) | Data labeling method and device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |