CN115098370A - Reliability evaluation method, system, device and storage medium for software system - Google Patents
Reliability evaluation method, system, device and storage medium for software system Download PDFInfo
- Publication number
- CN115098370A CN115098370A CN202210731811.3A CN202210731811A CN115098370A CN 115098370 A CN115098370 A CN 115098370A CN 202210731811 A CN202210731811 A CN 202210731811A CN 115098370 A CN115098370 A CN 115098370A
- Authority
- CN
- China
- Prior art keywords
- test
- fault
- reliability
- evaluation
- software system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3692—Test management for test results analysis
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the application provides a reliability evaluation method, a system, equipment and a storage medium of a software system, the method obtains test process information and test packet information through an auxiliary monitoring module, performs statistical analysis on the test process information according to preset evaluation indexes to obtain test process parameters, performs fusion processing on the test process parameters and the test packet information to generate auxiliary evaluation parameters, obtains test result information from the test packet information, calculates the reliability confidence coefficient of the software system according to the auxiliary evaluation parameters and the test result information and preset reliability evaluation rules, realizes accurate evaluation on the reliability of the software system, comprehensively considers the influence of the test process information on the reliability evaluation of the software system due to the combination of the auxiliary evaluation parameters and the test result information, compared with the traditional reliability evaluation only according to the test result information, the accuracy and the objectivity of the reliability evaluation of the software system are greatly improved.
Description
Technical Field
The present application relates to the field of software testing technologies, and in particular, to a method, a system, a device, and a storage medium for reliability evaluation of a software system.
Background
The reliability of the software system refers to the capability of the software system to complete specified functions under given conditions and within specified time, is used for testing the fault handling capability of the system, and is a core index for measuring the quality of the software system.
At present, a method for evaluating reliability of a software system in a software test generally evaluates according to a test result, that is, performs a multifunctional test on software to obtain a test result, and evaluates reliability of the software system according to the test result, for example, if the test result meets a software test requirement, it is determined that the reliability of the software system is high, and if the test result does not meet the software test requirement, it is determined that the software system has a defect. According to the method for evaluating the reliability based on the test result, because the test mode is only qualitative measurement, accurate quantification of the reliability of the software system cannot be met, the accuracy of reliability evaluation is reduced, and the quality of the software system is not favorably improved.
Content of application
The embodiment of the application provides a reliability evaluation method, system, equipment and storage medium of a software system, and aims to solve the technical problem of low accuracy of reliability evaluation of the software system caused by reliability evaluation according to a test result.
In one aspect, the present application provides a reliability assessment method for a software system, which is applied to a reliability assessment system, where the reliability assessment system includes an auxiliary monitoring module for monitoring a test process of the software system, and the method includes:
acquiring test process information and test packet information of the software system through the auxiliary monitoring module;
carrying out statistical analysis on the test process information according to preset evaluation indexes to obtain test process parameters;
performing fusion processing on the test process parameters and the test packet information to generate auxiliary evaluation parameters;
obtaining test result information from the test packet information;
and calculating the reliability confidence coefficient of the software system according to the auxiliary evaluation parameters and the test result information and a preset reliability evaluation rule.
In one aspect, the present application provides a reliability evaluation system, where the reliability evaluation system includes an auxiliary monitoring module, configured to monitor a test process of the software system, and includes:
the acquisition module is used for acquiring the test process information and the test packet information of the software system through the auxiliary monitoring module;
the analysis module is used for carrying out statistical analysis on the test process information according to preset evaluation indexes to obtain test process parameters;
the fusion module is used for fusing the test process parameters and the test packet information to generate auxiliary evaluation parameters;
the extraction module is used for acquiring test result information from the test packet information;
and the determining module is used for calculating the reliability confidence coefficient of the software system according to the auxiliary evaluation parameters and the test result information and a preset reliability evaluation rule.
In one aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed by the processor, causes the processor to execute the steps of the reliability assessment method of the software system.
In one aspect, the present application provides a computer readable medium storing a computer program, which when executed by a processor, causes the processor to execute the steps of the reliability assessment method of the software system.
The embodiment of the application provides a reliability evaluation method of a software system, which comprises the steps of acquiring test process information and test packet information of the software system through an auxiliary monitoring module, carrying out statistical analysis on the test process information according to preset evaluation indexes to obtain test process parameters, carrying out fusion processing on the test process parameters and the test packet information to generate auxiliary evaluation parameters, acquiring test result information from the test packet information, calculating reliability confidence of the software system according to the auxiliary evaluation parameters and the test result information and preset reliability evaluation rules, and realizing accurate evaluation on the reliability of the software system, wherein the influence of the test process information on the reliability evaluation of the software system is comprehensively considered due to the combination of the auxiliary evaluation parameters and the test result information, compared with the traditional reliability evaluation only according to the test result information, the accuracy and the objectivity of the reliability evaluation of the software system are greatly improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Wherein:
FIG. 1 is a schematic diagram of an application scenario of a reliability evaluation method of a software system in an embodiment;
FIG. 2 is a flow diagram of a method for reliability evaluation of a software system in one embodiment;
FIG. 3 is a block diagram of a reliability evaluation system in accordance with one embodiment;
FIG. 4 is a block diagram of a computer device in one embodiment.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The reliability evaluation method of the software system provided by the application can be applied to the application environment shown in fig. 1, wherein the terminal device communicates with the server through the network. The terminal device may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices. The server may be implemented as a stand-alone server or as a server cluster consisting of a plurality of servers.
The system framework 100 may include terminal devices, networks, and servers. The network serves as a medium for providing a communication link between the terminal device and the server. The network may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
A user may use a terminal device to interact with a server over a network to receive or send messages or the like.
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to a smart phone, a tablet computer, an e-book reader, an MP3 player (Moving Picture experts Group Audio Layer III, motion Picture experts compression standard Audio Layer 3), an MP4 player (Moving Picture experts Group Audio Layer IV, motion Picture experts compression standard Audio Layer 4), a laptop portable computer, a desktop computer, and the like.
The server 105 may be a server providing various services, such as a background server providing support for pages displayed on the terminal devices 101, 102, 103.
It should be noted that, the reliability evaluation method of the software system provided by the embodiment of the present invention is executed by the server, and accordingly, the reliability evaluation system is disposed in the server.
It should be understood that the number of the terminal devices, the networks, and the servers in fig. 1 is only illustrative, and any number of the terminal devices, the networks, and the servers may be provided according to implementation requirements, and the terminal devices in the embodiment of the present invention may specifically correspond to an application system in actual production.
As shown in fig. 2, in an embodiment, a reliability evaluation method for a software system is provided, and the reliability evaluation method for the software system is applied to a reliability evaluation system, and the reliability evaluation system includes an auxiliary monitoring module for monitoring a test process of the software system. The reliability evaluation method of the software system specifically comprises the following steps:
The auxiliary monitoring module is configured to monitor a test process of software, that is, a test process of a real-time monitoring software system, and specifically, the auxiliary monitoring module may implement auxiliary monitoring by communicatively connecting an automatic test tool and a performance test tool to a server, where the automatic test tool includes but is not limited to a Selenium (web automation test tool set), and the performance test tool includes but is not limited to a LoadRunner. Specifically, a starting instruction is sent to the auxiliary monitoring module through the server, so that the test process information and the test packet information are obtained through the auxiliary monitoring module.
The test process information includes operation information performed in the test process and information received by the service logic layer, and is used to reflect real-time test information of the software system in the test process, for example, the severity of a fault at a certain time point, the CPU occupancy at a certain time, and the like. Specifically, the information may be obtained through information acquisition monitored by a performance testing tool in the auxiliary monitoring module. The test package information includes test item conclusions and information concerned by the user, such as the number of test cases, test results, and the like, and specifically, the test package information can be obtained through information monitored by an automatic test tool in the auxiliary monitoring module.
In this embodiment, the auxiliary monitoring module is started to obtain the test process information and the test packet information, so that it is ensured that the parameters for reliability evaluation are more objective and comprehensive, and further processing is performed based on the test process information and the test packet information in the following.
The preset evaluation index refers to preset information attribute items for testing the reliability degree of the evaluation software system related to the test process, such as fault severity, failure data, fault quantity and the like. The statistical analysis refers to collecting the test process information corresponding to each preset evaluation index at different time points, and then analyzing to obtain the test process parameters.
Specifically, according to preset evaluation indexes, corresponding data are extracted from the test process information, the corresponding data of each time point are summarized to obtain summarized data corresponding to each preset evaluation index, and the summarized data are subjected to aggregation analysis, such as weighted summation, maximum value taking, minimum value taking or mean value taking, so that test process parameters are obtained, quantification of the test process parameters is achieved, and improvement of accuracy of subsequent reliability evaluation is facilitated.
And 203, fusing the test process parameters and the test packet information to generate auxiliary evaluation parameters.
The auxiliary evaluation parameters are parameters for reflecting the reliability degree of the software system, specifically, the test process parameters and the test packet information are subjected to correlation analysis to generate the auxiliary evaluation parameters, more specifically, the test process parameters and the system test parameters can be classified respectively according to data items included in the auxiliary evaluation parameters to obtain each related data item, correlation is performed for each data item, for example, summation or median is performed to obtain a result value of each data item, the result values of each data item are aggregated to generate the auxiliary evaluation parameters, and quantitative calculation of the auxiliary evaluation parameters is realized.
The test result information refers to conclusion information of the software system test, such as test cases, the number of test cases, the test success rate, the test time, and the like, and specifically, the test report is obtained from the test packet information, and the test result information is obtained according to the test report.
And step 205, calculating the reliability confidence of the software system according to the auxiliary evaluation parameters and the test result information and the preset reliability evaluation rule.
The preset reliability evaluation rule refers to a preset calculation rule for quantifying the reliability degree of the software system, and for example, the calculation rule may be a reliability evaluation function, a calculation formula, or a machine learning model, such as a regression model.
Specifically, the auxiliary evaluation parameters and the test result information are used as independent variables of the preset reliability evaluation rule, and the dependent variable of the preset reliability evaluation rule is the reliability confidence of the software system, so that quantitative data reflecting the reliability degree of the software system is obtained, and the reliability of the software system is accurately evaluated. It can be understood that, in the embodiment, the auxiliary evaluation parameters and the test result information are combined, so that the influence of the test process information on the reliability evaluation of the software system is comprehensively considered, and compared with the conventional reliability evaluation only according to the test result information, the accuracy and the objectivity of the reliability evaluation of the software system are greatly improved.
The reliability evaluation method of the software system obtains the test process information and the test packet information of the software system through the auxiliary monitoring module, performing statistical analysis on the test process information according to preset evaluation indexes to obtain test process parameters, performing fusion processing on the test process parameters and the test packet information to generate auxiliary evaluation parameters, acquiring test result information from the test packet information, according to the auxiliary evaluation parameters and the test result information, the reliability confidence coefficient of the software system is calculated according to the preset reliability evaluation rule, the accurate evaluation of the reliability of the software system is realized, the auxiliary evaluation parameters and the test result information are combined, so that the influence of the test process information on the reliability evaluation of the software system is comprehensively considered, and compared with the traditional reliability evaluation only according to the test result information, the accuracy and the objectivity of the reliability evaluation of the software system are greatly improved.
In one embodiment, the preset evaluation indexes comprise system operation performance, fault severity and failure data, and each evaluation index corresponds to a preset statistical analysis rule; carrying out statistical analysis on the test process information according to preset evaluation indexes to obtain test process parameters, wherein the statistical analysis comprises the following steps: respectively extracting parameters and test time point data corresponding to the system operation performance, the fault severity and the failure data to obtain a first parameter, a second parameter, a third parameter and test time points corresponding to the first parameter, the second parameter and the third parameter; according to each preset evaluation index, respectively calculating and analyzing the first parameter, the second parameter and the third parameter according to corresponding preset statistical analysis rules to obtain a first process parameter, a second process parameter and a third process parameter; the test process parameters comprise a first process parameter, a second process parameter and a third process parameter and respective corresponding test time points.
The system operation performance is an index for reflecting the software system operation performance, such as memory usage rate and CPU occupancy rate. The severity of the fault is an index used for reflecting the severity of the fault in the software system testing process, the failure data is corresponding test data when the fault is determined to exist in the testing process, and the system operation performance, the severity of the fault and the failure data all correspond to a preset statistical analysis rule. The preset statistical analysis rule refers to a preset calculation rule for quantifying parameters corresponding to each evaluation index, and the preset statistical analysis rule may be a calculation formula, a quantification function or a quantification model, for example, the calculation formula of the preset statistical analysis rule for the test process parameters corresponding to the severity of the fault may be a calculation formula
Wherein L is 2 A test process parameter corresponding at a certain point in time for the severity of the fault.
The test process parameters refer to parameters corresponding to each preset evaluation index, and specifically, parameters and test time point data corresponding to the system operation performance, the fault severity and the failure data are respectively extracted from the test process information to obtain a first parameter, a second parameter, a third parameter and test time points corresponding to the first parameter, the second parameter and the third parameter; according to each preset evaluation index, respectively calculating and analyzing the first parameter, the second parameter and the third parameter according to corresponding preset statistical analysis rules to obtain a first process parameter, a second process parameter and a third process parameter; the test process parameters comprise a first process parameter, a second process parameter, a third process parameter and respective corresponding test timeAnd the test time point refers to the acquisition time of each test process parameter. Taking the above calculation formula as an example, L 2 Is a second process parameter, a, b, c are second parameters, and a, b, c are constants between (0, 100).
It can be understood that, since the preset evaluation indexes all reflect the relevant information of the test process, in this embodiment, by extracting the parameters of the corresponding preset evaluation indexes and calculating the test process parameters including the first process parameter, the second process parameter, the third process parameter and the respective corresponding test time points according to the respective parameters, the quantitative analysis of the test process is realized, and the test time points corresponding to the respective test process parameters are considered, so that the test process parameters are more finely expressed and analyzed.
In one embodiment, the auxiliary evaluation parameters include fault density and fault strength, the test packet information includes a fault discovery number and a fault discovery rate, and the fusion processing is performed on the test process parameters and the test packet information to generate the auxiliary evaluation parameters, including: analyzing a second process parameter in the test process parameters and a corresponding test time point to obtain first fault intensity and first fault density; analyzing a first process parameter, a third process parameter and respective corresponding test time points in the test process parameters to obtain a second fault intensity and a second fault density; calculating to obtain fault intensity according to the first fault intensity, the second fault intensity, the fault finding number and the fault finding rate; and calculating the fault density according to the first fault density, the second fault density, the fault discovery number and the fault discovery rate.
The failure density refers to the probability of failure occurrence in a certain period, and the failure intensity refers to the severity degree of failure in a certain period. Specifically, the test time point corresponding to the second process parameter is analyzed to determine the duration t of the second process parameter, and the duration t is determined according to the second process parameter L 2 Cumulative amount of (S) 2 The first fault density may be calculated using the following equation:
C 1 ＝S 2 *(t/T)
C 1 for the first failure density, T is the time of one cycle, T>t. The first failure strength can be calculated using the following formula:
K 1 ＝∑L 2 /S 2
K 1 is the first failure intensity, L 2 A value L corresponding to the second process parameter 2 . Then, a first process parameter, a third process parameter and respective corresponding test time points in the test process parameters are analyzed to obtain a second fault intensity and a second fault density, wherein a value L corresponding to the first process parameter 1 The larger the value L is, the better the system performance is, and the value L corresponding to the third process parameter 3 The larger the fault data, the more the fault data is, so that the first process parameter has an inverse proportional relationship with the first fault intensity and the first fault density, and the third process parameter has a direct proportional relationship with the first fault intensity and the first fault density, and therefore, the second fault density can be calculated by using the following formula:
C 2 ＝L 1 /L 3 +α
C 2 for the second failure density, α is an adjustment value, for example, α is 0.8. The second failure strength can be calculated using the following formula:
K 2 ＝L 1 /L 3 +σ
K 2 for the second failure density, σ is an adjustment value, e.g., σ equals 0.9. Then, the fault strength is calculated according to the first fault strength, the second fault strength, the fault discovery number and the fault discovery rate, and the fault strength can be calculated by adopting the following formula:
C＝C 1 +C 2 +max(F/S 2 ，M)
c is the fault intensity, F is the fault finding number, M is the fault finding rate max (F/S) 2 M) denotes taking F/S 2 The largest value in M;
calculating the fault density according to the first fault density, the second fault density, the fault finding number and the fault finding rate, and calculating the fault intensity by adopting the following formula:
K＝K 1 +K 2 +max(F/S 2 ，M)*λ
c is the fault intensity, and lambda is the adjustment value. In the embodiment, the test process parameters and the test packet information are fused, so that the quantitative calculation of the fault density and the fault intensity is realized, and the accuracy and the integrity of reliability evaluation are greatly improved compared with the traditional qualitative analysis only considering the fault intensity.
In one embodiment, the test packet information further includes test data, a plurality of test cases, and a plurality of test results corresponding to the test cases; obtaining test result information from the test packet information, including: and carrying out statistical analysis on each test result to generate test result information.
Specifically, mapping tables of different test results and test confidence levels may be pre-constructed, and corresponding confidence levels may be determined according to a plurality of test results corresponding to each test case, respectively, and used as test result information.
In one embodiment, calculating the reliability confidence of the software system according to the preset reliability evaluation rule based on the auxiliary evaluation parameters and the test result information comprises: determining a first confidence corresponding to the fault intensity; determining a second confidence corresponding to the fault density; determining a third confidence corresponding to the test result information; and calculating the reliability confidence coefficient of the software system according to the first confidence coefficient, the second confidence coefficient and the third confidence coefficient.
Wherein, the first confidence coefficient is a quantized value of the fault intensity, the second confidence coefficient is a quantized value of the fault density, and the third confidence coefficient is a quantized value of the test result information, specifically, the first confidence coefficient may be C in the above embodiment, the second confidence coefficient may be K in the above embodiment, or C, K may be corrected to obtain a corrected first confidence coefficient and a corrected second confidence coefficient, a confidence in the reliability of the software system is then calculated based on the first confidence, the second confidence, and the third confidence, and, more particularly, the reliability confidence of the software system may be calculated as an average value of the first confidence, the second confidence and the third confidence or a result of weighted summation, or the reliability confidence may be determined according to the result of the comprehensive analysis and the third confidence by performing the comprehensive analysis according to the first confidence and the second confidence.
In one embodiment, calculating a reliability confidence for the software system based on the first confidence, the second confidence, and the third confidence includes: acquiring weights corresponding to the first confidence coefficient and the second confidence coefficient; performing weighted calculation according to the first confidence coefficient, the second confidence coefficient and corresponding weights to obtain a comprehensive confidence coefficient; calculating an absolute difference between the comprehensive confidence coefficient and the third confidence coefficient; and when the absolute difference is greater than a preset threshold, determining a reliability confidence coefficient according to the comprehensive confidence coefficient.
Specifically, weights corresponding to the first confidence coefficient and the second confidence coefficient are obtained, weighting calculation is carried out according to the first confidence coefficient, the second confidence coefficient and the corresponding weights to obtain a comprehensive confidence coefficient, an absolute difference value between the comprehensive confidence coefficient and the third confidence coefficient is calculated, when the absolute difference value is larger than a preset threshold value, the fact that the difference between the comprehensive confidence coefficient determined based on the auxiliary evaluation parameters obtained after the test process information and the test package information are fused and the third confidence coefficient corresponding to the test result information is large indicates that a certain deviation exists in the test result, therefore, the third confidence coefficient is not considered, the reliability confidence coefficient is determined according to the comprehensive confidence coefficient, the influence of the third confidence coefficient with the deviation is avoided, simplicity and convenience are achieved, and the determination efficiency and accuracy of the reliability confidence coefficient are greatly improved.
In one embodiment, before calculating the fault density according to the first fault density, the fault discovery number and the fault discovery rate, the method further comprises: carrying out time sequence analysis on the test process information to obtain a test time sequence; updating and calculating the first fault density according to the test time sequence; and calculating to obtain the fault density according to the updated first fault density, the updated fault discovery number and the updated fault discovery rate.
The time sequence analysis is a process of decomposing the test process information according to the time sequence so as to obtain a test time sequence, the first fault density is updated and calculated according to the test time sequence, and the fault density is calculated according to the updated first fault density, the updated fault finding number and the updated fault finding rate, so that the accuracy of the quantification of the fault density is greatly improved, and the accuracy of the reliability confidence coefficient is further improved.
In one embodiment, the system operation performance includes at least one of CPU occupancy, frequency of use, and fluency of use.
Specifically, the system operation performance comprises at least one of the CPU occupancy rate, the use frequency and the use fluency, so that in the process of quantifying the system performance, quantification can be performed according to one or a combination of the CPU occupancy rate, the use frequency and the use fluency, and the accuracy of the first process parameter corresponding to the system operation performance is improved.
As shown in fig. 3, in an embodiment, a reliability evaluation system is provided, where the reliability evaluation system includes an auxiliary monitoring module for monitoring a test process of the software system, and the reliability evaluation system includes:
an obtaining module 301, configured to obtain, through the auxiliary monitoring module, test process information and test packet information of the software system;
the analysis module 302 is configured to perform statistical analysis on the test process information according to a preset evaluation index to obtain a test process parameter;
a fusion module 303, configured to perform fusion processing on the test process parameters and the test packet information to generate auxiliary evaluation parameters;
an extracting module 304, configured to obtain test result information from the test packet information;
a determining module 305, configured to calculate a reliability confidence of the software system according to a preset reliability evaluation rule according to the auxiliary evaluation parameter and the test result information.
In one embodiment, the analysis module comprises:
the extraction unit is used for respectively extracting parameters and test time point data corresponding to the system operation performance, the fault severity and the failure data from the test process information to obtain a first parameter, a second parameter, a third parameter and test time points corresponding to the parameters;
the first analysis unit is used for respectively carrying out calculation analysis on the first parameter, the second parameter and the third parameter according to each preset evaluation index and the corresponding preset statistical analysis rule to obtain a first process parameter, a second process parameter and a third process parameter; the test process parameters include a first process parameter, a second process parameter, and a third process parameter and the respective corresponding test time points.
In one embodiment, the fusion module comprises:
the second analysis unit is used for analyzing the second process parameters in the test process parameters and the corresponding test time points to obtain first fault intensity and first fault density;
the third analysis unit is used for analyzing the first process parameter and the third process parameter in the test process parameters and the test time points corresponding to the first process parameter and the third process parameter to obtain a second fault intensity and a second fault density;
the first calculating unit is used for calculating the fault intensity according to the first fault intensity, the second fault intensity, the fault finding number and the fault finding rate;
and the second calculating unit is used for calculating the fault density according to the first fault density, the second fault density, the fault discovery number and the fault discovery rate.
In one embodiment, the extraction module comprises: and the statistical unit is used for performing statistical analysis on each test result to generate test result information.
In one embodiment, the determining module includes:
the first determining unit is used for determining a first confidence corresponding to the fault intensity;
the second determining unit is used for determining a second confidence coefficient corresponding to the fault density;
a third determining unit, configured to determine a third confidence corresponding to the test result information;
and the third calculating unit is used for calculating the reliability confidence coefficient of the software system according to the first confidence coefficient, the second confidence coefficient and the third confidence coefficient.
In one embodiment, the third calculation unit includes:
the obtaining subunit is configured to obtain weights corresponding to the first confidence degree and the second confidence degree;
the first calculating subunit is used for performing weighted calculation according to the first confidence coefficient, the second confidence coefficient and the corresponding weight to obtain a comprehensive confidence coefficient;
a second calculating subunit, configured to calculate an absolute difference between the comprehensive confidence level and the third confidence level;
and the determining subunit is used for determining the reliability confidence coefficient according to the comprehensive confidence coefficient when the absolute difference value is greater than a preset threshold value.
In one embodiment, the reliability assessment system further comprises:
the time sequence analysis module is used for carrying out time sequence analysis on the test process information to obtain a test time sequence;
the updating module is used for updating and calculating the first fault density according to the test time sequence;
and the calculating module is used for calculating the fault density according to the updated first fault density, the fault discovery number and the fault discovery rate.
FIG. 4 is a diagram illustrating an internal structure of a computer device in one embodiment. The computer device may specifically be a server including, but not limited to, a high performance computer and a cluster of high performance computers. As shown in fig. 4, the computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program which, when executed by the processor, causes the processor to implement the method of reliability evaluation of a software system. The internal memory may also have a computer program stored therein, which when executed by the processor, causes the processor to perform a method for reliability evaluation of a software system. It will be appreciated by those skilled in the art that the configuration shown in fig. 4 is a block diagram of only a portion of the configuration associated with the present application, and is not intended to limit the computing device to which the present application may be applied, and that a particular computing device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, the reliability evaluation method of the software system provided by the present application may be implemented in the form of a computer program, and the computer program may be run on a computer device as shown in fig. 4. The memory of the computer device may store therein the respective program templates constituting the reliability evaluation system. For example, the obtaining module 301, the analyzing module 302, the fusing module 303, the extracting module 304, and the determining module 305.
A computer device comprising a memory, a processor and a computer program stored in said memory and executable on said processor, said processor implementing the steps in the reliability assessment method of the above software system when executing said computer program.
A computer-readable storage medium, in which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the method for reliability evaluation of a software system as described above.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the program is executed. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), Rambus (Rambus) direct RAM (RDRAM), direct bused dynamic RAM (DRDRAM), and bused dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, and these are all within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (11)
1. A reliability assessment method of a software system is applied to a reliability assessment system, the reliability assessment system comprises an auxiliary monitoring module and is used for monitoring a test process of the software system, and the method comprises the following steps:
acquiring test process information and test packet information of the software system through the auxiliary monitoring module;
carrying out statistical analysis on the test process information according to preset evaluation indexes to obtain test process parameters;
performing fusion processing on the test process parameters and the test packet information to generate auxiliary evaluation parameters;
obtaining test result information from the test packet information;
and calculating the reliability confidence coefficient of the software system according to the auxiliary evaluation parameters and the test result information and a preset reliability evaluation rule.
2. The reliability evaluation method of the software system according to claim 1, wherein the preset evaluation indexes comprise system operation performance, fault severity and failure data, and each evaluation index corresponds to a preset statistical analysis rule; the statistical analysis is performed on the test process information according to the preset evaluation index to obtain test process parameters, and the method comprises the following steps:
respectively extracting parameters and test time point data corresponding to the system operation performance, the fault severity and the failure data from the test process information to obtain a first parameter, a second parameter, a third parameter and test time points corresponding to the first parameter, the second parameter and the third parameter;
according to each preset evaluation index, respectively carrying out calculation analysis on the first parameter, the second parameter and the third parameter according to the corresponding preset statistical analysis rule to obtain a first process parameter, a second process parameter and a third process parameter;
the test process parameters include a first process parameter, a second process parameter, and a third process parameter and the respective corresponding test time points.
3. The method for reliability evaluation of a software system according to claim 2, wherein the auxiliary evaluation parameters include a fault density and a fault intensity, the test packet information includes a fault finding number and a fault finding rate, and the fusing the test process parameters and the test packet information to generate the auxiliary evaluation parameters includes:
analyzing the second process parameter in the test process parameters and the corresponding test time point to obtain first fault intensity and first fault density;
analyzing the first process parameter, the third process parameter and the corresponding test time point in the test process parameters to obtain a second fault intensity and a second fault density;
calculating the fault intensity according to the first fault intensity, the second fault intensity, the fault finding number and the fault finding rate;
and calculating the fault density according to the first fault density, the second fault density, the fault discovery number and the fault discovery rate.
4. The method for reliability evaluation of a software system according to claim 3, wherein the test packet information further includes test data, a plurality of test cases, and a plurality of test results corresponding to the test cases; the obtaining of the test result information from the test packet information includes:
and carrying out statistical analysis on each test result to generate test result information.
5. The method for reliability evaluation of a software system according to claim 3, wherein the calculating the reliability confidence of the software system according to a preset reliability evaluation rule based on the auxiliary evaluation parameter and the test result information comprises:
determining a first confidence corresponding to the fault intensity;
determining a second confidence corresponding to the fault density;
determining a third confidence coefficient corresponding to the test result information;
and calculating the reliability confidence coefficient of the software system according to the first confidence coefficient, the second confidence coefficient and the third confidence coefficient.
6. The method of assessing the reliability of a software system according to claim 5, wherein calculating a reliability confidence of the software system based on the first confidence, the second confidence, and the third confidence comprises:
acquiring weights corresponding to the first confidence coefficient and the second confidence coefficient;
performing weighting calculation according to the first confidence coefficient, the second confidence coefficient and the corresponding weight to obtain a comprehensive confidence coefficient;
calculating an absolute difference between the composite confidence and the third confidence;
and when the absolute difference is larger than a preset threshold value, determining the reliability confidence coefficient according to the comprehensive confidence coefficient.
7. The method for reliability evaluation of a software system according to claim 3, wherein before said calculating said fault density from said first fault density, said second fault density, said fault discovery number and said fault discovery rate, further comprises:
performing time sequence analysis on the test process information to obtain a test time sequence;
updating and calculating the first fault density according to the test time sequence;
and calculating to obtain the fault density according to the updated first fault density, the fault discovery number and the fault discovery rate.
8. The method as claimed in claim 2, wherein the system performance includes at least one of CPU occupancy, usage frequency, and usage fluency 。
9. A reliability evaluation system, comprising an auxiliary monitoring module for monitoring a test process of the software system, comprising:
the acquisition module is used for acquiring the test process information and the test packet information of the software system through the auxiliary monitoring module;
the analysis module is used for carrying out statistical analysis on the test process information according to preset evaluation indexes to obtain test process parameters;
the fusion module is used for fusing the test process parameters and the test packet information to generate auxiliary evaluation parameters;
the extraction module is used for acquiring test result information from the test packet information;
and the determining module is used for calculating the reliability confidence coefficient of the software system according to the auxiliary evaluation parameters and the test result information and a preset reliability evaluation rule.
10. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the reliability assessment method of a software system according to any of claims 1 to 8 when executing the computer program.
11. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method for reliability assessment of a software system according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210731811.3A CN115098370A (en) | 2022-06-25 | 2022-06-25 | Reliability evaluation method, system, device and storage medium for software system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210731811.3A CN115098370A (en) | 2022-06-25 | 2022-06-25 | Reliability evaluation method, system, device and storage medium for software system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115098370A true CN115098370A (en) | 2022-09-23 |
Family
ID=83292437
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210731811.3A Pending CN115098370A (en) | 2022-06-25 | 2022-06-25 | Reliability evaluation method, system, device and storage medium for software system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115098370A (en) |
-
2022
- 2022-06-25 CN CN202210731811.3A patent/CN115098370A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106127363B (en) | User credit assessment method and device | |
| US20190392252A1 (en) | Systems and methods for selecting a forecast model for analyzing time series data | |
| CN112884092B (en) | AI model generation method, electronic device, and storage medium | |
| CN109992473B (en) | Application system monitoring method, device, equipment and storage medium | |
| CN112907128A (en) | Data analysis method, device, equipment and medium based on AB test result | |
| CN110941555A (en) | Test case recommendation method and device, computer equipment and storage medium | |
| CN115656673A (en) | Transformer data processing device and equipment storage medium | |
| US7617313B1 (en) | Metric transport and database load | |
| CN114202256B (en) | Architecture upgrading early warning method and device, intelligent terminal and readable storage medium | |
| CN115841046A (en) | Accelerated degradation test data processing method and device based on wiener process | |
| CN114355094B (en) | Product reliability weak link comprehensive evaluation method and device based on multi-source information | |
| CN111177500A (en) | Data object classification method and device, computer equipment and storage medium | |
| CN113707296B (en) | Medical scheme data processing method, device, equipment and storage medium | |
| CN117196322B (en) | Intelligent wind control method, intelligent wind control device, computer equipment and storage medium | |
| CN113110961B (en) | Equipment abnormality detection method and device, computer equipment and readable storage medium | |
| CN117593115A (en) | Feature value determining method, device, equipment and medium of credit risk assessment model | |
| CN115098370A (en) | Reliability evaluation method, system, device and storage medium for software system | |
| CN113780666B (en) | Missing value prediction method and device and readable storage medium | |
| CN115795928A (en) | Accelerated degradation test data processing method and device based on gamma process | |
| CN115630708A (en) | Model updating method and device, electronic equipment, storage medium and product | |
| CN114885152A (en) | Video service quality evaluation method, device, equipment and storage medium | |
| CN110865939B (en) | Application program quality monitoring method, device, computer equipment and storage medium | |
| CN110795255A (en) | Data batch value adjusting method and device, readable storage medium and equipment | |
| CN113743692A (en) | Business risk assessment method and device, computer equipment and storage medium | |
| CN116127326B (en) | Composite insulator detection method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |