CN110059013B - Method and device for determining normal operation after software upgrading - Google Patents
Method and device for determining normal operation after software upgrading Download PDFInfo
- Publication number
- CN110059013B CN110059013B CN201910335090.2A CN201910335090A CN110059013B CN 110059013 B CN110059013 B CN 110059013B CN 201910335090 A CN201910335090 A CN 201910335090A CN 110059013 B CN110059013 B CN 110059013B
- Authority
- CN
- China
- Prior art keywords
- software
- index
- value
- values
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/362—Software debugging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/362—Software debugging
- G06F11/3644—Software debugging by instrumenting at runtime
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/65—Updates
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Software Systems (AREA)
- Computer Security & Cryptography (AREA)
- Stored Programmes (AREA)
Abstract
The application provides a method and a device for determining normal operation after software upgrading, wherein the method comprises the following steps: and when determining whether the software is normal after upgrading, combining the change metric values of the key indexes before and after the software is upgraded for a plurality of times, and judging whether the change metric values of the key indexes before and after the upgrading of the experimental group software are abnormal. Therefore, by combining the change measurement values of the indexes before and after the software history is updated repeatedly, whether the change of the indexes before and after the update of the software is normal or not is accurately determined, the work of configuring the index threshold is reduced, the recall rate of inspection is improved, the loss caused by abnormal change is reduced, and a certain basis is provided for automatic operation and maintenance.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a method and an apparatus for determining normal operation after software upgrade.
Background
In the related art, some core indexes are manually selected for software, and an abnormality detection threshold is configured for the selected indexes in advance. When the software is changed (i.e., the version of the software is updated), an experimental group (made up of multiple machines) is selected, and then the software of all the machines in the experimental group is upgraded and checked to see if their associated metrics are within a manually configured threshold. If the index is not in the manual configuration range, the software change is considered abnormal, the software change on the rest machines is stopped, the checking effect is completely dependent on the manually selected index and the threshold configured for the index, and because the manually selected index range is usually a small set, a proper threshold is difficult to select for the index, and an excessive threshold can lead to abnormal change not to be found, thus causing large-scale faults. Too severe a threshold will result in frequent erroneous decisions affecting the efficiency of the change.
Disclosure of Invention
The present application aims to solve, at least to some extent, one of the technical problems in the related art.
Therefore, a first object of the present application is to provide a method for determining normal operation after upgrading software.
The second object of the present application is to provide a device for determining normal operation after software upgrade.
A third object of the present application is to provide a device for determining normal operation after a software upgrade.
A fourth object of the present application is to propose a computer readable storage medium.
In order to achieve the above objective, an embodiment of a first aspect of the present application provides a method for determining normal operation after software upgrade, including: acquiring a first variation measurement value of key indexes of an experiment group before and after the software upgrading; acquiring a change metric value set corresponding to the key index, wherein the change metric value set comprises change metric values before and after each upgrade of the software history; determining whether the first change metric value is abnormal according to the first change metric value and the change metric value set; and when the first change measurement value is determined to be normal, determining that the experiment group runs normally after the software is upgraded.
According to the method for determining the normal operation after the software upgrading, when determining whether the software is normal after the software upgrading, the method is combined with the change metric values of the key indexes before and after the software is upgraded for many times, and whether the change metric values of the key indexes before and after the upgrading of the experimental group software are abnormal or not is determined. Therefore, by combining the change measurement values of the indexes before and after the software history is updated repeatedly, whether the change of the indexes before and after the update of the software is normal or not is accurately determined, the work of configuring the index threshold is reduced, the recall rate of inspection is improved, the loss caused by abnormal change is reduced, and a certain basis is provided for automatic operation and maintenance.
To achieve the above object, an embodiment of a second aspect of the present application provides a device for determining normal operation after a software upgrade, including: the first acquisition module is used for acquiring first variation measurement values of key indexes of the experimental group before and after the software upgrading; the second acquisition module is used for acquiring a change measurement value set corresponding to the key index, wherein the change measurement value set comprises change measurement values before and after each upgrade of the software history; the first judging module is used for determining whether the first change metric value is abnormal or not according to the first change metric value and the change metric value set; and the first determining module is used for determining that the experiment group runs normally after the software is upgraded when the first change metric value is determined to be normal.
The device for determining normal operation after software upgrading provided by the embodiment of the application is used for determining whether the software is normal after upgrading, and judging whether the change metric values of the key indexes before and after upgrading the experimental group software this time are abnormal by combining the change metric values of the key indexes before and after upgrading the software many times. Therefore, by combining the change measurement values of the indexes before and after the software history is updated repeatedly, whether the change of the indexes before and after the update of the software is normal or not is accurately determined, the work of configuring the index threshold is reduced, the recall rate of inspection is improved, the loss caused by abnormal change is reduced, and a certain basis is provided for automatic operation and maintenance.
To achieve the above objective, an embodiment of a third aspect of the present application provides a method for determining normal operation after software upgrade, which includes a memory, a processor, and a computer program stored on the memory and capable of running on the processor, wherein the processor implements the method for determining normal operation after software upgrade as described above when executing the program.
In order to achieve the above object, a fourth aspect of the present application proposes a computer-readable storage medium, which when executed by a processor, implements a method for determining normal operation after a software upgrade as described above.
Additional aspects and advantages of the application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the application.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
fig. 1 is a flow chart of a method for determining normal operation after upgrading software according to an embodiment of the present application;
FIG. 2 is a flowchart illustrating another method for determining normal operation after upgrading software according to an embodiment of the present application;
FIG. 3 is a flowchart illustrating another method for determining normal operation after upgrading software according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a determining device for normal operation after software upgrading according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of another determination device for normal operation after software upgrade according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of another determination device that operates normally after software upgrade according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a determination device for normal operation after software upgrade according to an embodiment of the present application.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and intended for the purpose of explaining the present application and are not to be construed as limiting the present application.
When the software releases a new version, repairs the Bug or improves the system performance, the software needs to be online, namely, the change release is undoubtedly the necessary path for software iteration. However, the change always accompanies the risk, and some users of the internet company feel the trouble of great loss, and the trouble is often related to software change. Although the on-line fault quantity can be reduced to a certain extent by improving the development and test quality, the faults caused by changing cannot be completely avoided due to different production environments and test environments and complex calling relations of services.
In order to reduce the influence of abnormal changes on the stability of online service, in the related art, the change is generally divided into at least two stages, wherein the first stage is to change the software version of a designated machine, and the second stage is to determine that the software version is changed normally if the index of the designated machine is normal, and release the software version so that a user can update the software according to the new software version.
In the first stage, in the related art, typically, during the process of running the upgraded version of the specified machine, an index value of a preset performance index is obtained, and it is determined whether the index value is within a manually configured threshold range. If the existence index is not in the manual configuration range, the software change is considered abnormal. However, the manually set index range is prone to inaccuracy, resulting in abnormal changes not being found in time, leading to large-scale failure.
Therefore, the embodiment of the application provides a method for determining normal operation after software upgrading, which combines the change metric values of key indexes before and after the software upgrading for a plurality of times to judge whether the change metric values of the key indexes before and after the software upgrading of an experimental group are abnormal or not, that is, determine whether the change of the key indexes after the software upgrading of the experimental group accords with the change rule of the key indexes before and after the software upgrading for a plurality of times, if the key indexes before and after the software upgrading of the experimental group are normal, that is, determine that the key indexes cannot be caused after the software upgrading of the time, that is, the software upgrading of the time cannot cause risk on-line stability. Therefore, by combining the change measurement values of the indexes before and after the software history is updated repeatedly, whether the change of the indexes before and after the update of the software is normal or not is accurately determined, the work of configuring the index threshold is reduced, the recall rate of inspection is improved, the loss caused by abnormal change is reduced, and a certain basis is provided for automatic operation and maintenance.
The following describes a method and an apparatus for determining normal operation after software upgrade in the embodiment of the present application with reference to the accompanying drawings.
Fig. 1 is a flow chart of a method for determining normal operation after upgrading software according to an embodiment of the present application. It should be noted that, the executing subject of the method for determining normal operation after software upgrade in the embodiment of the present application is a device for determining normal operation after software upgrade, and the device for determining normal operation after software upgrade may be located in an electronic device or a server, which is not limited in this embodiment.
As shown in fig. 1, the method for determining normal operation after upgrading software may include:
The experimental group is composed of a designated machine, which is the machine that performs the current upgrade on the software.
The number of machines specified in the experimental set may be one or more.
The first change measurement value refers to measuring the change condition of key indexes representing the running state of the software in the experimental group before and after the software is updated.
The key index refers to an index representing the running state of software. And whether the running state of the software is normal or not can be determined through the key index.
The key metrics may include resource metrics (e.g., how much CPU resources are occupied, how much memory resources are occupied), traffic metrics (response time metrics, request processing metrics, request unresponsiveness metrics, etc.), error log metrics, etc.
It can be understood that, because load changes occur in the machine during the software changing process, a single machine is used to detect the change of the key index, which may cause erroneous judgment, so in this embodiment, in order to improve the accuracy of judgment, the experiment group is composed of a plurality of machines, and the index values of the key indexes of the plurality of machines before and after the software is upgraded are combined to determine the change metric value of the key index of the experiment group before and after the software is upgraded. Namely, index change values of key indexes of all machines in the experimental group before and after the software upgrading are measured to obtain a first change measurement value.
The method for determining whether the key index is normal before and after the software upgrading according to the first change metric value and the change metric value set is as follows:
in this example, as shown in fig. 2, step 103 may include:
step 1031, calculating the average value and standard variance value of the key indexes according to the variation metric values before and after each upgrade of the software history.
Step 1032, calculating a statistical Z value based on the average value, the standard deviation value, and the first variation metric value.
Step 1032, determining whether the key index is normal before and after the software upgrading according to the statistical Z value and the difference significance relation table.
In the present embodiment, it is assumed that the change metric value set T, t= { T, of the key targets before and after each change of the history 1 ，t 2 ，...，t n }。
From the set of variation metric values T, the historical variation (T) is calculated to calculate the average of the variation metrics
To be used for
Standard deviation sigma of variation measure T The larger the Z-check formula is, the more this change is indicated as shown in the following formula (3)
The smaller the probability of change, i.e., the change of the current change cannot be interpreted by the history change.
Wherein t in formula (3) represents a first variation metric value.
And determining whether the statistical Z value is significant according to the statistical Z value and the difference significance relation table, and if the statistical Z value is not significant, normalizing the key index before and after the software upgrading.
For example, the statistical Z value is 1.8, and the statistical Z value is in the range of the theoretical Z value with insignificant difference by comparing with the theoretical Z value in the difference significance relation table, and at this time, it is determined that the key index is normal before and after the software upgrading.
In another example, a normal range of the change metric of the key indicator may be determined according to the set of change metric values, and the first change metric value is compared to determine whether the first change metric value is within the normal range of the change metric, if so, it is determined that the key indicator is normal before and after the software is upgraded, and if not, it is determined that the key indicator is abnormal before and after the software is upgraded, that is, the software is upgraded to cause the change abnormality of the key indicator.
For example, according to a set of change measurement values before and after the software is updated repeatedly, determining a maximum change measurement value and a minimum change measurement value in the set of change measurement values, and obtaining a normal range of the change measurement according to the maximum change measurement value and the minimum change measurement value, if the change measurement value of the key index is between the minimum change measurement value and the maximum change measurement value before and after the software is updated, determining that the key index is normal before and after the software is updated, otherwise, determining that the key index is abnormal before and after the software is updated, namely, the software is updated this time to cause the change abnormality of the key index.
And 104, when the first change measurement value is determined to be normal, determining that the experiment group runs normally after the software is upgraded.
In the method for determining normal operation after upgrading software disclosed in this embodiment, when determining whether the software is normal after upgrading, the method is combined with the change metric values of the key indexes before and after upgrading the software for many times to determine whether the change metric values of the key indexes before and after upgrading the software of the experimental group are abnormal. Therefore, by combining the change measurement values of the indexes before and after the software history is updated repeatedly, whether the change of the indexes before and after the update of the software is normal or not is accurately determined, the work of configuring the index threshold is reduced, the recall rate of inspection is improved, the loss caused by abnormal change is reduced, and a certain basis is provided for automatic operation and maintenance.
It should be understood that the judging process of each key index is the same as that of the above embodiment, and when it is determined that an abnormal key index exists after the software is updated, it is determined that the abnormal key index is caused after the software is updated, and the software needs to be adjusted again after the software is stopped.
Based on the above embodiment, after the software version of the experimental group is upgraded, operation or hot events may be encountered, and these events may change the key index to a certain extent. Because these factors are applied to the unchanged machine (i.e. the comparison group) at the same time, in order to avoid misjudgment caused by index change caused by user behavior change, in this embodiment, when determining that the first change metric value is abnormal according to the first change metric value and the change metric value set, further determining whether the key index is abnormal according to the index information of the comparison group is further combined, that is, when determining that the change of the key index before and after the current upgrade of the software cannot be interpreted by the change before and after the multiple upgrades of the software history, further determining whether the change of the key index in the experimental group can be interpreted by the comparison group.
Thus, on the basis of the embodiment shown in fig. 1, as shown in fig. 2, the method may further comprise:
The prediction model is obtained based on time series fitting of index information of a comparison group, the index information comprises historical index values of key indexes, and the comparison group is operated in a version before the software is upgraded.
Specifically, according to the prediction model, an index prediction value of the key index of each machine in the experimental group in a third preset time period after the software is upgraded can be predicted.
If it is determined that the degree of difference is not significant, it is determined that the first measure of variation is normal, step 204.
In step 205, if it is determined that the degree of difference is significant, it is determined that the first variation is abnormal.
Specifically, after the difference is calculated, a probability value corresponding to the difference can be determined according to the t-bounded value table, if the probability value is smaller than the first probability value, the first variation is determined to be normal, and otherwise, the first variation is determined to be abnormal.
The first probability value is a probability value corresponding to the difference obviously.
In this embodiment, the difference degrees between the predicted values of the multiple indexes and the actual indexes of the key indexes are determined by T test, if the difference is not significant, the change of the key indexes is considered to be interpreted by the control group, otherwise, the change of the key indexes is considered to be not interpreted by the control group.
When it is determined that the change of the key indicator cannot be interpreted by the change metrics before and after the historical multiple upgrades and cannot be interpreted by the control group, it is determined that the key indicator is abnormal due to the current upgrade of the software.
When the abnormality of the key index caused by the current upgrade of the software is determined, prompt information can be output, so that maintenance personnel can determine that the abnormality exists in the software change according to the prompt information.
According to the embodiment of the application, after the change measurement set of the key indexes is combined before and after the software is upgraded for many times, the change measurement value of the key indexes before and after the software is upgraded is abnormal, the index information of the comparison group is combined, the predicted value of the key indexes of the experiment group in the preset time period after the upgrading is predicted, whether the first change value is abnormal is further determined according to the actual index value and the predicted value of the key indexes, and when the first change value is determined to be abnormal according to the actual index value and the predicted value of the key indexes, the key index abnormality caused by the software upgrading is determined. Therefore, by combining the historical software change and index information of the control group, whether key indexes before and after the software upgrading are abnormal or not is further accurately determined, the judging accuracy is further improved, the work of configuring index thresholds is reduced, the recall rate of inspection is improved, the loss caused by abnormal change is reduced, and a certain basis is provided for automatic operation and maintenance.
Fig. 3 is a flow chart of a method for determining normal operation after upgrading software according to an embodiment of the present application.
As shown in fig. 3, the method for determining normal operation after upgrading software may include:
The first actual index value sequence comprises a plurality of first actual index values which are ordered according to the acquisition time.
The second actual index value sequence comprises a plurality of second actual index values which are ordered according to the acquisition time.
Because in the software changing process, the machine will have load change, and the change of the key index is detected by one machine alone, which may cause misjudgment, in this embodiment, the experiment group is composed of a plurality of machines, the first actual index value corresponding to the corresponding collection time is the average value of the indexes of all machines in the experiment group at the corresponding collection time, and the second actual index value corresponding to the corresponding collection time is the average value of the indexes of all machines in the experiment group at the corresponding collection time. Therefore, the occurrence of misjudgment caused by the change of key indexes due to the change of the load of a single machine can be reduced, and the software is further improved.
In this embodiment, according to the first actual index value sequence and the second actual index value sequence, determining the first variation metric value of the key index of the experiment group before and after the software upgrade can be implemented by the following manner: calculating a first average value and a first variance of key indexes before the software is upgraded according to the first actual index value sequence; calculating a second average value and a second variance of the key index after the software is upgraded according to the second actual index value sequence; and determining a first variation metric value of the key indexes of the experimental group before and after the software upgrading according to the first average value, the first variance, the second average value and the second variance.
In an embodiment, the change of the key indicator before and after the change is measured according to the T test, and the obtained test statistic T is the first change measurement value of the key indicator.
Wherein, the first change metric value t of the key index is calculated as shown in the following formula (1):
wherein X, Y is the index value of the key index before and after the software upgrade; n is n 1 、n 2 The number of sample points, i.e. n, of X, Y respectively 1 Indicating the index value number of key index before the current upgrade of the software, n 1 The index value number of the key index before the software is upgraded;
the variance of X, Y, respectively. s is(s) p Represents the standard deviation of the paired sample differences.
That is, it is determined that the current upgrade of the software causes an index abnormality of the key index.
Step 310, determining that the experiment group runs normally after the software is upgraded.
According to the embodiment of the application, after the change measurement set of the key indexes is combined before and after the software is upgraded for many times, the change measurement value of the key indexes before and after the software is upgraded is abnormal, the index information of the comparison group is combined, the predicted value of the key indexes of the experiment group in the preset time period after the upgrading is predicted, whether the first change value is abnormal is further determined according to the actual index value and the predicted value of the key indexes, and when the first change value is determined to be abnormal according to the actual index value and the predicted value of the key indexes, the key index abnormality caused by the software upgrading is determined. Therefore, by combining the historical software change and index information of the control group, whether key indexes before and after the software upgrading are abnormal or not is further accurately determined, the judging accuracy is further improved, the work of configuring index thresholds is reduced, the recall rate of inspection is improved, the loss caused by abnormal change is reduced, and a certain basis is provided for automatic operation and maintenance.
Fig. 4 is a schematic structural diagram of a determination device for normal operation after software upgrade according to an embodiment of the present application.
As shown in fig. 4, the device for determining normal operation after upgrading software includes a first obtaining module 110, a second obtaining module 120, a first judging module 130 and a first determining module 140, where:
the first obtaining module 110 is configured to obtain a first variation metric value of key indicators of the experiment group before and after the software upgrade.
The second obtaining module 120 is configured to obtain a set of change metric values corresponding to the key indicators, where the set of change metric values includes change metric values before and after each upgrade of the software history.
The first determining module 130 is configured to determine whether the first variation metric value is abnormal according to the first variation metric value and the variation metric value set.
And the first determining module 140 is configured to determine that the experimental group operates normally after the software is upgraded when the first variation metric value is determined to be normal.
In one embodiment of the present application, when determining that the first change metric value is abnormal according to the first change metric value and the change metric value set, as shown in fig. 5, the apparatus further includes:
the second determining module 150 is configured to determine, according to the prediction model, a plurality of index prediction values of the key index of the experimental group in a third preset time period after the software is upgraded.
The prediction model is obtained based on time series fitting of index information of a comparison group, the index information comprises historical index values of key indexes, and the comparison group is operated in a version before the software is upgraded.
A third obtaining module 160, configured to obtain a plurality of actual index values of the key index of the experiment group in a third preset time period after the software is upgraded.
The second judging module 170 is configured to determine whether the first variation metric value is abnormal according to the difference degrees of the plurality of index prediction values and the plurality of actual index values of the key index.
In one embodiment of the present application, the first determining module 130 is specifically configured to: and calculating the average value and the standard variance value of the key indexes according to the change metric values before and after each upgrade of the software history. And calculating a statistical Z value according to the average value, the standard deviation value and the first variation measurement value. And determining whether the key index is normal before and after the software is upgraded according to the statistical Z value and the difference significance relation table.
In one implementation of the present application, based on the embodiment of the apparatus shown in fig. 4, as shown in fig. 6, the apparatus further includes:
the fourth obtaining module 180 is configured to obtain a first actual index value sequence of the key index of the experiment group in a first preset time period before the software is upgraded.
A fifth obtaining module 190, configured to obtain a second actual index value sequence of the key index of the experiment group in a second preset time period after the software is upgraded.
And the third determining module 200 is configured to determine, according to the first actual index value sequence and the second actual index value sequence, a first variation metric value of the key index of the experiment group before and after the software is upgraded.
It should be noted that, the structures of the fourth acquiring module 180, the fifth acquiring module 190 and the third determining module 200 in the apparatus embodiment shown in fig. 6 may also be included in the apparatus embodiment shown in fig. 5, and the implementation is not limited thereto.
In one embodiment of the present application, the third determining module 200 is specifically configured to: and calculating a first average value and a first variance of the key indexes before the software is upgraded according to the first actual index value sequence. And calculating a second average value and a second variance of the key index after the software is upgraded according to the second actual index value sequence. And determining a first variation metric value of the key indexes of the experimental group before and after the software upgrading according to the first average value, the first variance, the second average value and the second variance.
The first actual index value sequence comprises a plurality of first actual index values which are ordered according to the acquisition time, the experiment group consists of a plurality of machines, the first actual index values corresponding to the corresponding acquisition time are index average values of all machines in the experiment group at the corresponding acquisition time, the second actual index value sequence comprises a plurality of second actual index values which are ordered according to the acquisition time, and the second actual index values corresponding to the corresponding acquisition time are index average values of all machines in the experiment group at the corresponding acquisition time.
It should be noted that the explanation of the foregoing embodiment of the method for determining normal operation after software upgrade is also applicable to the device for determining normal operation after software upgrade of this embodiment, and will not be repeated here.
The device for determining normal operation after upgrading software disclosed in this embodiment determines whether the key index change metric values before and after upgrading the experimental group software this time are abnormal by combining the key index change metric values before and after upgrading the software many times when determining whether the software is normal after upgrading. Therefore, by combining the change measurement values of the indexes before and after the software history is updated repeatedly, whether the change of the indexes before and after the update of the software is normal or not is accurately determined, the work of configuring the index threshold is reduced, the recall rate of inspection is improved, the loss caused by abnormal change is reduced, and a certain basis is provided for automatic operation and maintenance.
Fig. 7 is a schematic structural diagram of a determination device for normal operation after software upgrade according to an embodiment of the present application.
As shown in fig. 7, the device for determining normal operation after upgrading software includes:
The processor 1002 implements the method for determining normal operation after the software upgrade provided in the above embodiment when executing the program.
Further, the method for determining normal operation after the software is upgraded further comprises the following steps:
a communication interface 1003 for communication between the memory 1001 and the processor 1002.
The processor 1002 is configured to implement the method for determining normal operation after the software upgrade in the above embodiment when executing the program.
If the memory 1001, the processor 1002, and the communication interface 1003 are implemented independently, the communication interface 1003, the memory 1001, and the processor 1002 may be connected to each other through a bus and perform communication with each other. The bus may be an industry standard architecture (Industry Standard Architecture, abbreviated ISA) bus, an external device interconnect (Peripheral Component, abbreviated PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated EISA) bus, among others. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, only one thick line is shown in fig. 7, but not only one bus or one type of bus.
Alternatively, in a specific implementation, if the memory 1001, the processor 1002, and the communication interface 1003 are integrated on a chip, the memory 1001, the processor 1002, and the communication interface 1003 may complete communication with each other through internal interfaces.
The processor 1002 may be a central processing unit (Central Processing Unit, abbreviated as CPU), or an application specific integrated circuit (Application Specific Integrated Circuit, abbreviated as ASIC), or one or more integrated circuits configured to implement embodiments of the present application.
The present embodiment also provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the above method for determining normal operation after a software upgrade.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present application, the meaning of "plurality" is at least two, such as two, three, etc., unless explicitly defined otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and additional implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present application.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
It is to be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. As with the other embodiments, if implemented in hardware, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
Those of ordinary skill in the art will appreciate that all or a portion of the steps carried out in the method of the above-described embodiments may be implemented by a program to instruct related hardware, where the program may be stored in a computer readable storage medium, and where the program, when executed, includes one or a combination of the steps of the method embodiments.
In addition, each functional unit in each embodiment of the present application may be integrated in one processing module, or each unit may exist alone physically, or two or more units may be integrated in one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules may also be stored in a computer readable storage medium if implemented in the form of software functional modules and sold or used as a stand-alone product.
The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, or the like. Although embodiments of the present application have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the application, and that variations, modifications, alternatives, and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the application.
Claims (6)
1. The method for determining normal operation after software upgrading is characterized by being applied to a first stage, wherein the first stage is to change a software version of a specified experiment group, and the experiment group comprises a plurality of machines and comprises the following steps:
acquiring a first change metric value of a key index of the experimental group representing the running state of software before and after the software upgrading according to a first actual index value sequence of the experimental group before the software upgrading and a second actual index value sequence of the experimental group after the software upgrading, wherein the first actual index value sequence comprises a plurality of first actual index values sequenced according to the acquisition time, the first actual index values are index average values of all machines in the experimental group at corresponding acquisition time, the second actual index value sequence comprises a plurality of second actual index values sequenced according to the acquisition time, and the second actual index values are index average values of all machines in the experimental group at corresponding acquisition time;
acquiring a change metric value set corresponding to the key index, wherein the change metric value set comprises change metric values before and after each upgrade of the software history;
calculating an average value and a standard variance value of the key index according to the variation metric value, calculating a statistical Z value according to the average value, the standard variance value and the first variation metric value, and determining whether the first variation metric value is abnormal according to the statistical Z value and a difference significance relation table;
when the first variation measurement value is determined to be normal according to the statistical Z value and the difference significance relation table, determining that the experiment group runs normally after the software is upgraded;
when the first variation metric value is determined to be abnormal according to the statistical Z value and the difference significance relation table, determining a plurality of index prediction values of the key index of the experiment group in a third preset time period after the software is upgraded according to a prediction model, obtaining a plurality of actual index values of the key index of the experiment group in the third preset time period after the software is upgraded, determining whether the first variation metric value is abnormal according to the plurality of index prediction values of the key index and the difference degrees of the plurality of actual index values, and if the difference degrees are determined to be not significant, determining that the experiment group operates normally after the software is upgraded, wherein the prediction model is obtained based on time sequence fitting of index information of a comparison group, the index information comprises a history index value of the key index, and the comparison group operates in a version before the software is upgraded.
2. The method of claim 1, wherein determining the first measure of variation of the key indicator of the experimental set before and after the current upgrade of the software based on the first actual indicator value sequence and the second actual indicator value sequence comprises:
calculating a first average value and a first variance of the key index before the software is upgraded according to the first actual index value sequence;
calculating a second average value and a second variance of the key index after the software is upgraded according to the second actual index value sequence;
and determining a first variation measurement value of the key index of the experimental group before and after the software upgrading according to the first average value, the first variance, the second average value and the second variance.
3. A device for determining normal operation after a software upgrade, the device being applied to a first stage, the first stage being a software version change performed on a specified experiment group, the experiment group including a plurality of machines, comprising:
the first acquisition module is used for acquiring a first variation metric value of a key index of the experimental group representing the running state of the software before and after the software is updated according to a first actual index value sequence of the experimental group before the software is updated and a second actual index value sequence of the experimental group after the software is updated, wherein the first actual index value sequence comprises a plurality of first actual index values which are ordered according to the acquisition time, the first actual index values are index average values of all machines in the experimental group at corresponding acquisition time, the second actual index value sequence comprises a plurality of second actual index values which are ordered according to the acquisition time, and the second actual index values are index average values of all machines in the experimental group at corresponding acquisition time;
the second acquisition module is used for acquiring a change measurement value set corresponding to the key index, wherein the change measurement value set comprises change measurement values before and after each upgrade of the software history;
the first judging module is used for calculating an average value and a standard variance value of the key index according to the variation metric value, calculating a statistical Z value according to the average value, the standard variance value and the first variation metric value, and determining whether the first variation metric value is abnormal according to the statistical Z value and a difference significance relation table;
the first determining module is used for determining that the experiment group runs normally after the software is upgraded when the first variation metric value is determined to be normal according to the statistical Z value and the difference significance relation table;
and the second determining module is used for determining a plurality of index prediction values of the key index of the experimental group in a third preset time period after the software is upgraded according to a prediction model when the first variation measurement value is abnormal according to the statistical Z value and the difference significance relation table, obtaining a plurality of actual index values of the key index of the experimental group in the third preset time period after the software is upgraded, determining whether the first variation measurement value is abnormal according to the plurality of index prediction values of the key index and the difference degrees of the plurality of actual index values, and determining that the experimental group runs normally after the software is upgraded if the difference degrees are not significant, wherein the prediction model is obtained by fitting based on a time sequence of index information of a comparison group, the index information comprises a historical index value of the key index, and the comparison group runs in a version before the software is upgraded.
4. The apparatus of claim 3, wherein determining a first measure of variation of the key indicator for the experimental set before and after the current upgrade of software based on the first sequence of actual indicator values and the second sequence of actual indicator values comprises:
calculating a first average value and a first variance of the key index before the software is upgraded according to the first actual index value sequence;
calculating a second average value and a second variance of the key index after the software is upgraded according to the second actual index value sequence;
and determining a first variation measurement value of the key index of the experimental group before and after the software upgrading according to the first average value, the first variance, the second average value and the second variance.
5. A device for determining normal operation after a software upgrade, comprising:
memory, processor and computer program stored in the memory and executable on the processor, wherein the processor implements the method for determining the normal operation after a software upgrade according to claim 1 or 2 when executing the program.
6. A computer-readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements a method for determining a normal operation after a software upgrade as claimed in claim 1 or 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910335090.2A CN110059013B (en) | 2019-04-24 | 2019-04-24 | Method and device for determining normal operation after software upgrading |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910335090.2A CN110059013B (en) | 2019-04-24 | 2019-04-24 | Method and device for determining normal operation after software upgrading |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110059013A CN110059013A (en) | 2019-07-26 |
| CN110059013B true CN110059013B (en) | 2023-06-23 |
Family
ID=67320519
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910335090.2A Active CN110059013B (en) | 2019-04-24 | 2019-04-24 | Method and device for determining normal operation after software upgrading |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110059013B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115599657B (en) * | 2022-12-15 | 2023-03-17 | 浪潮通信信息系统有限公司 | Software facility abnormity judgment method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108595669A (en) * | 2018-04-28 | 2018-09-28 | 国信优易数据有限公司 | A kind of unordered classified variable processing method and processing device |
| CN109492394A (en) * | 2018-10-25 | 2019-03-19 | 平安科技（深圳）有限公司 | The recognition methods of abnormal traffic request and terminal device |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9213728B2 (en) * | 2011-12-14 | 2015-12-15 | Sap Se | Change data capturing during an upgrade |
| EP2879074A1 (en) * | 2013-11-29 | 2015-06-03 | Gemalto SA | Method for loading a native code on a secure element |
| CN103945442A (en) * | 2014-05-07 | 2014-07-23 | 东南大学 | System anomaly detection method based on linear prediction principle in mobile communication system |
| CN106980905B (en) * | 2017-03-15 | 2020-07-07 | 南方电网科学研究院有限责任公司 | Power distribution network power supply reliability prediction method and system |
| CN109241482A (en) * | 2018-08-28 | 2019-01-18 | 优视科技新加坡有限公司 | Determine that altering event issues successful method and device thereof |
| CN109558295B (en) * | 2018-11-15 | 2022-05-24 | 新华三信息安全技术有限公司 | Performance index abnormality detection method and device |
-
2019
- 2019-04-24 CN CN201910335090.2A patent/CN110059013B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108595669A (en) * | 2018-04-28 | 2018-09-28 | 国信优易数据有限公司 | A kind of unordered classified variable processing method and processing device |
| CN109492394A (en) * | 2018-10-25 | 2019-03-19 | 平安科技（深圳）有限公司 | The recognition methods of abnormal traffic request and terminal device |
Non-Patent Citations (2)
| Title |
|---|
| How to use machine learning for anomaly detection and condition monitoring;Vegard Flovik;https://towardsdatascience.com/how-to-use-machine-learning-for-anomaly-detection-and-condition-monitoring-6742f82900d7;全文 * |
| 当我们在做数据异常分析时，我们在分析什么;时之沙;https://www.woshipm.com/data-analysis/999413.html;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110059013A (en) | 2019-07-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108923952B (en) | Fault diagnosis method, equipment and storage medium based on service monitoring index | |
| CN108388489B (en) | Server fault diagnosis method, system, equipment and storage medium | |
| CN112148887A (en) | Equipment fault diagnosis method and device, storage medium and electronic equipment | |
| US20130173215A1 (en) | Adaptive trend-change detection and function fitting system and method | |
| CN110059013B (en) | Method and device for determining normal operation after software upgrading | |
| CN112734138A (en) | Fault early warning method, device, equipment and storage medium | |
| CN108763005B (en) | Memory ECC fault error reporting method and system | |
| CN113468009B (en) | Pressure testing method and device, electronic equipment and storage medium | |
| CN110874311A (en) | Database detection method and device, computer equipment and storage medium | |
| CN114793132A (en) | Optical module detection method and device, electronic equipment and storage medium | |
| CN112053009B (en) | Fault prediction method, device, system and storage medium | |
| CN1893339B (en) | Continuous median failure control system and method | |
| CN114625106B (en) | Method, device, electronic equipment and storage medium for vehicle diagnosis | |
| CN109947639B (en) | Automatic test method for ESB interface | |
| CN116820821A (en) | Disk failure detection method, apparatus, electronic device and computer readable storage medium | |
| KR20170140753A (en) | Method and apparatus for analyzing safety of software | |
| CN116302736A (en) | Method and device for testing components of server, electronic equipment and storage medium | |
| KR101675986B1 (en) | Method and apparatus for analyzing safety of automotive software | |
| CN113360389A (en) | Performance test method, device, equipment and storage medium | |
| CN111176931A (en) | Operation monitoring method, operation monitoring device, server and storage medium | |
| JP2012145987A (en) | Method for indicating software performance problem | |
| CN117469152B (en) | Fluid pump abnormality detection method, fluid pump abnormality detection device, electronic device, and storage medium | |
| CN114253846B (en) | Automatic test abnormality positioning method, device, equipment and readable storage medium | |
| CN115391232B (en) | Program data stream diagnosis method, device and equipment | |
| CN117910148B (en) | BMS optimization method, system, medium and device based on shadow mode |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |