CN114048150A - Memory recovery abnormity detection method, device, equipment and medium - Google Patents

Memory recovery abnormity detection method, device, equipment and medium Download PDF

Info

Publication number
CN114048150A
CN114048150A CN202111383609.8A CN202111383609A CN114048150A CN 114048150 A CN114048150 A CN 114048150A CN 202111383609 A CN202111383609 A CN 202111383609A CN 114048150 A CN114048150 A CN 114048150A
Authority
CN
China
Prior art keywords
data
time sequence
characteristic
characteristic data
memory
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111383609.8A
Other languages
Chinese (zh)
Inventor
简拥军
金勇�
雷发林
吴泽君
高阳
张超杰
周明宏
梁晓冬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
CCB Finetech Co Ltd
Original Assignee
CCB Finetech Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by CCB Finetech Co Ltd filed Critical CCB Finetech Co Ltd
Priority to CN202111383609.8A priority Critical patent/CN114048150A/en
Publication of CN114048150A publication Critical patent/CN114048150A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023—Free address space management
    • G06F12/0253—Garbage collection, i.e. reclamation of unreferenced memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/006—Identification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3452—Performance evaluation by statistical analysis

Abstract

The present disclosure provides a memory recovery anomaly detection method, device, apparatus, medium, and program product, which can be applied to the technical field of big data, and also can be applied to the financial field. The memory recovery abnormity detection method comprises the following steps: acquiring first time sequence characteristic data and second time sequence characteristic data, wherein the first time sequence characteristic data comprises memory space occupation data, duration data consumed by recovery data and frequency data of the first type of data recovered; the second time series characteristic data comprises frequency data of the second type data which is recycled; inputting the first time sequence characteristic data and the second time sequence characteristic data into a preset model, and outputting a first correlation characteristic of the first time sequence characteristic data and the second time sequence characteristic data; calculating a first change trend of the first time-sequence characteristic data; and outputting early warning information for representing that the memory recovery is abnormal under the condition that the first change trend of the first time sequence characteristic data accords with the first correlation characteristic.

Description

Memory recovery abnormity detection method, device, equipment and medium
Technical Field
The present disclosure relates to the field of big data technologies, and in particular, to a method, an apparatus, a device, a medium, and a program product for detecting memory recovery anomalies.
Background
The memory recovery generally means that when a process/thread of a program uses a memory, the memory is returned to a system immediately after the memory use is finished, so that the memory is shared and recycled. The automatic memory recovery is that the system is responsible for managing the memory recovery, and the application program only needs to consider the memory application and does not need to consider the release. The method can reduce the complexity of codes and reduce memory leakage caused by untimely memory recovery or program operation abnormity caused by repeated memory recovery. However, the automatic memory recycling also causes additional problems: 1) the running of the program is often suspended during the recovery period, which results in increased delay of processing requests and influences user experience 2). frequent garbage recovery (frequent GC for short) greatly consumes system resources, which causes program performance deterioration, even causes program collapse and seriously influences service stability.
In the related art, a fixed threshold is mainly adopted to detect whether frequent garbage collection occurs. However, this method can only be detected after frequent garbage collection occurs, and the frequent garbage collection affects the service. Therefore, the memory recovery exception needs to be warned before the business is affected.
Disclosure of Invention
In view of the foregoing, the present disclosure provides a memory recovery abnormality detection method, apparatus, device, medium, and program product.
According to a first aspect of the present disclosure, a method for detecting memory recovery exception is provided, including:
acquiring first time sequence characteristic data and second time sequence characteristic data, wherein the first time sequence characteristic data comprises any one of the following data: memory space occupation data, time-length data consumed by recovery data and frequency data of the first type of data recovered; the second time series characteristic data comprises frequency data of the second type data which is recycled; wherein the frequency with which the first type of data is recycled is greater than the frequency with which the second type of data is recycled;
inputting the first time sequence characteristic data and the second time sequence characteristic data into a preset model, and outputting a first correlation characteristic of the first time sequence characteristic data and the second time sequence characteristic data;
calculating a first change trend of the first time-sequence characteristic data; and
and outputting early warning information for representing that the memory recovery is abnormal under the condition that the first change trend of the first time sequence characteristic data accords with the first correlation characteristic.
According to an embodiment of the present disclosure, the memory space occupation data includes any one of:
the memory space occupation amount before the data are recovered, the memory space occupation amount after the data are recovered, and the memory space occupation amount after the data are recovered; the recycling of the duration data spent on the data comprises recycling the duration data spent on the first type of data and/or recycling the duration data spent on the second type of data.
According to the embodiment of the disclosure, under the condition that the first change trend of the first timing characteristic data conforms to the first correlation characteristic, outputting the early warning information for representing that the memory recovery is abnormal, includes:
and under the condition that the first correlation characteristic is positive correlation and the first change trend of the first time sequence characteristic data is an ascending trend, the first change trend of the first time sequence characteristic data conforms to the first correlation characteristic, and early warning information for representing that the memory recovery is abnormal is output.
According to the embodiment of the disclosure, under the condition that the first change trend of the first timing characteristic data conforms to the first correlation characteristic, outputting the early warning information for representing that the memory recovery is abnormal, includes:
and under the condition that the first correlation characteristic is negative correlation and the first change trend of the first time sequence characteristic data is a descending trend, the first change trend of the first time sequence characteristic data conforms to the first correlation characteristic, and early warning information for representing that the memory recovery is abnormal is output.
According to an embodiment of the present disclosure, the memory recovery abnormality detection method further includes:
extracting first data from the memory recovery log according to the first sliding window;
extracting second data from the memory recovery log according to a second sliding window, wherein the data types of the first data and the second data are the same;
respectively calculating characteristic values of the first data and the second data to obtain first characteristic data and second characteristic data;
and marking the preset time of the first sliding window and the preset time of the second sliding window on the first characteristic data and the second characteristic data respectively to obtain first time sequence characteristic data.
According to an embodiment of the present disclosure, the preset time includes any one of: start time, end time.
According to an embodiment of the present disclosure, the feature value includes any one of: mean, maximum, minimum.
According to an embodiment of the present disclosure, the method for detecting memory exception recovery further includes:
extracting third time sequence feature data and fourth time sequence feature data from the historical abnormal memory log, wherein the fourth time sequence feature data comprise data with abnormal frequency of the recovered second type of data, and the third time sequence feature data and the first time sequence feature data are the same in data type;
analyzing a second variation trend of the third time sequence characteristic data within a preset time before the fourth time sequence characteristic data appears;
determining a second correlation characteristic of the third time sequence characteristic data and the fourth time sequence characteristic data according to the second variation trend;
and determining condition information for triggering the memory recovery to generate abnormity according to the second correlation characteristic and the second change trend.
A second aspect of the present disclosure provides a memory recovery abnormality detection apparatus, including: the device comprises an acquisition module, a detection module, a calculation module and an early warning module. The acquiring module is configured to acquire first time sequence feature data and second time sequence feature data, where the first time sequence feature data includes any one of: memory space occupation data, duration data consumed by data recovery, and frequency data obtained by recovering the first type of data; the second time series characteristic data includes frequency data in which the second type of data is recycled, and the frequency with which the first type of data is recycled is greater than the frequency with which the second type of data is recycled. The detection module is used for inputting the first time sequence characteristic data and the second time sequence characteristic data into a preset model and outputting a first correlation characteristic of the first time sequence characteristic data and the second time sequence characteristic data. And the calculation module is used for calculating the first change trend of the first time sequence characteristic data. And the early warning module is used for outputting early warning information for representing that the memory recovery is abnormal under the condition that the first change trend of the first time sequence characteristic data accords with the first correlation characteristic.
According to an embodiment of the present disclosure, the early warning module includes a first early warning unit and a second early warning unit. And the first early warning unit is used for outputting early warning information used for representing that the memory recovery is abnormal, wherein the change trend of the first time sequence characteristic data is in accordance with the correlation characteristic under the condition that the correlation characteristic is positive correlation and the change trend of the first time sequence characteristic data is an ascending trend. And the second early warning unit is used for outputting early warning information used for representing that the memory recovery is abnormal, wherein the change trend of the first time sequence characteristic data accords with the correlation characteristic under the condition that the correlation characteristic is negative correlation and the change trend of the first time sequence characteristic data is a descending trend.
According to an embodiment of the present disclosure, an acquisition module includes a first extraction unit, a second extraction unit, a calculation unit, and a labeling unit. The first extraction unit is used for extracting first data from the memory recovery log according to the first sliding window. And the second extraction unit is used for extracting second data from the memory recovery log according to a second sliding window, wherein the data types of the first data and the second data are the same. And the calculating unit is used for calculating the characteristic values of the first data and the second data respectively to obtain first characteristic data and second characteristic data. And the marking unit is used for respectively marking the preset time of the first sliding window and the preset time of the second sliding window on the first characteristic data and the second characteristic data to obtain first time sequence characteristic data.
According to the embodiment of the disclosure, the memory recovery abnormity detection device further comprises an extraction module, an analysis module, a first determination module and a second determination module. The extraction module is used for extracting third time sequence feature data and fourth time sequence feature data from the historical abnormal memory log, wherein the fourth time sequence feature data comprise data with abnormal frequency of the second type of data, and the third time sequence feature data are the same as the first time sequence feature data in data type. And the analysis module is used for analyzing a second variation trend of the third time sequence characteristic data in a preset time before the fourth time sequence characteristic data appears. And the first determining module is used for determining a second correlation characteristic of the third time sequence characteristic data and the fourth time sequence characteristic data according to the second variation trend. And the second determining module is used for determining condition information for triggering the memory recovery to be abnormal according to the second correlation characteristic and the second change trend.
A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the memory reclamation exception detection method described above.
A fourth aspect of the present disclosure also provides a computer-readable storage medium having stored thereon executable instructions, which when executed by a processor, cause the processor to execute the memory reclamation anomaly detection method described above.
A fifth aspect of the present disclosure also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the method for detecting the memory recovery exception is implemented.
According to the embodiment of the disclosure, by acquiring multiple groups of time sequence characteristic data, under the condition that the time sequence characteristic data is determined to have correlation with the frequent garbage collection data, and the change trend of the time sequence characteristic data is in accordance with the correlation characteristic of the frequent garbage collection data, the early warning information for representing that the memory collection is abnormal is output, so that the automatic early warning of the real-time data of the memory collection is realized.
Drawings
The foregoing and other objects, features and advantages of the disclosure will be apparent from the following description of embodiments of the disclosure, which proceeds with reference to the accompanying drawings, in which:
fig. 1 schematically illustrates an application scenario diagram of a memory reclamation anomaly detection method, apparatus, device, medium, and program product according to embodiments of the present disclosure;
FIG. 2 schematically illustrates a flow chart of a memory reclamation anomaly detection method according to an embodiment of the present disclosure;
FIG. 3 schematically illustrates a flow chart of a method of obtaining time series signature data according to an embodiment of the present disclosure;
FIG. 4 is a flow diagram that schematically illustrates a method for determining a condition from a historical abnormal memory log for triggering a memory reclamation exception, in accordance with an embodiment of the present disclosure;
5a-1 and 5a-2 schematically show the frequency change curve of the new generation data and the old generation data being recycled at the same time when the memory recycling is abnormal and the memory space change curve occupied by the new generation data at the same time;
5b-1 and 5b-2 are graphs schematically showing the frequency change curve of the new generation data and the old generation data being recycled at the same time when the memory recycling is abnormal and the time consumption change curve of the recycled data;
FIG. 5c is a schematic diagram showing the frequency of recovery of new generation data when an anomaly occurs in memory recovery;
FIG. 6 is a flow chart schematically illustrating a method of memory reclamation anomaly detection according to an embodiment of the present disclosure;
7 a-7 d schematically illustrate comparison graphs of the detection effects of the memory recovery anomaly detection method according to the embodiment of the disclosure;
fig. 8 is a block diagram schematically illustrating the structure of a memory recovery abnormality detection apparatus according to an embodiment of the present disclosure;
FIG. 9 is a block diagram that schematically illustrates an arrangement of memory reclamation anomaly detection apparatus, in accordance with further embodiments of the present disclosure; and
fig. 10 schematically shows a block diagram of an electronic device adapted to implement the memory reclamation anomaly detection method according to an embodiment of the present disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is illustrative only and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.
Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B and C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
It should be noted that the method and the apparatus for detecting memory recovery abnormality in the embodiments of the present disclosure may be used in the financial field, and may also be used in any field other than the financial field.
Fig. 1 schematically shows an application scenario diagram of a memory reclamation anomaly detection method according to an embodiment of the present disclosure.
As shown in fig. 1, the application scenario 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. The terminal devices 101, 102, 103 may have installed thereon various communication client applications, such as shopping applications, web browser applications, search applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 105 may be a server providing various services, such as a background management server (for example only) providing support for websites browsed by users using the terminal devices 101, 102, 103. The background management server may analyze and perform other processing on the received data such as the user request, and feed back a processing result (e.g., a webpage, information, or data obtained or generated according to the user request) to the terminal device.
It should be noted that the memory recovery abnormality detection method provided by the embodiment of the present disclosure may be generally executed by the server 105. Accordingly, the memory recovery abnormality detection apparatus provided by the embodiment of the present disclosure may be generally disposed in the server 105. The memory recovery abnormality detection method provided by the embodiment of the present disclosure may also be executed by a server or a server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, and 103 and/or the server 105. Accordingly, the memory recovery abnormality detection apparatus provided in the embodiment of the present disclosure may also be disposed in a server or a server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, and 103 and/or the server 105.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
The memory recovery anomaly detection method of the disclosed embodiment will be described in detail below with reference to fig. 2 to 4 based on the scenario described in fig. 1.
Fig. 2 schematically shows a flowchart of a memory reclamation anomaly detection method according to an embodiment of the present disclosure.
As shown in fig. 2, the method for detecting memory recycling abnormality in this embodiment includes operations S210 to S240.
In operation S210, first timing characteristic data and second timing characteristic data are acquired, where the first timing characteristic data includes any one of the following: memory space occupation data, duration data consumed by data recovery, and frequency data obtained by recovering the first type of data; the second timing characteristic data includes frequency data in which the second type of data is recovered; wherein the frequency with which the first type of data is recycled is greater than the frequency with which the second type of data is recycled.
According to the embodiment of the disclosure, the first type data may be an object with a short life cycle in the application program, and is stored in a new generation area, and may be frequently subjected to garbage collection. The second type of data includes objects with short life cycles in the application and objects with long life cycles in the application, and generally, objects with life cycles longer than that of the first type of data in the application are not frequently subjected to garbage collection. When the space of the old generation region is insufficient, the data garbage collection of the whole memory space is triggered, and the data of the whole memory space can comprise an object with a short life cycle in an application program (new generation data for short) and an object with a long life cycle in the application program (old generation data for short), and can also comprise metadata in a permanent storage region of the memory. When the frequency of the actions (called Full GC for short) of garbage collection in the new generation area and the old generation area exceeds a threshold value, the memory collection is abnormal. In the related art, a fixed threshold is set through expert experience to detect whether the memory recovery is abnormal, however, the fixed threshold can be detected only when the memory recovery is abnormal, and the memory recovery abnormal condition cannot be warned.
In operation S220, the first timing characteristic data and the second timing characteristic data are input into a preset model, and a first correlation characteristic of the first timing characteristic data and the second timing characteristic data is output.
According to an embodiment of the present disclosure, the preset model may adopt a PCMCI causal detection model. PCMCI is a method for analyzing causal correlation between variables and consists of two stage algorithms, PC and MCI. The PC algorithm is used to learn the bayesian network structure among a plurality of random variables. The mci (molar conditional index) detection algorithm is responsible for controlling false positives for highly interdependent time series cases. And checking the causal relationship between each first time sequence characteristic data and each second time sequence characteristic data by adopting a PCMCI algorithm. The test was performed as follows:
a. according to a fixed period, the method is recommended to be executed every 24 hours;
b. when the data is executed, the data in the latest period of time is taken, and the sliding time window needs to cover the execution period, for example, the execution period is 24 hours, then the sliding time window for acquiring the time sequence data can be set as 36 hours;
c. in order to achieve the prediction effect, the first time-series characteristic data must have a certain time delay relative to the second time-series characteristic data. A plurality of different time delays can be set for respective tests, and only one time delay value exists to enable the two time delays to have correlation. The time delay value can be selected from 10 minutes to 60 minutes.
d. And outputting a first correlation characteristic of the first time sequence characteristic data and the second time sequence characteristic data. The first correlation characteristic may include positive correlation, negative correlation, uncorrelated.
In operation S230, a first variation trend of the first timing characteristic data is calculated.
According to the embodiment of the disclosure, a Mann-Kendall algorithm can be adopted to detect the first change trend of the first timing characteristic data, and the specific steps are as follows:
a) let xi、xjFor the ith and jth values of the characteristic variable x, a characteristic value S of the trend is calculated. The calculation formula of the S value is as follows:
Figure BDA0003366128470000091
wherein f is a sign function defined as follows:
Figure BDA0003366128470000092
b) calculating the check value Z according to the formula (III)mk
Figure BDA0003366128470000101
Where VAR (S) is the variance of S, defined as follows:
Figure BDA0003366128470000102
wherein p is the number of repeating numbers, g is the number of unique numbers (number of groups of junctions), tpThe number of repetitions for each repetition number.
c) Determination of results
If it is not
Figure BDA0003366128470000103
Judging that the original data has a trend, otherwise, judging that the original data has no trend. Wherein Z1-nIs 100(1-a) in the standard normal distribution look-up tablethAnd data corresponding to the percentile. On the premise of judging the trend of the original data, when Z isMKAnd if the data is greater than 0, judging that the original data is in an ascending trend, and otherwise, judging that the original data is in a descending trend.
In operation S240, when the first variation trend of the first timing characteristic data conforms to the first correlation characteristic, early warning information for characterizing that the memory recovery may be abnormal is output.
According to the embodiment of the present disclosure, for example, when the first correlation characteristic is a positive correlation, that is, the dependent variable increases with an increase of the independent variable, and at this time, the first change trend of the first time series characteristic data is a rising trend, the first change trend of the first time series characteristic data conforms to the first correlation characteristic, and the second time series characteristic data is also raised with the rising of the first time series characteristic data, so that the occurrence frequency of the operation of predicting that the new generation region and the old generation region are garbage recycled at the same time is raised, and the early warning information of the abnormal memory recycling is output.
According to the embodiment of the disclosure, by acquiring multiple groups of time sequence characteristic data, under the condition that the time sequence characteristic data is determined to have correlation with the frequent garbage collection data, and the change trend of the time sequence characteristic data is in accordance with the correlation characteristic of the frequent garbage collection data, the early warning information for representing that the memory collection is abnormal is output, so that the automatic early warning of the memory collection real-time data is realized.
According to the embodiment of the disclosure, the memory space occupation data includes any one of: the memory space occupation amount before the data are recovered, the memory space occupation amount after the data are recovered, and the memory space occupation amount after the data are recovered; the duration data consumed by recycling the data comprises duration data consumed by recycling the first type of data and/or duration data consumed by recycling the second type of data.
According to the embodiment of the disclosure, the memory space occupation amount before the data is recycled can include the memory space occupation amount before the new generation data is recycled, the memory space occupation amount before the old generation data is recycled, and the memory space occupation amount before the metadata is recycled. The memory space occupation amount after the data is recycled can comprise the memory space occupation amount after the new generation data is recycled, the memory space occupation amount after the old generation data is recycled, and the memory space occupation amount after the metadata is recycled. The memory space occupation amount of the data to be recycled can be a difference value between the memory space occupation amount before the data of the same type is recycled and the memory space occupation amount after the data is recycled. The time length data consumed by the recovery data can be the time length consumed by the recovery of the new generation data, and can also be the time length consumed by the recovery of the new generation data and the old generation data at the same time.
According to the embodiment of the disclosure, the memory space occupation data and the duration data consumed by the recovery data are condition data which may trigger the memory recovery exception, and the correlation between the memory space occupation data and the duration data consumed by the recovery data and the memory recovery exception is analyzed so as to determine the condition for specifically triggering the memory recovery exception and perform early warning on the memory recovery exception.
According to the embodiment of the disclosure, under the condition that the first change trend of the first timing characteristic data conforms to the first correlation characteristic, outputting the early warning information for representing that the memory recovery is abnormal, includes:
and under the condition that the first correlation characteristic is positive correlation and the first change trend of the first time sequence characteristic data is an ascending trend, the first change trend of the first time sequence characteristic data conforms to the first correlation characteristic, and early warning information for representing that the memory recovery is abnormal is output.
According to an embodiment of the present disclosure, for example, the first correlation characteristic is a positive correlation, i.e., the dependent variable increases with increasing independent variable and the dependent variable decreases with decreasing independent variable. According to the embodiment of the disclosure, the change trend of the first time sequence feature data is an independent variable, the change trend of the second time sequence feature data is a dependent variable, when the change trend of the first time sequence feature data is an ascending trend, the independent variable is increased, the dependent variable is also increased, that is, the change trend of the second time sequence feature data is also an ascending trend, the occurrence frequency of actions of performing garbage collection on a new generation region and an old generation region at the same time is predicted to rise, and early warning information of abnormal memory collection is output.
According to the embodiment of the disclosure, the possibility of occurrence of the memory recovery abnormality is predicted by determining the change trend of the time sequence characteristic data having correlation with the memory recovery abnormality, so that the function of automatic early warning is realized, and the problem that the fixed threshold set by depending on expert experience in the related art can be detected only after the occurrence of the memory recovery abnormality is solved.
According to the embodiment of the disclosure, under the condition that the first change trend of the first timing characteristic data conforms to the first correlation characteristic, outputting the early warning information for representing that the memory recovery is abnormal, includes:
and under the condition that the first correlation characteristic is negative correlation and the first change trend of the first time sequence characteristic data is a descending trend, the first change trend of the first time sequence characteristic data conforms to the first correlation characteristic, and early warning information for representing that the memory recovery is abnormal is output.
According to an embodiment of the present disclosure, for example, the first correlation characteristic is a negative correlation, i.e., the dependent variable increases with decreasing independent variable and the dependent variable decreases with increasing independent variable. According to the embodiment of the disclosure, the change trend of the first time sequence feature data is an independent variable, the change trend of the second time sequence feature data is a dependent variable, when the change trend of the first time sequence feature data is a descending trend, the independent variable is reduced, the dependent variable is increased, that is, the change trend of the second time sequence feature data is an ascending trend, the occurrence frequency of actions for predicting that garbage collection is simultaneously performed in a new generation area and an old generation area is increased, and early warning information of abnormal memory collection is output.
According to the embodiment of the disclosure, the possibility of occurrence of the memory recovery abnormality is predicted by determining the change trend of the time sequence characteristic data having correlation with the memory recovery abnormality, so that the function of automatic early warning is realized, and the problem that the fixed threshold set by depending on expert experience in the related art can be detected only after the occurrence of the memory recovery abnormality is solved.
FIG. 3 schematically shows a flow chart of a method of acquiring time series characteristic data according to an embodiment of the disclosure.
As shown in fig. 3, the method of acquiring time-series characteristic data of this embodiment includes operations S310 to S340.
In operation S310, first data is extracted from the memory recycle log according to a first sliding window.
In operation S320, second data is extracted from the memory recycle log according to a second sliding window, where the first data and the second data have the same data type.
According to the embodiment of the present disclosure, the first sliding window and the second sliding window may be fixed time windows, for example: and if the fixed time window is 5s, extracting data from the memory recovery log every 5 s. The method for extracting data can adopt a regular expression to carry out template matching and is used for extracting data information.
According to the embodiment of the disclosure, the first data and the second data are of the same type, for example, the first data and the second data may both be memory space occupation amounts of the new generation data before being recycled.
According to the embodiment of the present disclosure, the following exemplary description is made of data information obtained from a memory recovery log by using a regular expression:
the memory reclamation log is as follows:
2021-08-09T10：39：53.244+0800：225010.501：[Full GC (Ergonomics)[PSYoungGen：1311744K-＞1173345K(1354752K)] [ParOldGen：2796430K-＞2796430K(2796544K)]4108174K-＞3969776K (4151296K)，[Metaspace：180752K-＞180752K(1212416K)]，1.7781550 secs][Times：user＝6.29sys＝0.01，real＝1.78secs]
the regular expression for acquiring data and the data information acquired from the memory recovery log are shown in the following table:
Figure BDA0003366128470000131
Figure BDA0003366128470000141
in operation S330, feature values of the first data and the second data are calculated, respectively, to obtain first feature data and second feature data.
According to an embodiment of the present disclosure, the feature value includes any one of: mean, maximum, minimum.
According to the embodiment of the present disclosure, the number of the first data of the same type extracted from the first sliding window may be 3, for example, the time period taken for recovering the new generation data is 1s, 1.2s, 1.1s, and the first feature data may be an average value of 1.1 of the first data, or may be a maximum value of 1.2 of the first data, or may be a minimum value of 1 of the first data. In the same way, the second data of the same type as the first data is extracted from the second sliding window, which may be 3, for example, the time period taken for 3 new generations of data to be recovered is 1.1s, 1.2s, 1.3s, and the second characteristic data may be an average value of 1.2, a minimum value of 1.1, or a maximum value of 1.3.
According to the embodiment of the present disclosure, the calculation method of the feature values of the first feature data and the second feature data is the same, for example: when the first characteristic data selects the average value 1.1 of the first data, the second characteristic data also selects the average value 1.2 of the second data.
In operation S340, the preset time of the first sliding window and the preset time of the second sliding window are respectively marked on the first feature data and the second feature data to obtain first timing feature data.
According to the embodiment of the present disclosure, the preset time includes any one of: start time, end time.
According to an embodiment of the present disclosure, the preset time may be a start time or an end time of each sliding window. It should be noted that, when the first feature data is marked with the start time of the first sliding window, the second feature data also needs to be marked with the start time of the second sliding window. The preset time can be extracted from the memory recycle log through a regular expression representing a timestamp. E.g. the start time of the first sliding window is t1The start time of the second sliding window is t2If the first characteristic data is 1.1 and the second characteristic data is 1.2, the first timing characteristic data is 1.1 (t)1)、1.2(t2)。
According to the embodiment of the disclosure, the time sequence characteristic data is obtained by extracting the variable data with physical significance from the memory recovery log and marking the time generated by the variable data, so that the correlation between the time sequence characteristic data and the memory recovery abnormity is conveniently checked by adopting a preset model.
Fig. 4 schematically illustrates a flowchart of a method for determining a condition for triggering a memory reclamation exception from a historical exception memory log according to an embodiment of the present disclosure.
As shown in fig. 4, the method of determining a condition for triggering occurrence of an exception in memory reclamation from a history exception memory log according to this embodiment includes operations S410 to S440.
In operation S410, third time series characteristic data and fourth time series characteristic data are extracted from the historical abnormal memory log, where the fourth time series characteristic data includes data with abnormal frequency of the second type of data being recovered, and the third time series characteristic data is the same as the first time series characteristic data in data type.
In operation S420, a second variation trend of the third time-series characteristic data within a preset time before the fourth time-series characteristic data occurs is analyzed.
In operation S430, a second correlation characteristic of the third time-series characteristic data and the fourth time-series characteristic data is determined according to the second variation trend.
In operation S440, condition information for triggering the memory reclamation exception is determined according to the second correlation characteristic and the second variation trend.
According to an embodiment of the present disclosure, the fourth timing characteristic data may be a frequency range in which the new generation data and the old generation data are simultaneously recycled when the memory is recycled abnormally in the historical abnormal memory log. The third timing characteristic data may include memory space occupancy data in historical abnormal memory logs, data on how long it takes to recycle data, data on how often new generation data is recycled, and so on.
According to the embodiment of the disclosure, a second variation trend of the third time series characteristic data can be obtained by analyzing the variation curve chart of the third time series characteristic data before the memory recovery is abnormal.
Fig. 5a-1 and 5a-2 schematically show the frequency change curve of the new generation data and the old generation data being recycled at the same time when the memory recycling is abnormal and the memory space change curve occupied by the new generation data at the same time.
As shown in fig. 5a-1, the abscissa of fig. 5a-1 represents time, and the ordinate represents the number of times each minute that new generation data and old generation data are simultaneously recovered, and the frequency at which the new generation data and the old generation data are simultaneously recovered rapidly increases when an abnormality occurs in memory recovery. As shown in fig. 5a-2, the abscissa represents time, and the ordinate represents the occupied memory space of a single new generation of data being recycled, and when the memory recycling is abnormal, the curve of the memory space occupied by the new generation of data continuously decreases.
According to the embodiment of the disclosure, when the frequency of the new generation data and the old generation data which are simultaneously recycled rapidly rises to exceed a preset fixed threshold, the memory recycling is indicated to be abnormal. Taking the third time series characteristic data as the memory space occupied by the new generation data as an example, as shown in fig. 5a-2, when the memory recovery is abnormal, the second change trend is a continuous decrease, the second correlation characteristic can be determined to be a continuous decrease along with the third time series characteristic data, and the fourth time series characteristic data is continuously increased, and then the condition information for triggering the memory recovery abnormality can be determined to be that when the memory space occupied by the new generation data is continuously decreased, the frequency of the new generation data and the old generation data being simultaneously recovered is rapidly increased, and the memory recovery is abnormal.
Fig. 5b-1 and 5b-2 schematically show frequency change curves of new generation data and old generation data being recycled simultaneously when memory recycling is abnormal and a time-consuming change curve of recycled data.
As shown in fig. 5b-1, the abscissa represents time, and the ordinate represents the number of times per minute that the new generation data and the old generation data are simultaneously recovered, and when an abnormality occurs in memory recovery, the frequency at which the new generation data and the old generation data are simultaneously recovered rapidly increases. As shown in fig. 5b-2, the abscissa represents time, and the ordinate represents the average time duration consumed by a single memory recycle, and when an abnormality occurs in the memory recycle, the time duration consumed by the recycled data rises rapidly and is maintained in a higher value range.
According to the embodiment of the present disclosure, taking the third time series characteristic data as the time duration consumed by the recovery data as an example, as shown in fig. 5b-2, when the memory recovery is abnormal, the second change trend is a rapid increase, it may be determined that the second correlation characteristic is a rapid increase along with the third time series characteristic data, and the fourth time series characteristic data continuously increases, it may be determined that the condition information for triggering the memory recovery abnormality is that when the time duration consumed by the data recovery rapidly increases, the frequency at which the new generation data and the old generation data are simultaneously recovered rapidly increases, and the memory recovery is abnormal.
Fig. 5c schematically shows a frequency variation curve (Full GC) in which the new generation data and the old generation data are simultaneously recovered and a frequency variation curve (Young GC) in which the new generation data is recovered when an abnormality occurs in the memory recovery.
As shown in fig. 5c, the abscissa represents time, and the ordinate represents the frequency at which data is recovered, wherein the solid line curve represents the frequency at which new generation data and old generation data are simultaneously recovered, and the dotted line curve represents the frequency at which new generation data is recovered. When the memory recovery is abnormal, the frequency of the new generation data and the old generation data which are simultaneously recovered rises rapidly, and the frequency of the new generation data which are recovered approaches to 0.
According to the embodiment of the disclosure, taking the third time series characteristic data as the frequency of the new generation data being recovered as an example, as shown in fig. 5c, when the memory recovery is abnormal, the second variation trend is that the curve approaches to 0, the second correlation characteristic can be determined that the third time series characteristic data approaches to 0 and the variation is small, the fourth time series characteristic data continuously rises, and the condition information for triggering the memory recovery abnormality can be determined that when the frequency of the new generation data being recovered approaches to 0 and the variation is small, the frequency of the new generation data and the old generation data being recovered simultaneously rapidly rises, and the memory recovery is abnormal.
According to the embodiment of the disclosure, the time sequence characteristic data is extracted from the historical abnormal memory log, the change trend of the time sequence characteristic data before the memory recovery is abnormal is analyzed, and the condition information for triggering the memory recovery to be abnormal is determined, so that the condition information for triggering the memory recovery to be abnormal is automatically determined according to the change rule of the time sequence characteristic data in the historical abnormal memory log, and the problem that a large amount of time and manpower are consumed for determining the memory recovery abnormal rule depending on manual experience in the related technology is solved.
Fig. 6 schematically shows a flowchart of detection performed by the method for detecting memory recovery exception according to the embodiment of the present disclosure.
As shown in fig. 6, first, time series data is extracted from the memory recycle log, the first time series characteristic data and the second time series characteristic data are obtained by calculating characteristic values of the time series data, and the variation trend of the first time series characteristic data is calculated by using a Mann-Kendall algorithm. And then determining whether the first time sequence characteristic data and the second time sequence characteristic data meet preset rules, wherein the preset rules comprise a condition rule for triggering the memory recovery to be abnormal, which is obtained by analyzing data in a historical abnormal recovery memory log, and an early warning rule for representing that the memory recovery is abnormal, which is determined by the correlation characteristic of the first time sequence characteristic data and the second time sequence characteristic data and the change trend of the first time sequence characteristic data, which are determined by a PCMCI causal detection model. And finally, judging whether the change trends of the first time sequence characteristic data, the second time sequence characteristic data and the first time sequence characteristic data meet the preset rule or not, and if so, outputting early warning information.
According to the embodiment of the disclosure, the early warning level can be set according to the number of the preset rules, and the more the number of the preset rules is, the higher the early warning level is, so that the early warning level can be conveniently used for performing subsequent processing according to the early warning level.
Fig. 7a to 7d schematically show comparison graphs of the detection effects of the memory recovery abnormality detection method according to the embodiment of the present disclosure.
As shown in fig. 7a, the abscissa in fig. 7a represents time, and the ordinate represents the number of times that the new generation data and the old generation data are simultaneously recovered per minute. The abnormal time of memory recovery is 10 points in 8 months, 9 days and 10 days. Before the memory reclamation is abnormal, as shown in fig. 7b, the abscissa in fig. 7b represents the time, and the total coordinate represents the average elapsed time of a single memory reclamation. The recovery data took time for the rapid rise of the temporal data curve at 18 points 8.8.8.8 months. As shown in fig. 7c, the abscissa in fig. 7c represents time, and the ordinate represents the memory space occupation amount by which a single new generation of data is reclaimed. The time for the memory space occupation amount data change curve of the new generation data to continuously decline is 18 points on 8 months and 8 days. Fig. 7d schematically shows a graph of early warning levels according to an embodiment of the present disclosure. As shown in fig. 7d, the abscissa in fig. 7d represents time, and the ordinate represents the early warning level of the memory recovery abnormality. The early warning is already sent out at 18 o' clock 8.8.8.8.8.8.8.8.8.8.8.8. The memory recovery abnormity detection method disclosed by the embodiment of the disclosure can send out early warning information before the memory recovery is abnormal.
Based on the memory recovery abnormity detection method, the disclosure also provides a memory recovery abnormity detection device. The apparatus will be described in detail below with reference to fig. 8.
Fig. 8 schematically shows a block diagram of the structure of the memory recovery abnormality detection apparatus according to the embodiment of the present disclosure.
As shown in fig. 8, the memory recovery abnormality detection apparatus 800 according to this embodiment includes: the system comprises an acquisition module 810, a detection module 820, a calculation module 830 and an early warning module 840. The obtaining module 810 is configured to obtain first timing characteristic data and second timing characteristic data, where the first timing characteristic data includes any one of: memory space occupation data, duration data consumed by recovering data and frequency data of the first type of data being recovered; the second time series characteristic data includes frequency data in which the second type of data is recovered, and the frequency with which the first type of data is recovered is greater than the frequency with which the second type of data is recovered. The detecting module 820 is configured to input the first timing characteristic data and the second timing characteristic data into a preset model, and output a first correlation characteristic of the first timing characteristic data and the second timing characteristic data. The calculating module 830 is configured to calculate a first variation trend of the first timing characteristic data. The early warning module 840 is configured to output early warning information for characterizing that the memory recovery is abnormal when the first change trend of the first timing characteristic data matches the first correlation characteristic.
According to an embodiment of the present disclosure, the early warning module 840 includes a first early warning unit and a second early warning unit. And the first early warning unit is used for outputting early warning information used for representing that the memory recovery is abnormal under the condition that the correlation characteristic is positive correlation and the change trend of the first time sequence characteristic data is an ascending trend, wherein the change trend of the first time sequence characteristic data accords with the correlation characteristic. And the second early warning unit is used for outputting early warning information used for representing that the memory recovery is abnormal, wherein the change trend of the first time sequence characteristic data accords with the correlation characteristic under the condition that the correlation characteristic is negative correlation and the change trend of the first time sequence characteristic data is a descending trend.
According to an embodiment of the present disclosure, the obtaining module 810 includes a first extracting unit, a second extracting unit, a calculating unit, and a marking unit. The first extraction unit is used for extracting first data from the memory recovery log according to the first sliding window. And the second extraction unit is used for extracting second data from the memory recovery log according to a second sliding window, wherein the data types of the first data and the second data are the same. And the calculating unit is used for calculating characteristic values of the first data and the second data respectively to obtain first characteristic data and second characteristic data. And the marking unit is used for respectively marking the preset time of the first sliding window and the preset time of the second sliding window on the first characteristic data and the second characteristic data to obtain first time sequence characteristic data.
Fig. 9 is a block diagram schematically illustrating a structure of a memory reclamation abnormality detection apparatus according to further embodiments of the present disclosure.
As shown in fig. 9, the memory recovery abnormality detection apparatus 800 includes an extraction module 910, an analysis module 920, a first determination module 930, and a second determination module 940, in addition to an acquisition module 810, a detection module 820, a calculation module 830, and an early warning module 840. The extracting module 910 is configured to extract third time series feature data and fourth time series feature data from the historical abnormal memory log, where the fourth time series feature data includes data with abnormal frequency, where the second type of data is recovered, and the third time series feature data is the same as the first time series feature data in data type. The analyzing module 920 is configured to analyze a second variation trend of the third time series characteristic data within a preset time before the fourth time series characteristic data appears. The first determining module 930 is configured to determine a second correlation characteristic of the third time-series characteristic data and the fourth time-series characteristic data according to the second variation trend. The second determining module 940 determines condition information for triggering the memory recovery to be abnormal according to the second correlation characteristic and the second variation trend.
According to the embodiment of the present disclosure, any multiple modules of the obtaining module 810, the detecting module 820, the calculating module 830, the early warning module 840, the extracting module 910, the analyzing module 920, the first determining module 930, and the second determining module 940 may be combined into one module to be implemented, or any one module thereof may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of the other modules and implemented in one module. According to an embodiment of the disclosure, at least one of the obtaining module 810, the detecting module 820, the calculating module 830, the early warning module 840, the extracting module 910, the analyzing module 920, the first determining module 930, and the second determining module 940 may be implemented at least partially as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented by hardware or firmware in any other reasonable manner of integrating or packaging a circuit, or implemented by any one of three implementations of software, hardware, and firmware, or by a suitable combination of any of them. Alternatively, at least one of the obtaining module 810, the detecting module 820, the calculating module 830, the early warning module 840, the extracting module 910, the analyzing module 920, the first determining module 930 and the second determining module 940 may be at least partially implemented as a computer program module, which when executed, may perform a corresponding function.
Fig. 10 schematically shows a block diagram of an electronic device adapted to implement the memory reclamation anomaly detection method according to an embodiment of the present disclosure.
As shown in fig. 10, an electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)1002 or a program loaded from a storage section 1008 into a Random Access Memory (RAM) 1003. Processor 1001 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or associated chipset, and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), among others. The processor 1001 may also include on-board memory for caching usage. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of a method flow according to embodiments of the present disclosure.
In the RAM 1003, various programs and data necessary for the operation of the electronic apparatus 1000 are stored. The processor 1001, ROM 1002, and RAM 1003 are connected to each other by a bus 1004. The processor 1001 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 1002 and/or the RAM 1003. Note that the programs may also be stored in one or more memories other than the ROM 1002 and the RAM 1003. The processor 1001 may also perform various operations of the method flows according to embodiments of the present disclosure by executing programs stored in the one or more memories.
Electronic device 1000 may also include an input/output (I/O) interface 1005, the input/output (I/O) interface 1005 also being connected to bus 1004, according to an embodiment of the present disclosure. Electronic device 1000 may also include one or more of the following components connected to I/O interface 1005: an input section 1006 including a keyboard, a mouse, and the like; an output section 1007 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 1008 including a hard disk and the like; and a communication section 1009 including a network interface card such as a LAN card, a modem, or the like. The communication section 1009 performs communication processing via a network such as the internet. The driver 1010 is also connected to the I/O interface 1005 as necessary. A removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 1010 as necessary, so that a computer program read out therefrom is mounted into the storage portion 1008 as necessary.
The present disclosure also provides a computer-readable storage medium, which may be contained in the apparatus/device/system described in the above embodiments; or may exist separately and not be assembled into the device/apparatus/system. The computer readable storage medium carries one or more programs which, when executed, implement the method according to an embodiment of the disclosure.
According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, a computer-readable storage medium may include the ROM 1002 and/or the RAM 1003 described above and/or one or more memories other than the ROM 1002 and the RAM 1003.
Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the method illustrated in the flow chart. When the computer program product runs in a computer system, the program code is used for enabling the computer system to realize the memory recovery abnormality detection method provided by the embodiment of the disclosure.
The computer program performs the above-described functions defined in the system/apparatus of the embodiments of the present disclosure when executed by the processor 1001. The above described systems, devices, modules, units, etc. may be implemented by computer program modules according to embodiments of the disclosure.
In one embodiment, the computer program may be hosted on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted in the form of a signal on a network medium, distributed, downloaded and installed via the communication part 1009, and/or installed from the removable medium 1011. The computer program may include program code that may be transmitted over any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 1009 and/or installed from the removable medium 1011. The computer program, when executed by the processor 1001, performs the functions defined in the system of the embodiments of the present disclosure. The systems, devices, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.
In accordance with embodiments of the present disclosure, program code for executing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, these computer programs may be implemented using high level procedural and/or object oriented programming languages, and/or assembly/machine languages. The programming language includes, but is not limited to, programming languages such as Java, C + +, python, the "C" language, or the like. The program code may execute entirely on the user computing device, partly on the user device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Those skilled in the art will appreciate that various combinations and/or combinations of features recited in the various embodiments and/or claims of the present disclosure can be made, even if such combinations or combinations are not expressly recited in the present disclosure. In particular, various combinations and/or combinations of the features recited in the various embodiments and/or claims of the present disclosure may be made without departing from the spirit or teaching of the present disclosure. All such combinations and/or associations are within the scope of the present disclosure.
The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in the embodiments cannot be used in advantageous combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be devised by those skilled in the art without departing from the scope of the present disclosure, and such alternatives and modifications are intended to be within the scope of the present disclosure.

Claims (12)

1. A memory recovery abnormity detection method comprises the following steps:
acquiring first time sequence characteristic data and second time sequence characteristic data, wherein the first time sequence characteristic data comprises any one of the following data: memory space occupation data, duration data consumed by data recovery, and frequency data obtained by recovering the first type of data; the second timing characteristic data comprises frequency data from which a second type of data is recovered; wherein the frequency with which the first type of data is recycled is greater than the frequency with which the second type of data is recycled;
inputting the first time sequence characteristic data and the second time sequence characteristic data into a preset model, and outputting a first correlation characteristic of the first time sequence characteristic data and the second time sequence characteristic data;
calculating a first variation trend of the first time-series characteristic data; and
and outputting early warning information for representing that the memory recovery is abnormal under the condition that the first change trend of the first time sequence characteristic data accords with the first correlation characteristic.
2. The method of claim 1, wherein the memory space occupancy data comprises any one of:
the memory space occupation amount before the data are recovered, the memory space occupation amount after the data are recovered, and the memory space occupation amount after the data are recovered; the duration data consumed by the recovery data comprises duration data consumed by the recovery of the first type data and/or duration data consumed by the recovery of the second type data.
3. The method according to claim 1, wherein outputting early warning information for characterizing that memory recovery may be abnormal when a first change trend of the first timing characteristic data conforms to the first correlation characteristic, comprises:
and under the condition that the first correlation characteristic is positive correlation and the first change trend of the first time sequence characteristic data is an ascending trend, the first change trend of the first time sequence characteristic data conforms to the first correlation characteristic, and early warning information for representing that the memory recovery is abnormal is output.
4. The method according to claim 1, wherein outputting early warning information for characterizing that memory recovery may be abnormal when a first change trend of the first timing characteristic data conforms to the first correlation characteristic, comprises:
and under the condition that the first correlation characteristic is negative correlation and the first change trend of the first time sequence characteristic data is a descending trend, the first change trend of the first time sequence characteristic data conforms to the first correlation characteristic, and early warning information for representing that the memory recovery is abnormal is output.
5. The method of claim 1, further comprising:
extracting first data from the memory recovery log according to the first sliding window;
extracting second data from the memory recovery log according to a second sliding window, wherein the data types of the first data and the second data are the same;
respectively calculating characteristic values of the first data and the second data to obtain first characteristic data and second characteristic data;
and marking the preset time of the first sliding window and the preset time of the second sliding window on the first characteristic data and the second characteristic data respectively to obtain the first time sequence characteristic data.
6. The method of claim 5, wherein the preset time comprises any one of: start time, end time.
7. The method of claim 5, wherein the characteristic value comprises any one of: mean, maximum, minimum.
8. The method of claim 1, further comprising:
extracting third time sequence feature data and fourth time sequence feature data from a historical abnormal memory log, wherein the fourth time sequence feature data comprise data with abnormal frequency of the second type of data, and the third time sequence feature data are the same as the first time sequence feature data in data type;
analyzing a second variation trend of the third time series characteristic data within a preset time before the fourth time series characteristic data appears;
determining a second correlation characteristic of the third time series characteristic data and the fourth time series characteristic data according to the second variation trend;
and determining condition information for triggering the memory recovery to be abnormal according to the second correlation characteristics and the second change trend.
9. A memory recovery abnormality detection device includes:
an obtaining module, configured to obtain first timing characteristic data and second timing characteristic data, where the first timing characteristic data includes any one of: memory space occupation data, duration data consumed by data recovery, and frequency data obtained by recovering the first type of data; the second time series characteristic data comprises frequency data in which second type data is recycled, and the frequency of recycling the first type data is greater than the frequency of recycling the second type data;
the detection module is used for inputting the first time sequence characteristic data and the second time sequence characteristic data into a preset model and outputting a first correlation characteristic of the first time sequence characteristic data and the second time sequence characteristic data;
the calculation module is used for calculating a first change trend of the first time sequence characteristic data; and
and the early warning module is used for outputting early warning information for representing that the memory recovery is abnormal under the condition that the first change trend of the first time sequence characteristic data accords with the first correlation characteristic.
10. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-8.
11. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 8.
12. A computer program product comprising a computer program which, when executed by a processor, implements a method according to any one of claims 1 to 8.
CN202111383609.8A 2021-11-22 2021-11-22 Memory recovery abnormity detection method, device, equipment and medium Pending CN114048150A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111383609.8A CN114048150A (en) 2021-11-22 2021-11-22 Memory recovery abnormity detection method, device, equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111383609.8A CN114048150A (en) 2021-11-22 2021-11-22 Memory recovery abnormity detection method, device, equipment and medium

Publications (1)

Publication Number Publication Date
CN114048150A true CN114048150A (en) 2022-02-15

Family

ID=80210451

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111383609.8A Pending CN114048150A (en) 2021-11-22 2021-11-22 Memory recovery abnormity detection method, device, equipment and medium

Country Status (1)

Country Link
CN (1) CN114048150A (en)

Similar Documents

Publication Publication Date Title
US8261278B2 (en) Automatic baselining of resource consumption for transactions
US9378015B2 (en) Predicting defects in code
US20160179588A1 (en) Anomaly detection using a data stream processing language for analyzing instrumented software
US20170153962A1 (en) Monitoring the performance of threaded applications
CN113342559A (en) Diagnostic framework in a computing system
US11809505B2 (en) Method for pushing information, electronic device
CN110704772A (en) Page abnormity monitoring method, system, device, electronic equipment and computer readable medium
CN115174353B (en) Fault root cause determining method, device, equipment and medium
US20180052882A1 (en) Isolation anomaly quantification through heuristical pattern detection
CN113297287B (en) Automatic user policy deployment method and device and electronic equipment
CN113495825A (en) Line alarm processing method and device, electronic equipment and readable storage medium
CN117076280A (en) Policy generation method and device, electronic equipment and computer readable storage medium
CN110704614B (en) Information processing method and device for predicting user group type in application
CN116541069A (en) Key function evaluation method, device, electronic equipment, medium and program product
US9922071B2 (en) Isolation anomaly quantification through heuristical pattern detection
CN111858267A (en) Early warning method and device, electronic equipment and storage medium
CN114048150A (en) Memory recovery abnormity detection method, device, equipment and medium
CN113342604B (en) Method and device for detecting memory utilization rate, terminal and storage medium
CN115098326A (en) System anomaly detection method and device, storage medium and electronic equipment
CN114218283A (en) Abnormality detection method, apparatus, device, and medium
CN111651753A (en) User behavior analysis system and method
CN114710397B (en) Service link fault root cause positioning method and device, electronic equipment and medium
CN112989555B (en) Time sequence data processing method, device, equipment and computer medium
CN114996119B (en) Fault diagnosis method, fault diagnosis device, electronic device and storage medium
CN115859937A (en) Abnormal event detection method, device, electronic equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination