CN115309571A

CN115309571A - Abnormality detection method, abnormality detection device, electronic apparatus, and storage medium

Info

Publication number: CN115309571A
Application number: CN202210953956.8A
Authority: CN
Inventors: 刘俊君; 王卓; 张佳
Original assignee: CCB Finetech Co Ltd
Current assignee: CCB Finetech Co Ltd
Priority date: 2022-08-10
Filing date: 2022-08-10
Publication date: 2022-11-08

Abstract

The disclosure provides an anomaly detection method which can be applied to the technical fields of artificial intelligence and big data. The abnormality detection method includes: acquiring time sequence data to be detected; inputting the time series data to be detected into a sequence abnormality detection model trained in advance, and outputting an abnormality detection result; the pre-trained sequence anomaly detection model is determined by executing a sample data acquisition script template to generate initial training sample data information, executing a sample data pre-processing script template to process the initial training sample data information to generate target training sample data information, and executing a model training script template to process the target training sample data information according to a preset training period and preset training time parameter values. The present disclosure also provides an abnormality detection apparatus, an electronic device, a storage medium, and a program product.

Description

Abnormality detection method, abnormality detection device, electronic apparatus, and storage medium

Technical Field

The present disclosure relates to the field of artificial intelligence and big data technologies, and in particular, to an anomaly detection method, an anomaly detection apparatus, an electronic device, a storage medium, and a program product.

Background

In real life, a large amount of time series data is contained in the fields of clinical medicine, military affairs, geological exploration, network security and the like. The time-series data is a sequence in which data is arranged in the order of occurrence time in a certain action. Thus, the time series data records fluctuating information of a certain action in the time dimension. For example, in the field of network security, when operation and maintenance management is performed, time series data can be used as an observation index to perform anomaly detection on the time series data, and a foundation can be laid for subsequent fault location and other processing.

In the process of implementing the disclosure, it is found that in the task of detecting the time series abnormality, the time series abnormality detection model is trained, and needs to be implemented by manually writing scripts, and when detection is needed each time, a relevant server or other clients need to be logged in to write the scripts again or the scripts are executed after some parameters are modified, which not only leads to a complicated process, but also affects the update efficiency of the model detection service.

Disclosure of Invention

In view of the above, the present disclosure provides an abnormality detection method, an abnormality detection apparatus, an electronic device, a storage medium, and a program product.

According to a first aspect of the present disclosure, there is provided an abnormality detection method including:

acquiring time sequence data to be detected; and

inputting time sequence data to be detected into a sequence abnormality detection model trained in advance, and outputting an abnormality detection result;

the pre-trained sequence anomaly detection model is determined by executing a sample data acquisition script template to generate initial training sample data information, executing a sample data pre-processing script template to process the initial training sample data information to generate target training sample data information, and executing a model training script template to process the target training sample data information according to a preset training period and preset training time parameter values.

According to an embodiment of the present disclosure, further comprising:

constructing a sample data acquisition script template by using a first preset parameter, wherein the first preset parameter represents an associated parameter of initial training sample data;

constructing a sample data preprocessing script template by using a second preset parameter, wherein the second preset parameter is determined based on the first preset parameter; and

and constructing a model training script template by using a third preset parameter, wherein the third preset parameter is determined based on the second preset parameter.

According to an embodiment of the present disclosure, further comprising:

under the condition that the test of the sample data acquisition script template, the sample data preprocessing script template and the model training script template is determined to pass, determining training identification information of the sequence anomaly detection model;

calling sample data acquisition script templates, sample data preprocessing script templates and model training script templates according to the training identification information; and

and according to the preset training period and the preset training time parameter value, generating model parameter information after executing the sample data acquisition script template, the sample data preprocessing script template and the model training script template.

According to an embodiment of the present disclosure, further comprising:

determining a first test parameter value of the correlation parameter according to the correlation parameter;

executing sample data to acquire a script template by using the first test parameter value, and generating initial test sample data information; and

and under the condition that the initial test sample data information is determined to be abnormal, determining the sample data acquisition script template as a test pass.

According to an embodiment of the present disclosure, further comprising:

determining a second test parameter value according to the first test parameter value;

executing a sample data preprocessing script template by using the second test parameter value and the initial test sample data information to generate target test sample data information; and

and under the condition that the target test sample data information meets the first preset condition, determining the sample data preprocessing script template as a test pass.

According to an embodiment of the present disclosure, further comprising:

determining a third test parameter value according to the second test parameter value;

determining an execution result of the model training script template according to the third test parameter value, the target test sample data information and a preset test period; and

and under the condition that the execution result is determined to be abnormal, determining the model training script template as a test pass.

According to the embodiment of the disclosure, after executing the sample data acquisition script template, the sample data preprocessing script template and the model training script template according to the preset training period and the preset training time parameter value, generating model parameter information, including:

repeatedly executing the following operations according to a preset training period:

executing sample data to acquire a script template by using a preset training time parameter value, and generating initial training sample data information;

executing a sample data preprocessing script template by using a preset training time parameter value and initial training sample data information to generate target training sample data information; and

and executing the model training script template by using the preset training time parameter value and the target training sample data information to generate model parameter information.

According to an embodiment of the present disclosure, the association parameters include a storage address parameter of the initial training sample data and a time parameter of the initial training sample data.

A second aspect of the present disclosure provides an abnormality detection apparatus including:

the acquisition module is used for acquiring time series data to be detected; and

the detection module is used for inputting the time series data to be detected into a pre-trained sequence abnormality detection model and outputting an abnormality detection result;

A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the above-described anomaly detection method.

The fourth aspect of the present disclosure also provides a computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the above-described anomaly detection method.

A fifth aspect of the present disclosure also provides a computer program product comprising a computer program which, when executed by a processor, implements the above-described anomaly detection method.

According to the embodiment of the disclosure, an automatic initial training sample data information acquisition, initial training sample data information preprocessing and sequence anomaly detection model training method is provided for the sequence anomaly detection model through the sample data acquisition script template, the sample data preprocessing script template and the model training script template, and a pre-trained sequence anomaly detection model for anomaly detection is obtained, so that not only is the labor input saved, but also the training time of the model is saved, and the service updating efficiency of the sequence anomaly detection model is improved.

Drawings

The foregoing and other objects, features and advantages of the disclosure will be apparent from the following description of embodiments of the disclosure, taken in conjunction with the accompanying drawings of which:

fig. 1 schematically illustrates an application scenario diagram of an anomaly detection method, an anomaly detection apparatus, an electronic device, a storage medium, and a program product according to embodiments of the present disclosure;

FIG. 2 schematically illustrates a flow chart of an anomaly detection method according to an embodiment of the present disclosure;

FIG. 3 schematically illustrates a flow diagram of an anomaly detection method according to another embodiment of the present disclosure;

FIG. 4 schematically illustrates a flow chart of an anomaly detection method according to yet another embodiment of the present disclosure;

FIG. 5 is a flowchart schematically illustrating a method for generating model parameter information after executing a sample data acquisition script template, a sample data preprocessing script template, and a model training script template according to a preset training period and preset training time parameter values, according to an embodiment of the present disclosure;

fig. 6 schematically shows a block diagram of the structure of an abnormality detection apparatus according to an embodiment of the present disclosure; and

fig. 7 schematically shows a block diagram of an electronic device adapted to implement an anomaly detection method according to an embodiment of the present disclosure.

Detailed Description

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is illustrative only and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.

In those instances where a convention analogous to "at least one of A, B, and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B, and C" would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.).

In the technical scheme of the disclosure, the acquisition, storage, application and the like of the personal information of the related user all accord with the regulations of related laws and regulations, necessary security measures are taken, and the commonness and the customs are not violated.

In the technical scheme of the embodiment of the disclosure, before the personal information of the user is obtained or collected, the authorization or the consent of the user is obtained.

In the process of implementing the present disclosure, it is found that, for training of a detection model in time series anomaly detection, a manual script writing mode needs to be used to search data from a database each time, a time series data set is obtained, and then a relevant server or client software is logged in to execute a script, so that the problems of low efficiency, complex operation execution steps and the like when the script is written or modified exist. When the acquired time series data set is preprocessed, data is generally read by manually using data processing software or a writing program, and then operations such as aggregation, calculation and the like are performed on field features of the data, so that the data is converted into a data format required by a time series anomaly detection model. Training of the time series anomaly detection algorithm model requires manual calling of the algorithm model to read in the latest preprocessed time series data and execute the model training process. In the whole time sequence abnormity detection task, the process is complicated, and the updating efficiency of the model detection service is low.

An embodiment of the present disclosure provides an anomaly detection method, including: acquiring time sequence data to be detected; inputting the time sequence data to be detected into a sequence anomaly detection model trained in advance, and outputting an anomaly detection result; the pre-trained sequence anomaly detection model is determined by executing a sample data acquisition script template to generate initial training sample data information, executing a sample data pre-processing script template to process the initial training sample data information to generate target training sample data information, and executing a model training script template to process the target training sample data information according to a preset training period and preset training time parameter values.

Fig. 1 schematically shows an application scenario diagram of an abnormality detection method, an abnormality detection apparatus, an electronic device, a storage medium, and a program product according to an embodiment of the present disclosure.

As shown in fig. 1, the application scenario 100 according to this embodiment may include

terminal devices

101, 102, 103, a network 104, and a server 105. Network 104 is the medium used to provide communication links between

terminal devices

101, 102, 103 and server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

A user may use

terminal devices

101, 102, 103 to interact with a server 105 over a network 104 to receive or send messages or the like. The

terminal devices

101, 102, 103 may have installed thereon various communication client applications, such as shopping-like applications, web browser applications, search-like applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).

The

terminal devices

101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.

The server 105 may be a server providing various services, such as a background management server (for example only) providing support for websites browsed by users using the

terminal devices

101, 102, 103. The backend management server may analyze and process the received data such as the user request, and feed back a processing result (for example, a web page, information, or data obtained or generated according to the user request) to the terminal device.

It should be noted that the anomaly detection method provided by the embodiment of the present disclosure may be generally executed by the server 105. Accordingly, the abnormality detection apparatus provided by the embodiment of the present disclosure may be generally disposed in the server 105. The anomaly detection method provided by the embodiments of the present disclosure may also be performed by a server or a server cluster that is different from the server 105 and is capable of communicating with the

terminal devices

101, 102, 103 and/or the server 105. Accordingly, the abnormality detection apparatus provided in the embodiment of the present disclosure may also be provided in a server or a server cluster that is different from the server 105 and is capable of communicating with the

terminal devices

101, 102, 103 and/or the server 105.

The abnormality detection method provided by the embodiments of the present disclosure may also be executed by the

terminal devices

101, 102, 103. Accordingly, the abnormality detection apparatus provided in the embodiments of the present disclosure may also be generally provided in the

terminal devices

101, 102, and 103. The abnormality detection method provided by the embodiment of the present disclosure may also be executed by other terminals different from the

terminal devices

101, 102, 103. Accordingly, the abnormality detection apparatus provided in the embodiments of the present disclosure may also be provided in other terminals different from the

terminal devices

101, 102, and 103.

It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for an implementation.

The abnormality detection method of the disclosed embodiment will be described in detail below with reference to fig. 2 to 5 based on the scenario described in fig. 1.

Fig. 2 schematically shows a flow chart of an anomaly detection method according to an embodiment of the present disclosure.

As shown in fig. 2, the abnormality detection method 200 of this embodiment includes operations S201 to S202.

In operation S201, time-series data to be detected is acquired.

According to the embodiment of the disclosure, the time series data to be detected can be acquired from a database or a network by writing a script or a data acquisition technology.

In operation S202, the time-series data to be detected is input to a sequence anomaly detection model trained in advance, and an anomaly detection result is output.

The pre-trained sequence anomaly detection model is determined by executing a sample data acquisition script template to generate initial training sample data information, executing a sample data pre-processing script template to process the initial training sample data information to generate target training sample data information, and executing the model training script template to process the target training sample data information to generate model parameter information according to a preset training period and preset training time parameter values.

According to the embodiment of the disclosure, the preset training period can be determined according to the accuracy of the actual requirement training model. The preset training time parameter value can be determined according to the time of initial sample data of the model needing to be trained actually. The sample data acquisition script template can be constructed according to the associated parameters of the initial sample data which needs to be acquired actually. The sample data preprocessing script template and the model training script template can be constructed according to the associated parameters of the initial sample data in the constructed sample data acquisition script template.

According to an embodiment of the present disclosure, the associated parameters of the initial sample data may include: the storage address parameter of the initial sample data and the time parameter of the initial sample data. The time parameter of the initial sample data may include a start time parameter of the initial sample data, an end time parameter of the initial sample data, a time interval parameter of the initial sample data, and the like.

For example, the sample data acquisition script template may be constructed according to a storage address parameter of initial sample data of the initial sample data, a start time parameter of the initial sample data, an end time parameter of the initial sample data, and a time interval parameter of the initial sample data.

The start time parameter of the target sample data, the end time parameter of the target sample data and the time interval parameter of the target sample data required by the model training can be determined according to the start time parameter of the initial sample data, the end time parameter of the initial sample data and the time interval parameter of the initial sample data. And constructing a sample data preprocessing script template according to the storage address parameter of the initial sample data, the starting time parameter of the target sample data required by model training, the ending time parameter of the target sample data and the time interval parameter of the target sample data.

The model training script template can be constructed according to the storage address parameter of the target training sample data information generated by the sample data preprocessing script template, the starting time parameter of the target sample data required by the model training, the ending time parameter of the target sample data and the time interval parameter of the target sample data.

According to an embodiment of the present disclosure, the abnormality detection method may further include: constructing a sample data acquisition script template by using a first preset parameter, wherein the first preset parameter represents an associated parameter of initial training sample data; constructing a sample data preprocessing script template by using a second preset parameter, wherein the second preset parameter is determined based on the first preset parameter; and constructing a model training script template by using a third preset parameter, wherein the third preset parameter is determined based on the second preset parameter.

According to an embodiment of the present disclosure, the association parameters may include a storage address parameter of the initial training sample data and a time parameter of the initial training sample data. Wherein, the time parameter of the initial training sample data may include: a start time parameter of the initial sample data, an end time parameter of the initial sample data, and a time interval parameter of the initial sample data.

For example, the storage address of the initial training sample data may include: a search server (Elasticsearch) address, a distributed file system (Hadoop HDFS) address, a network storage system (ceph, nfs) address and the like. The start time and the end time of the initial sample data may be represented by a timestamp, which may be accurate to seconds. The time interval of the initial sample data may be expressed in integers, which may be in seconds, a time interval no greater than the start time and the end time of the initial sample data. The first preset parameter may represent a start time parameter of the initial sample data, a termination time parameter of the initial sample data, a time interval parameter of the initial sample data, and a storage address parameter of the initial training sample data.

According to an embodiment of the present disclosure, the determining of the second preset parameter based on the first preset parameter may include: determining a time parameter representing target training sample data according to the time parameter representing the initial training sample data; and determining a second preset parameter according to the time parameter representing the target training sample data and the storage address parameter representing the initial training sample data. Wherein, the time parameter of the target training sample data may include: a start time parameter of the target sample data, an end time parameter of the target sample data, a time interval parameter of the target sample data, and the like.

For example, the start time and the end time of the target sample data may be represented by a time stamp, which may be accurate to seconds. The starting time of the target sample data is not less than the starting time of the initial training sample data. The termination time of the target sample data is not greater than the termination time of the initial training sample data. The time interval of the target sample data may be expressed by an integer, which may be in units of seconds, a time interval no greater than the start time and the end time of the target sample data. The second preset parameter may represent a start time parameter of the target sample data, an end time parameter of the target sample data, a time interval parameter of the target sample data, and a storage address parameter of the initial training sample data.

According to an embodiment of the present disclosure, the determining of the third preset parameter based on the second preset parameter may include: target training sample data information generated by a sample data preprocessing script template constructed by using second preset parameters is used for determining storage address parameters representing the target training sample data; and determining a third preset parameter according to the time parameter representing the target training sample data and the storage address parameter representing the target training sample data.

For example, the third preset parameter may represent a start time parameter of the target sample data, an end time parameter of the target sample data, a time interval parameter of the target sample data, and a storage address parameter of the target training sample data.

It should be noted that when the sample data acquisition script template, the sample data preprocessing script template, and the model training script template are constructed, the shell/python script language or other script languages, such as go language, java language, etc., may be used to write the script template or executable program.

According to the embodiment of the disclosure, a sample data acquisition script template, a sample data preprocessing script template and a model training script template are respectively constructed and obtained through related parameters (such as a first preset parameter, a second preset parameter and a third preset parameter) and are used for automatically training a sequence anomaly detection model. The process of initial training sample data information acquisition, initial training sample data information preprocessing and sequence anomaly detection model training is automated, the problems of complexity in manual processing and low processing efficiency are solved, and the service updating efficiency of the sequence anomaly detection model is improved.

Fig. 3 schematically shows a flow chart of an anomaly detection method according to another embodiment of the present disclosure.

As shown in fig. 3, the abnormality detection method 300 of this embodiment may further include operations S301 to S303.

In operation S301, when it is determined that the test of the sample data acquisition script template, the test of the sample data preprocessing script template, and the test of the model training script template all pass, the training identification information of the sequence anomaly detection model is determined.

According to the embodiment of the disclosure, the initial test sample data information can be output by inputting the parameter value corresponding to the storage address parameter of the initial training sample data and the parameter value corresponding to the time parameter of the initial training sample data into the constructed sample data acquisition script template. And carrying out exception identification on the initial test sample data information, and if the initial test sample data information is not abnormal, the sample data acquisition script template passes the test.

According to the embodiment of the disclosure, the target training sample data information can be output by inputting the parameter value corresponding to the storage address parameter of the initial training sample data and the parameter value corresponding to the time parameter of the target training sample data into the constructed sample data preprocessing script template. And performing abnormity identification on the target training sample data information, and if the target training sample data information is not abnormal, passing the test of the sample data preprocessing script template.

According to the embodiment of the disclosure, the parameter value corresponding to the storage address parameter of the target training sample data and the parameter value corresponding to the time parameter of the target training sample data can be input into the constructed model training script template for operation, if the model training script template can be normally operated and the preset test period is completed or the preset test model precision is reached, the operation is finished, the generated model parameter information can be stored, and the model training script template passes the test. The generated model parameter information may be stored in a local disk or a remote File System such as HDFS (Hadoop Distributed File System) in a File format.

According to an embodiment of the present disclosure, the training identification information may include training task information. For example, it may be training task ID information.

In operation S302, according to the training identification information, a sample data acquisition script template, a sample data preprocessing script template, and a model training script template are called.

According to the embodiment of the disclosure, the sample data acquisition script template, the sample data preprocessing script template and the model training script template can be called through the script template interface according to the training identification information.

In operation S303, after the sample data acquisition script template, the sample data preprocessing script template, and the model training script template are executed according to the preset training period and the preset training time parameter value, model parameter information is generated.

According to the embodiment of the present disclosure, the preset training period may be determined according to the accuracy of an actual training model or according to the number of iterations of the training model convergence. The preset training time parameter value may include a start time of actual training, an end time of actual training, a time interval of actual training, and the like. The model parameter information may be, for example, a parameter value of a trained sequence anomaly detection model.

According to the embodiment of the disclosure, under the condition that the built sample data acquisition script template, the sample data preprocessing script template and the model training script template pass the test, the training identification information is obtained, and the corresponding script template is called to be used for automatically training the sequence anomaly detection model, so that the automatic training sequence anomaly detection model can normally operate, and the service updating efficiency of the sequence anomaly detection model is improved.

According to an embodiment of the present disclosure, the abnormality detection method may further include: determining a first test parameter value of the correlation parameter according to the correlation parameter; executing sample data to obtain a script template by using the first test parameter value, and generating initial test sample data information; and under the condition that the initial test sample data information is determined to be abnormal, determining the sample data acquisition script template as a test pass.

According to an embodiment of the present disclosure, the association parameter may include: a storage address parameter of the initial sample data and a time parameter of the initial sample data. The time parameter of the initial sample data may include a start time parameter of the initial sample data, an end time parameter of the initial sample data, a time interval parameter of the initial sample data, and the like. The first test parameter value of the associated parameter may be determined from actual test experience. The initial test sample data information may include initial test sample data and a storage address of the initial test sample data.

For example, the first test parameter value may include a parameter value of a memory address parameter of the initial test data, a parameter value of a start time parameter of the initial test data, a parameter value of an end time parameter of the initial test data, and a parameter value of a time interval parameter of the initial test data. If the initial test sample data information is incomplete or the obtained initial test sample data information is empty, the condition that the initial test sample data information is abnormal can be determined.

According to the embodiment of the disclosure, the sample data acquisition script template is executed according to the first test parameter value, and whether the test is passed is determined according to the generated initial test sample data information, so that the constructed sample data acquisition script template can be used for acquiring the original sample data in the training process of the sequence anomaly detection model.

According to an embodiment of the present disclosure, the abnormality detection method may further include: determining a second test parameter value according to the first test parameter value; executing a sample data preprocessing script template by using the second test parameter value and the initial test sample data information to generate target test sample data information; and determining the sample data preprocessing script template as a test pass under the condition that the target test sample data information meets the first preset condition.

According to an embodiment of the present disclosure, the second test parameter value may include a parameter value of a storage address parameter of the initial test sample data, a parameter value of a start time parameter of the target test data, a parameter value of an end time parameter of the target test data, and a parameter value of a time interval parameter of the target test data.

Determining the second test parameter value according to the first test parameter value may include obtaining a parameter value of a storage address parameter of the initial test sample data according to the first test parameter value; determining a parameter value of a start time parameter of the target test data and a parameter value of an end time parameter of the target test data between the start time of the initial test sample data and the end time of the initial test sample data according to the start time of the target test data and the end time of the target test data; and determining the parameter value of the time interval parameter of the target test data according to the time interval that the time interval of the target test data is not more than the starting time of the target test data and the ending time of the target test data.

According to an embodiment of the present disclosure, the first preset condition may be whether the initial test sample data information has completed standardization and normalization processing. The target test sample data information obtained after the standardization and normalization processing is finished can be directly used for training the model training script template. The target test sample data information may include the target test sample data and a storage address of the target test sample data.

According to the embodiment of the disclosure, the sample data preprocessing script template is executed according to the determined second test parameter value and the initial test sample data information, whether the test is passed or not is determined according to the generated target test sample data information, and the constructed sample data preprocessing script template can be used for preprocessing the original sample data in the training process of the sequence anomaly detection model.

According to an embodiment of the present disclosure, the abnormality detection method may further include: determining a third test parameter value according to the second test parameter value; determining an execution result of the model training script template according to the third test parameter value, the target test sample data information and a preset test period; and under the condition that the execution result is determined to be abnormal, determining the model training script template as a test pass.

According to an embodiment of the present disclosure, the third test parameter value may include: the parameter values of the storage address parameters of the target test sample data, the parameter values of the start time parameters of the target test data, the parameter values of the end time parameters of the target test data and the parameter values of the time interval parameters of the target test data. According to the second test parameter value, a parameter value of a storage address parameter of the target test sample data, a parameter value of a start time parameter of the target test data, a parameter value of an end time parameter of the target test data, and a parameter value of a time interval parameter of the target test data can be determined.

According to the embodiment of the disclosure, the execution result can represent whether the model training script template can run according to the third test parameter value, the target test sample data information and the preset test period. If the operation can be normally performed, the condition that the execution result is not abnormal can be determined.

According to the embodiment of the disclosure, the execution result of the model training script template is determined according to the execution result and whether the test passes through the third test parameter value, the target test sample data information and the preset test period, so that the constructed model training script template can be used for determining the model parameter information in the training process of the sequence anomaly detection model.

Fig. 4 schematically illustrates a flow chart of an anomaly detection method according to yet another embodiment of the present disclosure.

As shown in fig. 4, the abnormality detection method 400 of this embodiment includes operations S401 to S410.

In operation S401, a sample data acquisition script template is constructed.

According to the embodiment of the disclosure, the sample data acquisition script template can be constructed according to the storage address parameter of the initial sample data, the starting time parameter of the initial sample data, the ending time parameter of the initial sample data and the time interval parameter of the initial sample data.

In operation S402, a sample data preprocessing script template is constructed.

According to the embodiment of the present disclosure, the start time parameter of the target sample data, the end time parameter of the target sample data, and the time interval parameter of the target sample data, which are required for model training, may be determined according to the start time parameter of the initial sample data, the end time parameter of the initial sample data, and the time interval parameter of the initial sample data. And constructing a sample data preprocessing script template according to the storage address parameter of the initial sample data, the starting time parameter of the target sample data required by model training, the ending time parameter of the target sample data and the time interval parameter of the target sample data.

In operation S403, a model training script template is constructed.

According to the embodiment of the disclosure, the model training script template can be constructed according to the storage address parameter of the target training sample data information generated by the sample data preprocessing script template, the starting time parameter of the target sample data required by the model training, the ending time parameter of the target sample data and the time interval parameter of the target sample data.

In operation S404, the test sample data acquires a script template.

In operation S405, the test sample data preprocesses the script template.

In operation S406, the test model trains the script template.

According to the embodiment of the disclosure, the parameter value corresponding to the storage address parameter of the target training sample data and the parameter value corresponding to the time parameter of the target training sample data can be input into the constructed model training script template to run, if the model training script template can run normally and finishes running when the preset test period is completed or the preset test model precision is reached, the generated model parameter information can be stored, and the model training script template passes the test.

In operation S407, training identification information of the sequence anomaly detection model is determined.

According to the embodiment of the disclosure, under the condition that the test of the sample data acquisition script template, the test of the sample data preprocessing script template and the test of the model training script template are all determined to pass, the training identification information of the sequence anomaly detection model is determined. The training identification information may include training task information. For example, it may be training task ID information.

In operation S408, a sample data acquisition script template, a sample data preprocessing script template, and a model training script template are called.

In operation S409, according to the preset training period and the preset training time parameter value, after the sample data acquisition script template, the sample data preprocessing script template, and the model training script template are executed, model parameter information is generated.

According to the embodiment of the disclosure, the preset training period and the preset training time parameter value can be determined according to actual requirements.

In operation S410, a sequence anomaly detection model trained in advance is obtained according to the model parameter information.

According to the embodiment of the disclosure, the normal operation of the script template can be ensured by respectively constructing and testing the sample data acquisition script template, the sample data preprocessing script template and the model training script template and determining the training identification information of the sequence anomaly detection model under the condition that all three script templates pass. Model parameter information is obtained by respectively calling and executing the three script templates, and a pre-trained sequence anomaly detection model is further obtained so as to carry out anomaly detection on sequence data. The method and the device have the advantages that the automatic initial training sample data information acquisition, initial training sample data information preprocessing and the pre-training of the sequence anomaly detection model are realized, the labor input is saved, the training time of the model is saved, and the service updating efficiency of the sequence anomaly detection model is improved.

Fig. 5 schematically shows a flowchart of a method for generating model parameter information after executing a sample data acquisition script template, a sample data preprocessing script template, and a model training script template according to a preset training period and preset training time parameter values according to an embodiment of the present disclosure.

As shown in fig. 5, the method 500 for generating model parameter information after executing the sample data acquisition script template, the sample data preprocessing script template, and the model training script template according to the preset training period and the preset training time parameter value according to this embodiment may include repeatedly executing operations S501 to S503 according to the preset training period.

In operation S501, a sample data acquisition script template is executed by using a preset training time parameter value, and initial training sample data information is generated.

According to the embodiment of the disclosure, the preset training time parameter value can be determined according to actual requirements. The preset training time parameter values may include an initial sample data start time, an initial sample data end time, and an initial sample data time interval. The initial training sample data information may be path information storing the initial sample data.

For example, the initial sample data start time may be represented by a timestamp accurate to seconds, such as 0 o' clock 0 integration per day. The initial sample data expiration time may be expressed in terms of a time stamp accurate to seconds, such as 24 o' clock 0 minutes per day. The initial sample data time interval may be in seconds, such as 60 seconds. The path information storing the initial sample data may be represented in the form of a file name "training identification information-start time-end time-time interval-raw".

In operation S502, a sample data preprocessing script template is executed by using a preset training time parameter value and the initial training sample data information, so as to generate target training sample data information.

According to an embodiment of the present disclosure, a preset preprocessing parameter value may be determined according to path information of stored initial sample data determined by a preset training time parameter value in operation S501. And executing a sample data preprocessing script template according to the preset preprocessing parameter value and the initial training sample data information to generate target training sample data information. The preprocessing parameter value may include a target sample data start time, a target sample data end time, and a target sample data time interval. The target training sample data information may be path information storing the target sample data.

For example, the target sample data start time may be represented by a timestamp accurate to seconds, such as 9 o' clock 0 minutes per day. The target sample data expiration time may be expressed in a timestamp accurate to seconds, such as 21 o' clock 0 integration per day. The target sample data time interval may be in units of seconds, such as 60 seconds. The path information of the storage target sample data may be represented in the form of a file name of "training identification information-start time-end time-time interval-ready".

In operation S503, the model training script template is executed by using the preset training time parameter value and the target training sample data information, and model parameter information is generated.

According to the embodiment of the present disclosure, the model training script template may be executed according to the target training sample data information determined by the preset training time parameter value in operation S502, so as to generate the model parameter information. The model parameter information may be path information storing model training and model parameters.

For example, the path information storing the model training and the model parameters may be represented in the form of a file name "training identification information-start time-end time-time interval-model".

It should be noted that the information generated by the corresponding script template is named in a unified manner, which is beneficial to unified management, and avoids the problems of disordered and non-unified data storage naming and the like easily caused in the manual processing process.

According to the embodiment of the disclosure, the execution of the sample data acquisition script template, the sample data preprocessing script template and the model training script template is triggered according to the preset training period and the preset training time parameter value, and the model parameter information is generated to obtain the pre-trained sequence anomaly detection model, so that the method is simpler and more efficient than a manual processing mode.

Based on the anomaly detection method, the disclosure also provides an anomaly detection device. The apparatus will be described in detail below with reference to fig. 6.

Fig. 6 schematically shows a block diagram of the structure of an abnormality detection apparatus according to an embodiment of the present disclosure.

As shown in fig. 6, the abnormality detection apparatus 600 of this embodiment includes an acquisition module 610 and a detection module 620.

The acquiring module 610 is configured to acquire time series data to be detected. In an embodiment, the obtaining module 610 may be configured to perform the operation S201 described above, which is not described herein again.

The detection module 620 is configured to input the time-series data to be detected into a sequence anomaly detection model trained in advance, and output an anomaly detection result. In an embodiment, the detecting module 620 may be configured to perform the operation S202 described above, which is not described herein again.

According to an embodiment of the present disclosure, the anomaly detection apparatus 600 may further include a first script template building module, a second script template building module, and a third script template building module.

The first script template construction module is used for constructing a sample data acquisition script template by using a first preset parameter, wherein the first preset parameter represents an associated parameter of initial training sample data.

The second script template construction module is used for constructing a sample data preprocessing script template by using a second preset parameter, wherein the second preset parameter is determined based on the first preset parameter.

The third script template building module is used for building the model training script template by using a third preset parameter, wherein the third preset parameter is determined based on the second preset parameter.

According to an embodiment of the present disclosure, the anomaly detection apparatus 600 may further include a determination identification module, a call script template module, and an execute script template module.

The identification determining module is used for determining the training identification information of the sequence anomaly detection model under the condition that the test of the sample data acquisition script template, the test of the sample data preprocessing script template and the test of the model training script template are all determined to pass.

And the calling script template module is used for calling the sample data acquisition script template, the sample data preprocessing script template and the model training script template according to the training identification information.

The script execution template module is used for generating model parameter information after executing the sample data acquisition script template, the sample data preprocessing script template and the model training script template according to a preset training period and a preset training time parameter value.

According to an embodiment of the present disclosure, the abnormality detection apparatus 600 may further include a first test parameter determination module, a first test execution module, and a first test result determination module.

The first test parameter determining module is used for determining a first test parameter value of the associated parameter according to the associated parameter.

The first test execution module is used for executing the sample data acquisition script template by using the first test parameter value to generate initial test sample data information.

The first test result determining module is used for determining the sample data acquisition script template as a test pass under the condition that the initial test sample data information is determined to be abnormal.

According to an embodiment of the present disclosure, the abnormality detection apparatus 600 may further include a second test parameter determination module, a second test execution module, and a second test result determination module.

The second test parameter determining module is used for determining a second test parameter value according to the first test parameter value.

The second test execution module is used for executing the sample data preprocessing script template by using the second test parameter value and the initial test sample data information to generate target test sample data information.

The second test result determining module is used for determining the sample data preprocessing script template as a test pass under the condition that the target test sample data information is determined to meet the first preset condition.

According to an embodiment of the present disclosure, the abnormality detection apparatus 600 may further include a third test parameter determination module, a third test execution module, and a third test result determination module.

And the third test parameter determining module is used for determining a third test parameter value according to the second test parameter value.

And the third test execution module is used for determining an execution result of the model training script template according to the third test parameter value, the target test sample data information and the preset test period.

And the third test result determining module is used for determining the model training script template as a test pass under the condition that the execution result is determined to be abnormal.

According to an embodiment of the present disclosure, any multiple of the obtaining module 610 and the detecting module 620 may be combined and implemented in one module, or any one of the modules may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of the other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the obtaining module 610 and the detecting module 620 may be implemented at least partially as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in hardware or firmware by any other reasonable manner of integrating or packaging a circuit, or may be implemented in any one of three implementations of software, hardware, and firmware, or in a suitable combination of any of them. Alternatively, at least one of the obtaining module 610 and the detecting module 620 may be at least partially implemented as a computer program module, which when executed may perform a corresponding function.

As shown in fig. 7, an electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. The processor 701 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or associated chipset, and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), among others. The processor 701 may also include on-board memory for caching purposes. The processor 701 may comprise a single processing unit or a plurality of processing units for performing the different actions of the method flows according to embodiments of the present disclosure.

In the RAM 703, various programs and data necessary for the operation of the electronic apparatus 700 are stored. The processor 701, the ROM 702, and the RAM 703 are connected to each other by a bus 704. The processor 701 performs various operations of the method flows according to the embodiments of the present disclosure by executing programs in the ROM 702 and/or the RAM 703. It is noted that the programs may also be stored in one or more memories other than the ROM 702 and RAM 703. The processor 701 may also perform various operations of method flows according to embodiments of the present disclosure by executing programs stored in the one or more memories.

Electronic device 700 may also include input/output (I/O) interface 705, which input/output (I/O) interface 705 also connects to bus 704, according to an embodiment of the present disclosure. The electronic device 700 may also include one or more of the following components connected to the I/O interface 705: an input portion 706 including a keyboard, a mouse, and the like; an output section 707 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 708 including a hard disk and the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. A drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read out therefrom is mounted into the storage section 708 as necessary.

The present disclosure also provides a computer-readable storage medium, which may be contained in the apparatus/device/system described in the above embodiments; or may exist alone without being assembled into the device/apparatus/system. The computer-readable storage medium carries one or more programs which, when executed, implement the method according to an embodiment of the disclosure.

According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, a computer-readable storage medium may include the ROM 702 and/or the RAM 703 and/or one or more memories other than the ROM 702 and the RAM 703 described above.

Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the method illustrated by the flow chart. When the computer program product runs in a computer system, the program code is used for causing the computer system to realize the method provided by the embodiment of the disclosure.

The computer program performs the above-described functions defined in the system/apparatus of the embodiments of the present disclosure when executed by the processor 701. The systems, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.

In one embodiment, the computer program may be hosted on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted in the form of a signal over a network medium, distributed, and downloaded and installed via the communication section 709, and/or installed from the removable medium 711. The computer program containing program code may be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 709, and/or installed from the removable medium 711. The computer program, when executed by the processor 701, performs the above-described functions defined in the system of the embodiment of the present disclosure. The systems, devices, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.

In accordance with embodiments of the present disclosure, program code for executing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, these computer programs may be implemented using high level procedural and/or object oriented programming languages, and/or assembly/machine languages. The programming language includes, but is not limited to, programming languages such as Java, C + +, python, the "C" language, or the like. The program code may execute entirely on the user computing device, partly on the user device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Those skilled in the art will appreciate that various combinations and/or combinations of features recited in the various embodiments and/or claims of the present disclosure can be made, even if such combinations or combinations are not expressly recited in the present disclosure. In particular, various combinations and/or combinations of the features recited in the various embodiments and/or claims of the present disclosure may be made without departing from the spirit or teaching of the present disclosure. All such combinations and/or associations are within the scope of the present disclosure.

The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in the embodiments cannot be used in advantageous combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be devised by those skilled in the art without departing from the scope of the present disclosure, and such alternatives and modifications are intended to be within the scope of the present disclosure.

Claims

1. An anomaly detection method comprising:

acquiring time sequence data to be detected; and

inputting the time sequence data to be detected into a sequence abnormality detection model trained in advance, and outputting an abnormality detection result;

2. The method of claim 1, further comprising:

constructing the sample data acquisition script template by using a first preset parameter, wherein the first preset parameter represents the associated parameter of the initial training sample data;

constructing the sample data preprocessing script template by using a second preset parameter, wherein the second preset parameter is determined based on the first preset parameter; and

and constructing the model training script template by using a third preset parameter, wherein the third preset parameter is determined based on the second preset parameter.

3. The method of claim 2, further comprising:

under the condition that the sample data acquisition script template, the sample data preprocessing script template and the model training script template are determined to pass the test, determining training identification information of a sequence anomaly detection model;

calling the sample data acquisition script template, the sample data preprocessing script template and the model training script template according to the training identification information; and

and according to the preset training period and the preset training time parameter value, generating the model parameter information after executing the sample data acquisition script template, the sample data preprocessing script template and the model training script template.

4. The method of claim 3, further comprising:

determining a first test parameter value of the associated parameter according to the associated parameter;

executing the sample data acquisition script template by using the first test parameter value to generate initial test sample data information; and

5. The method of claim 4, further comprising:

executing the sample data preprocessing script template by using the second test parameter value and the initial test sample data information to generate target test sample data information; and

and under the condition that the target test sample data information is determined to meet a first preset condition, determining the sample data preprocessing script template as a test pass.

6. The method of claim 5, further comprising:

and under the condition that the execution result is determined to be abnormal, determining that the model training script template passes the test.

7. The method according to any one of claims 3 to 6, wherein the generating the model parameter information after executing the sample data acquisition script template, the sample data preprocessing script template, and the model training script template according to the preset training period and the preset training time parameter value comprises:

executing the sample data acquisition script template by using the preset training time parameter value to generate the initial training sample data information;

executing the sample data preprocessing script template by using the preset training time parameter value and the initial training sample data information to generate the target training sample data information; and

and executing the model training script template by using the preset training time parameter value and the target training sample data information to generate the model parameter information.

8. The method of claim 2, wherein the association parameters comprise a storage address parameter of initial training sample data and a time parameter of initial training sample data.

9. An abnormality detection device comprising:

10. An electronic device, comprising:

one or more processors;

a storage device for storing one or more programs,

wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-8.

11. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 8.

12. A computer program product comprising a computer program which, when executed by a processor, carries out the method according to any one of claims 1 to 8.