RU2809254C1 - Method and system for monitoring automated systems - Google Patents

Abstract

FIELD: computer technology.

SUBSTANCE: method and system for monitoring automated systems (AS) in operation, with the aim of projectively and reactively identifying deviations in their operation at various levels of architecture, diagnosing the cause of the identified deviation , as well as notifying the interested parties about the detected event. The effect is achieved through creation of a monitoring system for the AS comprising: a data collection device containing: a control module, a module for extracting data from the monitored object, and a data conversion module; a data streaming device containing: a memory module for intermediate data storage, a data management module for validation, and a data storage module intended for transmission; a stream data processing device containing: a validation control module, a module for storing rules for determining deviations, a module for storing validation rules, and a module for determining deviations; a data storage system containing: a memory module for storing metrics data and a memory module for storing predicted data; forecasting device; a device for determining deviations, containing: a module for generating notifications, a module for controlling the rules for determining deviations, and a module for determining deviations; device for informing users and communicating between them.

EFFECT: improved reliability of the AS monitoring system.

9 cl, 2 dwg

Description

[0001] The presented technical solution relates, in general, to the field of computer technology, and in particular to a method and system for monitoring automated systems (AS) in operation, with the aim of projectively and reactively identifying deviations in their operation at various levels of architecture (infrastructure , execution environment, application level), diagnosing the cause of the detected deviation, as well as notifying interested parties about the detected event.

BACKGROUND OF THE ART

[0002] Various systems are known from the prior art for monitoring automated systems, developed by Splunk, Zabbix, Tivoli, Naumen Network Manager, Naumen Business Service.

[0003] The mentioned systems currently on the market have the following sequence of data processing: collecting data from the monitored object, recording the received information into a storage system, processing the stored data in order to identify deviations. Each stage of data processing requires time, which in turn leads to an increase in the time it takes to detect deviations. Unlike analogues in the claimed solution, the system processes data and searches for deviations simultaneously with recording data into the storage system, i.e. the stage of saving data into a database (DB) is excluded from the processing process, which in turn reduces the time for detecting deviations, as well as will maintain the functionality of the monitoring system in terms of identifying deviations in the operation of the NPP and notifying the relevant users about identified deviations in the event of a database failure. At the same time, the failure of the mechanism for searching for deviations does not affect the performance of the system as a whole and allows you to work with monitoring data.

[0004] The second distinctive feature of the claimed solution is the way the user interacts with the system. In addition to traditional methods (working with dashboards, informing by sending email messages and SMS), the system implements a mechanism for obtaining information about the state of the monitored object via a messenger, which allows reducing the time for making decisions on corrective actions aimed at restoring the agreed state of the system, due to the absence the need to connect engineers and other decision makers to the workplace. Information is obtained both by receiving messages about deviations in the operation of systems from the monitoring platform to the messenger, and by requesting monitoring data from the messenger interface to the monitoring platform for the necessary information.

[0005] The third distinctive feature of the proposed solution is the presence of a built-in system for constructing a predictive model: the baseline and corridor boundaries. The system uses information accumulated recently about the state of the monitoring object, which makes it possible to predict the behavior of the NPP at a specific time and identify in advance a negative trend in the behavior of the NPP before the occurrence of events that could lead to deviation.

[0006] Most monitoring systems are aimed at working with a specific technological layer, for example, server or network infrastructure. In the stated solution, the system collects information from all available technological layers, including through existing connectors to other monitoring systems. Thus, it is possible to build multi-level monitoring in a single place, which makes it possible to reduce the time for diagnosing the root cause of deviations in the operation of the system.

[0007] The key mechanism of any monitoring system is alerting (identifying deviations and reporting them). The quality of alerting determines the quality of monitoring, but the monitoring systems known to us do not contain mechanisms for assessing the quality of alerting. In the stated solution, the system contains functionality that allows you to evaluate each event from the point of view of its correspondence to the real state of the system. Quality assessment is carried out by alert recipients through a specialized interface. Based on the feedback in the system, it is possible, based on the implemented reporting, to identify and correct events with poor response statistics.

DISCLOSURE OF INVENTION

[0008] The technical problem or task posed in this technical solution is the creation of a new effective, simple and reliable system for monitoring speakers.

[0009] The technical result is to increase the reliability of the speaker monitoring system.

[0010] The specified technical result is achieved through the creation of a system (100) for monitoring speakers, containing:

- data collection device (10), containing: control module (11), module (12) for extracting data from the monitoring object (1) and data conversion module (13);

- a device (20) for streaming data, containing: a module (21) for intermediate data storage, a module (22) for managing data for validation, and a module (23) for storing data intended for transmission;

- device (30) for stream data processing, containing: a validation control module (31), a module (32) for storing rules for determining deviations, a module (33) for storing validation rules, and a module (34) for determining deviations;

- a data storage system (40) containing: a memory module (41) for storing metrics data and a memory module (42) for storing predicted data;

- forecasting device (50);

- a device (60) for determining deviations, containing: a module (61) for generating notifications, a module (62) for managing the rules for determining deviations, and a module (63) for determining deviations;

- device (70) for informing users;

wherein the control module (11) is connected to the data retrieval module (12), the data retrieval module (12) is connected to the monitoring object and the data conversion module (13), the intermediate data storage memory module (21) is connected to the data conversion module (13), and validation data management module (22), the validation data management module (22) is connected to the validation management module (31) and the data storage module (23), the validation control module (31) is connected to the validation rules storage module (33) and the module (23) data storage, module (23) is connected to a module (34) for determining deviations and a memory module (41) for storing metrics data, a module (34) for determining deviations is connected to a module (32) for storing rules for determining deviations and a module (61 ) notification generation, the module (62) for managing the rules for determining deviations is connected to the module (32) for storing the rules for determining deviations, the module (63) for determining deviations is connected to the memory module (41) for storing metrics data, the memory module (42) for storing predicted data, module (62) for managing the rules for determining deviations and module (61) for generating notifications, the module (61) for generating notifications is connected to the device (70) for informing users.

[0011] In one particular example of the system, the data streaming device (20) further comprises a data storage module (24) connected to the monitoring object 1 and the data retrieval module (12).

[0012] In another particular example of the implementation of the system, the forecasting device (50) contains interconnected: a control module (51), a data processing module (52), a memory module (53) intended for operational operation of data, and a machine model (54). training;

wherein the data processing module (52) is connected to a memory module (41) for storing metrics data and a memory module (42) for storing predicted data.

[0013] In another particular embodiment of the system, it further comprises a data visualization module (43) connected to a memory module (41) for storing metrics data, a memory module (42) for storing predicted data, and a notification generation module (61).

[0014] In another particular example of the implementation of the system, the deviation determination module (63) is equipped with counters for determining the number of deviations over a given period of time, for determining the difference in the values of various metrics per unit of time, for determining the difference in metric values at different observation times, for determining the sum of values metrics for a certain time interval.

[0015] In another particular embodiment of the system, the data streaming device (20) further includes a memory area designed to store data about metrics that have not passed the validation procedure.

[0016] In another preferred embodiment of the claimed solution, a method for automatic monitoring of speakers is presented, performed by a deviation determination module (63), containing the steps of: - receiving instructions for determining deviations;

- extracting from memory, according to the mentioned instructions, data on metrics for a given period of time;

- extract from memory, according to the mentioned instructions, the upper and lower limits of the metric values defined for the mentioned metrics;

- each metric value is compared with the mentioned limits of the metric values, and if the metric value goes beyond the mentioned limits of values, then the module (63) increases the counter indicator by 1;

- after comparing all metric values with the mentioned boundaries of metric values, the counter indicator is extracted and compared with the threshold value of the counter specified in the instruction, and if the counter value exceeds or corresponds to the specified threshold value of the counter, then a command for generating notifications about detected deviations in the metric value.

[0017] In one of the particular examples of the method, the step of recording the presence of deviations in the metrics data is additionally performed.

[0018] In another particular example of the method, the steps are additionally performed in which:

- extracting from memory, according to the mentioned instructions, data on metrics for the next period of time;

- extract from memory, according to the mentioned instructions, the upper and lower limits of the metric values defined for the mentioned metrics;

- each metric value is compared with the mentioned limits of the metric values, and if the metric value goes beyond the mentioned limits of values, then the module (63) increases the counter indicator by 1;

- after comparing all metric values with the mentioned boundaries of metric values, the counter indicator is extracted and compared with the threshold value of the counter specified in the instruction, and if the counter value does not exceed the specified threshold value of the counter, then the elimination of the deviation is recorded and generated in the module (61) for generating notifications command for generating a notification about eliminating a deviation in the metric value.

BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The features and advantages of the present technical solution will become apparent from the following detailed description of the invention and the accompanying drawings, in which:

[0020] in FIG. Figure 1 shows a general scheme for diagnosing AS.

[0021] in FIG. Figure 2 shows an example of a general view of a computing device.

IMPLEMENTATION OF THE INVENTION

[0022] The concepts and terms necessary to understand this technical solution will be described below.

[0023] In this technical solution, a system means, including a computer system, a computer (electronic computer), CNC (computer numerical control), PLC (programmable logic controller), computerized control systems and any other devices capable of performing a given task. , a clearly defined sequence of operations (actions, instructions).

[0024] By command processing device is meant an electronic unit, a computing device, or an integrated circuit (microprocessor) that executes machine instructions (programs).

[0025] A command processing device reads and executes machine instructions (programs) from one or more storage devices. Storage devices can include, but are not limited to, hard drives (HDD), flash memory, ROM (read-only memory), solid-state drives (SSD), and optical drives.

[0026] Program - a sequence of instructions intended for execution by a computer control device or command processing device.

[0027] Database (DB) - a collection of data organized in accordance with a conceptual structure that describes the characteristics of that data and the relationships between them, and such a collection of data that supports one or more areas of application (ISO/IEC 2382:2015, 2121423 " database").

[0028] A signal is a material embodiment of a message for use in the transmission, processing and storage of information.

[0029] Logical element - an element that implements certain logical dependencies between input and output signals. Logic elements are usually used to construct logical circuits of computers and discrete automatic monitoring and control circuits. All types of logical elements, regardless of their physical nature, are characterized by discrete values of input and output signals.

[0030] According to the diagram shown in FIG. 1, the AC monitoring system 100 includes: a data collection device 10, a data streaming device 20, a data streaming device 30, a data storage system 40, a forecasting device 50, a deviation detection device 60, and a user information device 70.

[0031] The data collection device 10 can be implemented on the basis of at least one computing device and contain: a control module 11, a module 12 for extracting data from the monitoring object 1, and a data conversion module 13 with a buffering function. The mentioned modules can be implemented using logic elements on transistors, for example, “AND”, “OR”, “NOT”, “NAND”, “NOR”, etc., placed on a printed circuit board by a well-known method to ensure transmission between the mentioned modules of signals containing data, as well as to ensure processing of the received data. To enable processing of data from different sources, two or more devices 10 may be implemented, or device 10 may be configured to simultaneously process data from multiple data sources. Also, to ensure additional reliability, a cluster of two or more devices 10 (for each type of device) located in different data centers can be implemented.

[0032] The data streaming device 20 may be implemented on at least one computing device and include: a data store memory module 21, a validation data management module 22, a data storage module 23 for transmission, and a data storage module 24 , received from monitoring object 1. The mentioned modules can be implemented using logic elements on transistors, for example, “AND”, “OR”, “NOT”, “NAND”, “NOR”, etc., placed on a printed circuit board by a well-known method to ensure transmission between the mentioned modules of signals containing data, as well as to ensure processing of the received data. Also, to provide additional reliability, a cluster of two or more devices 20 located in different data centers can be implemented.

[0033] The stream data processing device 30 can be implemented on the basis of at least one computing device and contain: a validation control module 31, a deviation determination rules storage module 32, a validation rules storage module 33, and a deviation determination module 34. The mentioned modules can be implemented using logic elements on transistors, for example, “AND”, “OR”, “NOT”, “NAND”, “NOR”, etc., placed on a printed circuit board by a well-known method to ensure transmission between the mentioned modules of signals containing data, as well as to ensure processing of the received data. To provide additional reliability, a cluster of two or more devices 30 located in different data centers can be implemented.

[0034] The data storage system 40 may be implemented on the basis of at least one database and include: a memory module 41 for storing metrics data and a memory module 42 for storing predicted data. These modules can be implemented on the basis of a permanent data storage device, for example, a hard disk (HDD), a solid-state data drive (SSD), flash memory (EEPROM, NAND, etc.), optical storage media (CD-R/RW , DVD-R/RW, BlueRay Disc, MD), etc. Additionally, the data storage system 40 can be equipped with a data visualization module 43, implemented on a computed device using transistor logic elements, designed to retrieve metrics and predicted data and displaying them to users of the AC monitoring system 100. In an alternative embodiment of the presented solution, the mentioned module 43 can be an independent device, i.e. external to the system 40. Also, said device 43 may be equipped with hardware, such as LCD panels, for displaying information, such as stored in the data storage system 40, in real time.

[0035] To ensure reliability, a cluster of two or more systems 40 located in different data centers can be implemented. Additionally, each data center may implement a copy of system 40 containing data from system 40 located in another data center.

[0036] The forecasting device 50 can be implemented on the basis of at least one computing device and contain: a control module 51, a data processing module 52, a memory module 53 designed for online work with data, and a machine learning model 54. The mentioned modules can be implemented using logic elements on transistors, for example, “AND”, “OR”, “NOT”, “NAND”, “NOR”, etc., placed on a printed circuit board by a well-known method to ensure transmission between the mentioned modules of signals containing data, as well as to ensure processing of the received data.

[0037] The device 60 for determining deviations can be implemented on the basis of at least one computing device and contain: a module 61 for generating notifications, a module 62 for managing rules for determining deviations, and a module 63 for determining deviations. The mentioned modules can be implemented using logic elements on transistors, for example, “AND”, “OR”, “NOT”, “NAND”, “NOR”, etc., placed on a printed circuit board by a well-known method to ensure transmission between the mentioned modules of signals containing data, as well as to ensure processing of the received data. To provide additional reliability, a cluster of two or more devices 60 located in different data centers can be implemented.

[0038] The device 70 for informing users can be implemented on the basis of at least one computing device, implemented in hardware and software in such a way as to ensure the transmission of a message about a detected deviation, or about eliminating the deviation, to the devices of users of the AC monitoring system 100. Also, said device 70 may be equipped with hardware, such as LCD panels, for displaying information, for example, stored in the data storage system 40 in real time.

[0039] The automated system (AS), which is monitored by the said AS monitoring system, can be any system designed to automate production processes. Each AS is equipped with a monitoring object 1, which is further understood in the text as an information system, or its subsystem or a separate component, designed to collect and store data on metrics, which can be presented, for example, in the form of a numerical characteristic, by the magnitude of which one can judge state of the AS or processes occurring in the AS.

[0040] For example, monitoring object 1 may be located in a telephony system, and the metric may be the number of incoming or outgoing calls per given unit of time. A sharp decrease or increase in the number of calls may indicate potential problems in the operation of the speaker system. Metric data can be collected by well-known methods in manual or automated mode and stored in the monitoring object 1.

[0041] Accordingly, depending on the operating mode control module 11 specified by the developer, said module 11 generates a command to connect to the monitoring object 1 to retrieve data on metrics for a time interval specified by the developer; data on the current state of the monitoring object is predominantly retrieved, i.e. the value of the metrics at the time of extraction, or for the last minute. The generated command is sent by the mentioned module 11 to the data retrieval module 12, which, through a wired or wireless connection, connects to the monitoring object 1 and sends a request for metrics data for a given time interval to the monitoring object 1 to retrieve metrics data from the memory of the mentioned object 1. Data about metrics may contain the AS identifier (ID) or the ID of the monitoring object, a list of metrics IDs, metric values and time interval values characterizing the time the metrics were received by monitoring object 1.

[0042] The extracted metric data, for example, in the form of a data packet, is sent by the mentioned module 12 to the data conversion module 13, which converts the data to a format specified by the developer of the module 13, in particular suitable for processing in the following stages, as well as the calculation of aggregates. For example, the mentioned module 13 may receive metric data characterizing a list of all calls that have been received over the past minute, where the metric ID will indicate that the metric is a call metric, and the metric value may characterize the duration of the call. Based on the received metrics data, in particular the list of all calls, module 13 can determine the number of calls received over the past minute, after which the said number can be included in the metrics data as an additional metric. Additionally, as part of the above-mentioned data transformation, unnecessary attributes can be removed from the received metrics data, for example, characterizing the topics of calls, or, conversely, all incoming calls can be grouped by topic and transferred to further stages in the form of separate metrics - the number of calls on specific topics in the past minute.

[0043] In an alternative embodiment of the presented solution, the metrics data can be retrieved by the data extraction module 12 from the data storage module 24 received from the monitoring object 1. Loading of metrics data into module 24 can be performed by monitoring object 1 in accordance with the software algorithm stored in its memory.

[0044] The converted metric data by the data conversion unit 13 is written to the data intermediate storage unit 21 of the data streaming device 20, in which the metric data is stored for a predetermined period of time, after which the data is deleted. The command to delete the data may be received by said module 21, for example, from a corresponding controller with which the module 21 may be equipped, configured to assign a timestamp to the recorded data indicating the time at which it was recorded, and to delete the data recorded in the module 21 , after the expiration of a time interval specified, for example, by the developer of module 21. For example, the delete data command may indicate that all data from module 21 whose timestamp indicates that 6 hours have passed since the data was stored should be deleted. In an alternative embodiment of the presented solution, the command to delete data by the mentioned controller can be received when module 21 reaches a specified limit on the amount of data.

[0045] Next, through the data management module 22, the metric data for which it is necessary to carry out the metric data validation process is determined, and the list of metric data (for example, a list of IDs of speakers or monitoring objects) that need to be validated can be specified by the developer the mentioned module 22, or received from the validation control module 31. To determine the need to validate the metrics data, said module 22 compares the metrics data stored in module 21 with the said list of metrics data, for example, the ID of the vehicle or monitoring object - with the list of IDs of the vehicle or monitoring objects, and if module 22 has determined that the data about the metrics require validation (for example, the IDs of the AS or monitoring object are contained in the list of IDs of the AS or monitoring objects), then module 22 retrieves data about metrics from module 21 and sends them to validation control module 31. If module 22 has determined that the metrics data does not require validation (for example, the ID of the AS or monitoring object is not contained in the list of IDs of the AS or monitoring objects), then module 22 retrieves the metrics data from module 21 and sends them to data storage module 23, intended for transmission.

[0046] When the validation management module 31 receives the metrics data, the module will check them for compliance with the established requirements. To determine the specified requirements for the metrics data, said module 31 accesses the validation rules storage module 32, in which the said requirements can be entered by the module developer or its operator. For example, as part of validation by module 31, the following metric attributes of each record contained in the metric data may be validated:

- Metric ID (for example, name);

- Timestamp for creating/receiving metrics;

- Metric value;

- Name of the metric source;

- Identifier of the monitoring object (name of the automated system);

- And others

[0047] The metric validator implemented in module 31 is configured to check both each field of all records contained in the metrics data separately, and to check fields specified by the developer, for example, checking the presence of a metric value in the “Metric Value” field (t i.e. the field is not empty) and checking that the “Monitoring object identifier” is included in the set of valid values. A directory of valid values of identifiers of monitoring objects 1 can be stored in the memory of module 32.

[0048] In particular, for each field in the record, the following checks can be performed by module 31:

- Checking whether the value matches a regular expression;

- Non-zero value (for any field type);

- Correspondence of the attribute value to one of the values in a given dictionary or enumeration.

[0049] For example, the metric data received by module 31 may contain a field characterizing the time the metric was received (datearrival), which may contain the following information: "date_arrival": "2022-12-08 10:35:00", where ID The fields are characterized by the attribute "date_arrival", and the field value is "2022-12-08 10:35:00".

Accordingly, module 31, when validating metrics data, retrieves the "date_arrival" field ID and accesses the metrics data requirements data received from module 32 to determine the data validation algorithm.

[0050] For example, the data validation algorithm may indicate that said module 31 should check for the presence of a metric acquisition time field value in the metric data. If module 31 determines that the field value is missing (i.e., the field value is empty), then module 31 determines that the metrics data does not meet the specified requirements, after which the metrics data is removed from the module 31 buffer. Thus, in the future, metric data that has not passed the validation procedure will be deleted from the intermediate data storage memory module 21 according to the previously described algorithm. Additionally, metric data that has not passed the validation procedure can be placed in a memory area of the device 20 for further analysis of the reasons why the data did not pass validation.

[0051] If module 31 determines that the field value is present in the metrics data, then module 31 retrieves the field value "2022-12-08 10:35:00" from the metrics data and compares said value with the established requirements received from the module 32. Specified requirements may indicate, for example, that the field value entry must comply with a specified format, for example, "yyyy-MM-dd HH:mm:ss", and the field value itself must correspond to a specified range of field values, for example, value fields characterizing the time the data was received cannot be older than 6 hours, and cannot be later than the current time. To determine the current time, said module 31 may be equipped with a suitable timer, which is widely known in the art.

[0052] If module 31 determines that the field value does not conform to a specified format or metric value range, then module 31 determines that the metric data does not meet the specified requirements, after which the metric data is removed from the module 31 buffer. Thus , in the future, data on metrics that have not passed the validation procedure will be deleted from the intermediate data storage memory module 21 according to the previously described algorithm. Additionally, metric data that has not passed the validation procedure can be placed in a memory area of the device 20 for further analysis of the reasons why the data did not pass validation. If the module 31 determines that the metric data meets the specified requirements, then the module 31 writes the metric data to the data storage module 23 for transmission.

[0053] Similar validation processes by module 31 can be carried out in relation to fields containing information characterizing:

- time of creation of the metric, which can be presented in the form: "date_created": "2022-12-08 10:33:05", where the field ID characterizes the "date_created" attribute, and the field value is "2022-12-08 10: 33:05";

- label name, which can be represented in the form: "metric_name": "Avaya CRM.PB: Active Calls \"Total\" \"Total\"", where the field ID characterizes the "metric_name" attribute, and the field value is "Avaya CRM.PB: Active Calls \"Total\" \"Total\"";

- agent name, which can be represented in the form: "agent host": "10.111.0.39", where the field ID characterizes the "agent_host" attribute, and the field value is "10.111.0.39";

- source name, which can be represented in the form: "source_host": "aggregate.data.mpp", where the field ID characterizes the "source host" attribute, and the field value is "aggregate.data.mpp";

- metric type, which can be represented in the form: "metric_type": "app", where the field ID characterizes the source_host attribute", and the field value is "app";

- AS name, which can be represented in the form: “as_name”: “esgr”, where the field ID characterizes the “as_name” attribute, and the field value is “esgr”;

- uuid process, which can be represented in the form: "process uuid": "47b331b8-b773-401c-bcce-84d74eac6ef4", where the field ID characterizes the "process_uuid" attribute, and the field value is "47b331b8-b773-401c-bcce- 84d74eac6ef4";

- metric value, which can be represented in the form: "value_int": 844, where the field ID characterizes the "value_int" attribute, and the field value is 844.

[0054] Accordingly, information about the time of creation of the metric by the mentioned module 31 can be checked in the manner described above for compliance with the established format and the established value threshold; The label name information can be checked for the field value, and this value cannot be a “null” value; agent name information can be checked for the presence of a field value and for compliance of the field value with a specified format, for example, ip or dns format; information about the source name can be checked for the presence of a field value and for compliance of the field value with a specified format, for example, ip or dns, or url format; metric type information can be checked for the presence of a field value and whether the field value matches, for example, one of the following values: "app", "regex", "business", "synt", "infra", "logs", "sql" ", "unknown"; AC name information can be checked for the field value, and the field value must match the predefined values in the established requirements; the process uuid information can be checked for a field value, and this value cannot be a “null” value; the metric value can be checked for the presence of a field value, and this value cannot be a “null” value.

[0055] The metric data stored in the module 23, which has passed the validation procedure, can then be read by the deviation determination module 34, as well as the memory module 41 for storing metrics data. Data reading by the mentioned modules can be carried out both in real time and according to schedules specified by the module developers, and metrics data is retrieved from the moment the previous portion of data is received. Module 23, like module 21, stores metrics data for a specified period of time, after which the said data is deleted.

[0056] From the received metric data, the variance determination module 34 extracts a metric value and then compares the extracted metric value with a threshold value for the metric. The threshold value of the metric by the mentioned module 34 can be determined by accessing the deviation determination rules storage module 33, in which for each metric its threshold value and instructions for processing data for determining the deviation are stored. Said threshold values can be written to module 33 by means of module 62 for managing the rules for determining deviations according to the algorithm for updating threshold values laid down by the developer of said module 62, which eliminates the possibility of unauthorized changes to the data stored in module 33 by the operator or developer. Also, instructions for processing metrics data to determine deviations can be written to module 33 by module 62.

[0057] If the module 34 determines that the metric value does not correspond to the threshold values, for example, the metric value characterizing the number of calls per minute is equal to “0” or exceeds a predetermined threshold value of “1000”, then the mentioned module 34 records the presence of a deviation in the data about metrics and sends a command to the notification generation module 61 to generate a notification about a detected deviation in the metric value, which is then sent to the user information device 70 to notify interested parties about the detected deviation, after which module 34 reads the next portion of data from module 23 containing data about metrics for their analysis. If module 34 determines that the metric value meets the thresholds, then said module 34 reads the next piece of data from module 23 containing metric data for analysis.

[0058] If, in the process of comparing the next portion of data, module 34 determines that the metric value does not exceed a specified threshold value, and a deviation was previously detected, then module 34 records the elimination of the deviation, and then sends a command to the notification generation module 61 to generate a notification about eliminating the deviation in the metric value, which is then sent to the device 70 for informing users to notify interested parties about the elimination of the deviation, after which module 34 reads the next portion of data from module 23 containing data on metrics for their analysis.

[0059] Metric data read by memory module 41 from module 23 is stored in the database, for example, in the form of a time series, after which the metric data can be retrieved from the database for analysis and processing. For example, the metric data may be retrieved by the predictor 50 by the data processing module 52 upon a command received from the control module 51. Said command may be generated and sent by module 51 according to a schedule specified by the developer of module 51, which may also specify a time period for which metrics data should be retrieved.

[0060] The metric data retrieved by module 52 for a given period of time is stored in memory module 53 for on-line data processing, after which control module 51 generates a command to run the machine learning model 54 to obtain prediction data for a given period of time. The machine learning model 54, upon receiving the mentioned command from the module 51, retrieves data on metrics from the memory module 53 and, using known methods, determines the predicted values of the metrics for a given period of time.

[0061] For example, the metric data stored in the memory module 53 may contain values characterizing the number of calls per minute for the previous three months, and the machine learning model 54 can determine predictive values of metrics characterizing the number of calls per minute using known methods. minute, for the next 24 hours. Also, together with the mentioned forecast values, module 54 can determine the upper and lower boundaries (corridors) of the metric values, into which, with a probability of 95%, the previously determined forecast metric values should fall at a specific minute.

[0062] To determine the above-mentioned predicted metric values and metric value bounds, said model 54 can be pre-trained by known methods on a training data set of metrics and metric bounds. Predicted metric values and metric value boundary data are stored in memory module 53 and are then retrieved by module 52 and stored in memory module 42 for storing predicted data. After the predicted metric values and metric value boundary data are stored, they can be retrieved by the data visualization module 43 along with the metric data and sent to the notification generation module 61, which generates a corresponding notification that includes said received data, e.g. in the form of a schedule, after which the generated notification is sent to the user information device 70 to notify interested parties.

[0063] To determine deviations in the metric data stored in the data storage system 40, taking into account the predicted values of the metrics, the AC monitoring system 100 is equipped with a deviation detection device 60, which detects deviations in the said data using more complex instructions in accordance with the algorithm laid down by the developer .

[0064] In particular, in accordance with the underlying algorithm, the deviation determination module 63 accesses the deviation determination rule control module 62 to retrieve instructions for determining deviations. Said instructions stored in module 62 typically contain the following parameters:

- Unique ID of instructions (generated by module 62 automatically by known methods when receiving data with instructions based on the algorithm stored in the memory of module 62);

- Name of instructions;

- Metric ID or set of metrics IDs that should be processed;

- At least one threshold value - a numerical value with which the metric values will be compared;

- At least one data processing instruction to determine the deviation - a comparison operation that will be used to compare the metric and the threshold value. For example, “More than”, “Less than”, “Greater than or equal to”, “Equal”, “Not equal”, etc.;

- Phone and email of recipients of information about a deviation identified based on instructions, indicating the notification channels (SMS, email, messenger) through which messages will be delivered.

[0065] These anomaly detection instructions may be specified by the designer or operator of the anomaly detection device 60 through a dedicated communication channel with the module 62 or through data input/output devices. Data processing instructions may be:

- a simple instruction instructing module 63 to perform an operation of comparing the metric value with at least one threshold value to determine the presence or absence of deviation. For example, for the number of incoming calls, the threshold value can be set to “2000”, and the metric value characterizing the number of incoming calls per minute should not exceed 2000. These instructions are automatically recorded in the module 32 for storing rules for determining deviations;

- compound instructions instructing module 63 to perform a comparison operation of more than one metric value with at least one threshold value specified for all metrics to determine the presence or absence of deviation. For example, within the framework of a composite instruction, a metric value characterizing the number of incoming calls per minute and a metric value characterizing the number of outgoing calls in the same minute can be compared by module 63 with the threshold value of “2000”, and the presence of a deviation will be determined in the case if one of the metric values exceeds the threshold value;

- a data post-processing instruction instructing module 63 to perform flexible aggregation and correlation of data using historical data for instructing module 63 to perform. As part of this post-processing, metric data can be compared to predicted metric values and metric value bounds to determine whether there is an outlier, for example, the number of incoming calls can be compared to the upper and lower bounds of metric values for a given minute. Flexible aggregation and correlation of data can be implemented by equipping the device with 63 specialized counters: a counter for determining the number of deviations over a given period of time, a counter for determining the difference in the values of various metrics per unit of time, a counter for determining the difference in metric values at different observation times, a counter for the sum of metric values for a certain time interval and other counters based on mathematical and logical operations on metric values.

[0066] As for simple instructions, according to the schedule set by the developer, they are written by module 62 to module 32 for storing rules for determining deviations. The remaining instructions, in particular composite and post-processing of data, are sent by the mentioned module 62 to the deviation determination module 63, which, in accordance with the received instructions, checks the metric data stored in the storage module 41.

[0067] Additionally, the deviation determination module 63 may be equipped with a deviation counter and configured to determine the number of deviations over a predetermined period of time in accordance with the deviation determination data processing instructions. For example, in accordance with these instructions, module 63 may retrieve from memory module 41 a metric value, such as the number of calls in a particular minute, and from memory module 42 the upper and lower bounds of metric values defined for said metric, and then perform a comparison of the metric value and the mentioned value limits.

[0068] If the metric value does not exceed the mentioned value limits, then the module 63 proceeds to analyze other metric values stored in the storage module 41. If the metric value goes beyond the mentioned values, then module 63 detects the presence of a deviation and sends a command to the notification generation module 61 to generate a notification about the detected deviation in the metric value, which is then sent to the user information device 70 to notify interested parties about the detected deviation, after whereby module 63 reads the next portion of data from module 41 containing metric data for analysis, and from memory module 42 the upper and lower bounds of metric values defined for said metric.

[0069] If, in the process of comparing the next portion of data, module 63 determines that the metric value does not go beyond the mentioned boundaries, and a deviation was previously detected, then module 63 records the elimination of the deviation, and then sends a command to the notification generation module 61 to generate a notification about eliminating the deviation in the metric value, which is then sent to the device 70 for informing users to notify interested parties about the elimination of the deviation, after which module 63 reads the next portion of data from module 41 containing data on metrics for their analysis, and from memory module 42 the upper and lower bounds of metric values defined for the mentioned metric,

[0070] Additionally, the deviation determination module 63 can be equipped with counters for determining the number of deviations over a given period of time, for determining the difference in the values of various metrics per unit of time, for determining the difference in metric values at different observation times, for determining the sum of metric values for a certain time interval etc.

[0071] For example, in accordance with instructions received from module 62, module 63 can retrieve metric data for a given period of time, for example, characterizing the number of calls in the last 5 minutes, and from memory module 42 the upper and lower bounds of metric values determined for the mentioned metrics, and then compare each metric value characterizing the number of calls per minute with the mentioned boundaries of the metric values. If the metric value goes beyond the mentioned value limits, then module 63 increases the counter value, in particular to determine the number of deviations over a given period of time, by 1.

[0072] After comparing all metric values with said metric value boundaries, module 63 retrieves the counter value and compares it to a counter threshold value, for example, which may be a value of 4 contained in the instruction data.

[0073] If the counter value does not exceed the specified counter threshold value, i.e. the counter indicator is a value from 1 to 3, then the module 63 proceeds to retrieve the next portion of metric data from the memory module 41.

[0074] If the counter value meets a predetermined counter threshold value, for example, the counter value is 4 (i.e., four metric values out of 5 do not meet the upper or lower bounds of the metric values), then module 63 sends a command to notification generation module 61 to generate notification of a detected deviation in the metric value, which is then sent to the user information device 70 to notify interested parties about the detected deviation.

[0075] Also, the module 63 can be configured to detect the presence of a deviation in the metrics data, and the data about the presence of a deviation in the metrics data is entered by the module 63 into a random access memory with which it may be equipped, or into an appropriate register. After the presence of a deviation is detected, module 63 retrieves from the memory of module 41 the next piece of metric data for the next minute or metric data for the past 5 minutes, which is processed in the manner described above by comparing them with the upper and lower limits of the metric values. If, when analyzing the next portion of data using the above-mentioned method, module 63 detects the presence of a deviation in the metric value and the corresponding RAM register contains data on the presence of a deviation, then module 63 will not send a repeated command to notification generation module 61 to generate a notification about the detected deviation.

[0076] If, during the comparison process, module 63 determines that the metric data received over the next minute, or based on the results of processing metric data over the past 5 minutes, the counter value does not exceed a specified threshold value, then module 63 records the elimination of the deviation, and then sends in the notification generation module 61, a command is sent to generate a notification about eliminating the deviation in the metric value, which is then sent to the user information device 70 to notify interested parties about the elimination of the deviation, after which the module 63 proceeds to extracting the next portion of metric data from the memory module 41.

[0077] Additionally, said module 63 may retrieve, for example, a metric value indicative of the number of calls per minute and a metric value indicative of the number of calls per minute at a given point in time in the past, for example, 3 minutes ago. Next, module 63, using a counter to determine the difference in the values of various metrics at different observation times, calculates the difference value in the extracted metric values and compares the resulting difference value with the set threshold value in accordance with the instructions. If a certain difference value determined by said counter exceeds a set threshold value, for example, the difference exceeds 300 units, then module 63 sends a command to notification generation module 61 to generate a notification about a detected deviation in the value of said counter, which is then sent to user information device 70 to notify interested parties of detected deviations.

[0078] Additionally, said module 63 may retrieve metric values for a given time interval and compare each metric value with upper or lower bounds of metric values calculated for said time interval to determine difference values between said values, wherein the determined difference values are recorded by module 63 into a counter. to determine the sum of metric values over a certain time interval. For example, module 63 can compare the values of metrics characterizing the number of calls per minute for the time interval in the last 5 minutes with the lower limits of the values of these metrics. Accordingly, module 63 determines, for each minute of the selected interval, the value of the difference between the lower bound and the metric value, updating the counter value by the determined difference value. After all metric values for a given time interval have been analyzed, module 63 compares the indicator of said counter with the set threshold value in accordance with the instructions. If the counter value exceeds a set threshold value, for example, the difference exceeds 200 units, then the module 63 sends a command to the notification generation module 61 to generate a notification about a detected deviation in the value of the mentioned counter, which is then sent to the device 70 for informing users to notify interested parties about the detected deviation. persons

[0079] Additionally, said module 63 can determine the difference of predetermined metric values in a given unit of time. For example, module 63 can use a counter to determine the difference between the number of calls received by the operator and the bot, and if this difference is less than a threshold value, for example, less than 50, then module 63 sends a command to notification generation module 61 to generate a notification about the detected deviation, which then is sent to the device 70 for informing users to notify interested parties about the detected deviation.

[0080] Additionally, module 34 and module 63 can be configured not to send, at a given time interval, a command to module 61 to generate a notification in relation to those deviations that were previously recorded by them, but not eliminated.

[0081] Thus, due to the presence in the AC monitoring system 100 of a data streaming device (20) containing: an intermediate data storage memory module (21), a data management module (22) for validation, and a data storage module (23) intended for for transmission, filtering of data on metrics that require validation from data that does not require validation is ensured, and therefore errors that may arise when the said system processes unvalidated data are eliminated. Also, due to the presence of a device (30) for stream data processing and a device (60) for detecting deviations, the reliability in detecting deviations is increased if one of the devices fails.

[0082] In general (see Fig. 2), a computing device (200) contains one or more processors (201), memory devices such as RAM (202) and ROM (203), and input/ output (204), input/output devices (205), and network communication devices (206).

[0083] The processor (201) (or multiple processors, multi-core processor, etc.) may be selected from a variety of devices commonly used today, for example, from manufacturers such as: Intel™, AMD™, Apple™, Samsung Exynos ™, MediaTEK™, Qualcomm Snapdragon™, etc. The processor or one of the usable processors in the device (200) must also include a graphics processor, for example an NVIDIA GPU with a CUDA-compatible programming model or Graphcore, the type of which is also suitable for carrying out the method in whole or in part, and can also be used for training and application of machine learning models in various information systems.

[0084] RAM (202) is a random access memory and is designed to store machine-readable instructions executed by the processor (201) to perform the necessary logical data processing operations. The RAM (202) typically contains executable operating system instructions and associated software components (applications, program modules, etc.). In this case, the available memory capacity of the graphics card or graphics processor can act as RAM (202).

[0085] The ROM (203) is one or more permanent storage devices, such as a hard disk drive (HDD), a solid state drive (SSD), flash memory (EEPROM, NAND, etc.), optical storage media ( CD-R/RW, DVD-R/RW, BlueRay Disc, MD), etc.

[0086] To organize the operation of device components (200) and organize the operation of external connected devices, various types of I/O interfaces (204) are used. The choice of appropriate interfaces depends on the specific design of the computing device, which can be, but is not limited to: PCI, AGP, PS/2, IrDa, FireWire, LPT, COM, SATA, IDE, Lightning, USB (2.0, 3.0, 3.1, micro, mini, type C), TRS/Audio jack (2.5, 3.5, 6.35), HDMI, DVI, VGA, Display Port, RJ45, RS232, etc.

[0087] To ensure user interaction with the computing device (200), various means (205) of I/O information are used, for example, a keyboard, a display (monitor), a touch display, a touch pad, a joystick, a mouse, a light pen, a stylus, touch panel, trackball, speakers, microphone, augmented reality tools, optical sensors, tablet, light indicators, projector, camera, biometric identification tools (retina scanner, fingerprint scanner, voice recognition module), etc.

[0088] The network communication means (206) provides data transmission via an internal or external computer network, for example, an Intranet, the Internet, a LAN, etc. One or more means (206) may be used, but not limited to: Ethernet card, GSM modem, GPRS modem, LTE modem, 5G modem, satellite communication module, NFC module, Bluetooth and/or BLE module, Wi-Fi module and etc.

[0089] Additionally, satellite navigation tools can also be used as part of the device (200), for example, GPS, GLONASS, BeiDou, Galileo.

[0090] The specific selection of device elements (200) for implementing various software and hardware architectural solutions may vary while maintaining the required functionality provided.

[0091] Modifications and improvements to the above-described embodiments of the present technical solution will be apparent to those skilled in the art. The foregoing description is provided by way of example only and is not intended to be limiting. Thus, the scope of the present technical solution is limited only by the scope of the attached claims.

Claims (27)

1. System (100) for monitoring automated systems (AS), containing: - data collection device (10), containing: control module (11), module (12) for extracting data from the monitoring object (1) and data conversion module (13); - a device (20) for streaming data, containing: a memory module (21) for intermediate data storage, a data management module (22) for validation, and a module (23) for storing data intended for transmission; - device (30) for stream data processing, containing: a validation control module (31), a module (32) for storing rules for determining deviations, a module (33) for storing validation rules, and a module (34) for determining deviations; - a data storage system (40) containing: a memory module (41) for storing metrics data and a memory module (42) for storing predicted data; - a forecasting device (50) connected to a memory module (41) for storing metrics data and a memory module (42) for storing predicted data; - a device (60) for determining deviations, containing: a module (61) for generating notifications, a module (62) for managing the rules for determining deviations, and a module (63) for determining deviations; - device (70) for informing users; wherein the control module (11) is connected to the data retrieval module (12), the data retrieval module (12) is connected to the monitoring object and the data conversion module (13), the intermediate data storage memory module (21) is connected to the data conversion module (13), and validation data management module (22), the validation data management module (22) is connected to the validation management module (31) and the data storage module (23), the validation control module (31) is connected to the validation rules storage module (33) and the module (23) data storage, module (23) is connected to a module (34) for determining deviations and a memory module (41) for storing metrics data, a module (34) for determining deviations is connected to a module (32) for storing rules for determining deviations and a module (61 ) notification generation, the module (62) for managing the rules for determining deviations is connected to the module (32) for storing the rules for determining deviations, the module (63) for determining deviations is connected to the memory module (41) for storing metrics data, the memory module (42) for storing predicted data, module (62) for managing the rules for determining deviations and module (61) for generating notifications, the module (61) for generating notifications is connected to the device (70) for informing users. 2. The system according to claim 1, characterized in that the data streaming device (20) further comprises a data storage module (24) connected to the monitoring object 1 and the data retrieval module (12). 3. The system according to claim 1, characterized in that the forecasting device (50) contains interconnected: a control module (51), a data processing module (52), a memory module (53) designed for operational data processing, and a model (54) machine learning; wherein the data processing module (52) is connected to a memory module (41) for storing metrics data and a memory module (42) for storing predicted data. 4. The system according to claim 1, characterized in that it additionally contains a data visualization module (43) connected to a memory module (41) for storing metrics data, a memory module (42) for storing predicted data, and a generation module (61). notifications. 5. The system according to claim 1, characterized in that the module (63) for determining deviations is equipped with counters for determining the number of deviations for a given period of time, for determining the difference in the values of various metrics per unit of time, for determining the difference in metric values at different observation times, for determining the sum of metric values over a certain time interval. 6. The system according to claim 1, characterized in that the data streaming device (20) additionally contains a memory area intended for storing data on metrics that have not passed the validation procedure. 7. A method for automatic monitoring of speakers, performed by the module (63) for determining deviations, containing stages in which: - receive instructions to determine deviations; - extracting from memory, according to the mentioned instructions, data on metrics for a given period of time; - extract from memory, according to the mentioned instructions, the upper and lower limits of the metric values defined for the mentioned metrics; - each metric value is compared with the mentioned limits of the metric values, and if the metric value goes beyond the mentioned limits of values, then the module (63) increases the counter indicator by 1; - after comparing all metric values with the mentioned boundaries of metric values, the counter indicator is extracted and compared with the threshold value of the counter specified in the instruction, and if the counter value exceeds or corresponds to the specified threshold value of the counter, then a command for generating notifications about detected deviations in the metric value. 8. The method according to claim 7, characterized in that the step of recording the presence of deviations in the metrics data is additionally performed. 9. The method according to claim 8, characterized in that additional steps are performed in which: - extracting from memory, according to the mentioned instructions, data on metrics for the next period of time; - extract from memory, according to the mentioned instructions, the upper and lower limits of the metric values defined for the mentioned metrics; - each metric value is compared with the mentioned limits of the metric values, and if the metric value goes beyond the mentioned limits of values, then the module (63) increases the counter indicator by 1; - after comparing all metric values with the mentioned boundaries of metric values, the counter indicator is extracted and compared with the threshold value of the counter specified in the instruction, and if the counter value does not exceed the specified threshold value of the counter, then the elimination of the deviation is recorded and generated in the module (61) for generating notifications command for generating a notification about eliminating a deviation in the metric value.

Images