CN112612929A - Data processing method and device - Google Patents
Data processing method and device Download PDFInfo
- Publication number
- CN112612929A CN112612929A CN202011600573.XA CN202011600573A CN112612929A CN 112612929 A CN112612929 A CN 112612929A CN 202011600573 A CN202011600573 A CN 202011600573A CN 112612929 A CN112612929 A CN 112612929A
- Authority
- CN
- China
- Prior art keywords
- target
- index data
- data
- index
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/90335—Query processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
Abstract
The application provides a data processing method and a device, wherein the data processing method comprises the following steps: acquiring an index dataset of at least one operating module, wherein the index dataset comprises: at least one piece of index data; under the condition that target index data exist in the index data set, inquiring at least one piece of second index data related to the target index data, and determining the target second index data in the at least one piece of second index data; matching the target index data and the target second index data with abnormal rules in a preset rule base; and determining a processing strategy of a target operation module in the at least one operation module based on the matching result, wherein the target operation module is determined based on the target index data.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data processing method and apparatus, a computing device, and a computer-readable storage medium.
Background
With the development of the internet, the complexity of the service is higher and higher, and in order to provide a better service to the user, more and more servers are used to support the service operation, and the service experience of the user on the service is affected when the performance problem occurs in the operation module of any one server. However, since the performance index in the server is too many, when the server has a performance problem, the information to be searched is also extremely huge, so that it is difficult to make a correct problem diagnosis and provide a solution, and the labor cost and time cost consumed in the problem solution process are also high, and therefore, it is desirable to provide a data processing method for solving the above problem.
Disclosure of Invention
In view of this, embodiments of the present application provide a data processing method and apparatus, a computing device, and a computer-readable storage medium, so as to solve technical defects in the prior art.
According to a first aspect of embodiments of the present application, there is provided a data processing method, including:
acquiring an index dataset of at least one operating module, wherein the index dataset comprises: at least one piece of index data;
under the condition that target index data exist in the index data set, inquiring at least one piece of second index data related to the target index data, and determining the target second index data in the at least one piece of second index data;
matching the target index data and the target second index data with abnormal rules in a preset rule base;
and determining a processing strategy of a target operation module in the at least one operation module based on the matching result, wherein the target operation module is determined based on the target index data.
Optionally, the querying at least one piece of second index data associated with the target index data and determining the target second index data in the at least one piece of second index data in the case that the target index data exists in the index data set includes:
judging whether the at least one index data does not meet the corresponding preset index condition;
if so, determining the index data which does not meet the preset index condition as the target index data;
inquiring at least one piece of second index data related to the target index data according to the target index data;
and taking second index data which does not meet a second preset index condition in the at least one piece of second index data as target second index data.
Optionally, the matching the target index data and the target second index data with an abnormal rule in a preset rule base includes:
similarity matching is carried out on the target index data and the target second index data and abnormal rules in a preset rule base, and target matching degree is obtained;
and determining the abnormal rule corresponding to the target matching degree which is greater than the threshold value of the matching degree as the target abnormal rule corresponding to the target index data and the target second index data, and taking the target abnormal rule as a matching result.
Optionally, after determining the processing policy of the target operating module in the at least one operating module based on the matching result, the method further includes:
determining a reach mode corresponding to the processing strategy;
and sending an exception notification aiming at the target operation module based on the touch manner.
Optionally, the exception notification includes: the target metric data, the target second metric data, and the processing policy.
Optionally, after determining the processing policy of the target operating module in the at least one operating module based on the matching result, the method further includes:
and under the condition that the processing priority contained in the processing strategy is lower than the preset priority, processing the target operation module according to the processing strategy.
Optionally, after determining the processing policy for the target operating module, the method further includes:
recording a target processing strategy determined based on the processing strategy;
and under the condition that the processing strategy is inconsistent with the target processing strategy, adding the target index data and the target second index data serving as first abnormal rules to the preset rule base, and using the target processing strategy as a processing strategy corresponding to the first abnormal rules.
According to a second aspect of embodiments of the present application, there is provided a data processing apparatus including:
an acquisition module configured to acquire an index dataset of at least one operating module, wherein the index dataset comprises: at least one piece of index data;
a query module configured to query at least one piece of second index data associated with target index data in a case where the target index data exists in the index data set, and determine target second index data in the at least one piece of second index data;
the matching module is configured to match the target index data and the target second index data with abnormal rules in a preset rule base;
a determination module configured to determine a processing strategy of a target operation module of the at least one operation module based on the matching result, wherein the target operation module is determined based on the target index data.
Optionally, the data processing apparatus further includes:
a recording module configured to record a target processing policy determined based on the processing policy;
and the adding module is configured to add the target index data and the target second index data to the preset rule base as first abnormal rules under the condition that the processing strategy is inconsistent with the target processing strategy, and take the target processing strategy as a processing strategy corresponding to the first abnormal rules.
According to a third aspect of embodiments herein, there is provided a computing device comprising a memory, a processor and computer instructions stored on the memory and executable on the processor, the processor implementing the steps of the data processing method when executing the computer instructions.
According to a fourth aspect of embodiments of the present application, there is provided a computer-readable storage medium storing computer instructions which, when executed by a processor, implement the steps of the data processing method.
The data processing method provided by the embodiment of the application acquires an index data set of at least one operation module, wherein the index data set comprises: at least one piece of index data; under the condition that target index data exist in the index data set, inquiring at least one piece of second index data related to the target index data, and determining the target second index data in the at least one piece of second index data; matching the target index data and the target second index data with abnormal rules in a preset rule base; and determining a processing strategy of a target operation module in the at least one operation module based on a matching result, wherein the target operation module is determined based on the target index data, so that the purpose of analyzing the acquired index data, determining the target index data firstly, and further determining the target second index data associated with the target index data through query is realized, the purpose of determining the index data twice is realized, a large amount of index data is prevented from being acquired or queried at one time, the efficiency of judging the abnormality of the index data is improved, the abnormality is quickly positioned by matching the abnormal rule of the index data, and the efficiency of recovering the abnormality is improved by determining the processing strategy corresponding to the abnormality.
Drawings
FIG. 1 is a block diagram of a computing device provided by an embodiment of the present application;
fig. 2 is a flowchart of a data processing method provided in an embodiment of the present application;
fig. 3 is a flowchart of a data processing method applied in a server according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application.
Detailed Description
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is capable of implementation in many different ways than those herein set forth and of similar import by those skilled in the art without departing from the spirit of this application and is therefore not limited to the specific implementations disclosed below.
The terminology used in the one or more embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of the present application. As used in one or more embodiments of the present application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present application refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It will be understood that, although the terms first, second, etc. may be used herein in one or more embodiments of the present application to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first aspect may be termed a second aspect, and, similarly, a second aspect may be termed a first aspect, without departing from the scope of one or more embodiments of the present application. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
In the present application, a data processing method and apparatus, a computing device, and a computer-readable storage medium are provided, and detailed descriptions are individually made in the following embodiments.
FIG. 1 shows a block diagram of a computing device 100 according to an embodiment of the present application. The components of the computing device 100 include, but are not limited to, memory 110 and processor 120. The processor 120 is coupled to the memory 110 via a bus 130 and a database 150 is used to store data.
Computing device 100 also includes access device 140, access device 140 enabling computing device 100 to communicate via one or more networks 160. Examples of such networks include the Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the internet. Access device 140 may include one or more of any type of network interface (e.g., a Network Interface Card (NIC)) whether wired or wireless, such as an IEEE802.11 Wireless Local Area Network (WLAN) wireless interface, a worldwide interoperability for microwave access (Wi-MAX) interface, an ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a bluetooth interface, a Near Field Communication (NFC) interface, and so forth.
In one embodiment of the present application, the above-mentioned components of the computing device 100 and other components not shown in fig. 1 may also be connected to each other, for example, by a bus. It should be understood that the block diagram of the computing device architecture shown in FIG. 1 is for purposes of example only and is not limiting as to the scope of the present application. Those skilled in the art may add or replace other components as desired.
Computing device 100 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), a mobile phone (e.g., smartphone), a wearable computing device (e.g., smartwatch, smartglasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. Computing device 100 may also be a mobile or stationary server.
Wherein the processor 120 may perform the steps of one of the data processing methods shown in fig. 2. Fig. 2 shows a flowchart of a data processing method according to an embodiment of the present application, which specifically includes the following steps:
step 202: acquiring an index dataset of at least one operating module, wherein the index dataset comprises: at least one piece of index data.
Specifically, the operation module may be one or more hardware devices or function modules such as a CPU, a disk, a memory, an Input/Output (I/O), an I/O (Input/Output), a network, and the like, and accordingly, the index data may be understood as data indicating an operation state and/or a performance of the operation module, and in specific implementation, the index data corresponding to each hardware device or function module may be acquired through a system monitoring command (such as a netstat command, vmstat, pidstat, iostat, and the like) or a monitoring tool, and the like.
Taking a CPU as an example, the index data of the CPU includes: CPU utilization rate, CPU average load, CPU context switching, CPU zombie process number, interruption number and the like.
Taking a magnetic disk as an example, index data of a hard disk includes: the available amount of the disk, the total size of the disk, the percentage of the available amount of the disk in the total amount and the like.
Step 204: and under the condition that target index data exist in the index data set, inquiring at least one piece of second index data related to the target index data, and determining the target second index data in the at least one piece of second index data.
In practical application, under the condition that the running module runs normally, the corresponding index data is in a reasonable range, and if the range is exceeded, it indicates that the running module, the server where the running module is located, and/or the application program running in the server and the like may be abnormal, for example, the usage rate of the CPU exceeds 90%.
The second index data refers to index data having a relation with the above abnormal index data, and in particular, in the case of an abnormal index data, the index data may be caused by other more detailed index data associated with the abnormal index data, so that on the basis of determining the abnormal index data (i.e., the target index data), it is necessary to further determine other index data related to the abnormal index data, so as to better analyze the reason for the abnormality of the target index data. Accordingly, the target second index data may be understood as abnormal index data in the second index data, and specifically, one or more target second index data may be used.
In specific implementation, when there is target index data in the index data set, at least one piece of second index data associated with the target index data is queried, and the target second index data in the at least one piece of second index data is determined, which is specifically implemented by the following method:
judging whether the at least one index data does not meet the corresponding preset index condition;
if so, determining the index data which does not meet the preset index condition as the target index data;
inquiring at least one piece of second index data related to the target index data according to the target index data;
and taking second index data which does not meet a second preset index condition in the at least one piece of second index data as target second index data.
The preset index condition can be understood as a condition of normal operation of the operation module, a preset reasonable value range of each index data is determined, if the index data does not satisfy the corresponding preset index condition, it indicates that the index data is abnormal, the index data which does not satisfy the corresponding preset index condition is determined as target index data (namely abnormal index data), for example, the preset index condition corresponding to the CPU utilization rate is 5% < CPU utilization rate < 90%, and if the collected CPU utilization rate is 99%, that is, the collected CPU utilization rate does not satisfy the preset index condition, the CPU utilization rate is 99% determined as the target index data.
Similarly, the second preset index condition may be understood as a reasonable value range of each second index data preset for a normal operation condition of the operation module, and if the index data does not satisfy the corresponding second preset index condition, indicating that the second index data is abnormal, the second index data that does not satisfy the corresponding second preset index condition is determined as the target second index data (i.e., the abnormal second index data).
According to the embodiment of the application, the index data in normal operation and the reasonable range (condition) of the second index data are preset, whether the index data meet the preset condition or not is judged, abnormal index data in the index data are determined, whether the second index data meet the second preset condition or not is judged, the abnormal index data in the second index data are determined, and the positioning and screening of the abnormal index data are achieved, so that the performance problem of an operation module is positioned based on the abnormal index data.
Step 206: and matching the target index data and the target second index data with abnormal rules in a preset rule base.
Specifically, the preset rule base may be understood as a database, a data table, a file, or the like storing the exception rule, and is not limited herein, where the exception rule refers to a determination basis for determining the exception, and specifically, an exception rule may also be understood as a set of exception indicator data (including the exception indicator data and also including the exception second indicator data) collected or recorded for one exception, and in practical application, by matching the target indicator data and the target second indicator data with the exception rule in the preset rule base, which exception rule corresponds to the current exception may be determined.
Specifically, the matching of the target index data and the target second index data with the abnormal rule in the preset rule base is specifically realized by adopting the following method:
similarity matching is carried out on the target index data and the target second index data and abnormal rules in a preset rule base, and target matching degree is obtained;
and determining the abnormal rule corresponding to the target matching degree which is greater than the threshold value of the matching degree as the target abnormal rule corresponding to the target index data and the target second index data, and taking the target abnormal rule as a matching result.
The matching degree threshold is a preset numerical value for judging whether abnormal index data (target index data and the target second index data) are matched with (similar to) the abnormal rule, if the target matching degree is greater than the matching degree threshold, the abnormal index data are similar to the abnormal rule, the abnormal rule corresponding to the target matching degree is determined as the target matching rule corresponding to the abnormal index data, and if the target matching degree is less than or equal to the matching degree threshold, the abnormal index data are not similar to the abnormal rule, and no operation is required.
It should be noted that, if there are multiple matched exception rules (i.e., the target matching degree is greater than the matching degree threshold) with the target index data and the target second index data, the exception rule with the highest target matching degree may be selected from the multiple matched exception rules as the target exception rule, i.e., the matching result.
In specific implementation, because some slight differences may exist between the abnormal index data and the abnormal rules in the preset rule base, but the abnormal index data still belongs to the same type of abnormal condition, similarity matching is performed between the abnormal index data and the abnormal rules, and the abnormal rules similar to the abnormal index data are used as target abnormal rules, so that data errors possibly existing are ignored, and matching (classification) of the same type of abnormal conditions is realized.
Step 208: and determining the processing strategy of the target operation module in the at least one operation module based on the matching result.
The target operation module is determined based on the target index data, and specifically, the target operation module refers to an operation module in which an abnormality occurs in the at least one operation module, and by determining which operation module the abnormal index data is the index data of which operation module, it can be determined which operation module has an abnormality.
Specifically, the processing policy may be understood as a cause (such as an exception function, an exception application, and the like) causing the exception, a repair instruction/repair step for the exception, and/or a historical repair step for the exception, and the processing policy may be stored in the preset rule base together with the exception rule and the processing policy corresponding to the exception rule in specific implementation.
Further, after determining the processing policy of the target operating module in the at least one operating module based on the matching result, the method further includes:
determining a reach mode corresponding to the processing strategy;
and sending an exception notification aiming at the target operation module based on the touch manner.
Specifically, the touch manner may be a mail, a short message, a telephone, or the like, which is not limited herein, and it should be noted that after the touch manner is determined, a contact manner of the touched person corresponding to the touch manner needs to be determined, and an abnormal notification for the target operation module is sent to the touched person based on the touch manner and the contact manner corresponding to the touch manner.
In practical application, the reach mode may be determined according to the influence degree or severity of the abnormality, that is, the reach mode may be determined according to the reach indication in the processing policy or the corresponding relationship between the processing policy and the reach mode, for example, in the case that the influence degree of the abnormality is strong or severe, the reach mode may be performed by a telephone, for example, in the case that the influence degree of the abnormality is weak or the abnormality is not severe, the reach mode may be performed by a mail, and the like.
In the embodiment of the application, after the processing strategy for the abnormal operation module is determined, in order to timely process the abnormal situation, an abnormal notification needs to be sent to the relevant personnel, so that the abnormal situation is explained to the relevant personnel.
Optionally, the exception notification includes: the target index data, the target second index data and the processing strategy are used as notification information in an exception notification, and in order to accelerate the processing speed of the touched personnel on the exception, the exception notification comprises the determined exception index data and the processing strategy determined aiming at the exception index data, so that the exception is quickly positioned, and the exception recovery speed is accelerated.
In addition, after determining the processing policy of the target running module in the at least one running module based on the matching result, the method further includes:
and under the condition that the processing priority contained in the processing strategy is lower than the preset priority, processing the target operation module according to the processing strategy.
Specifically, under the condition of a low processing priority, it is indicated that the exception is not serious, the exception is simple to repair, or even if the exception judgment is wrong, no great influence is caused, the recovery instruction included in the processing strategy can be directly executed, the exception recovery processing is performed, manual participation is not required, and the exception processing efficiency is also improved.
Further, after determining the processing policy for the target operation module, the method further includes:
recording a target processing strategy determined based on the processing strategy;
and under the condition that the processing strategy is inconsistent with the target processing strategy, adding the target index data and the target second index data serving as first abnormal rules to the preset rule base, and using the target processing strategy as a processing strategy corresponding to the first abnormal rules.
In practical application, due to the complexity of the index data, the reasons for causing the abnormality of the index data are also various, so after the processing strategy is determined, in the process of performing the abnormality processing by the abnormality processing personnel according to the processing strategy, if the actual reason for generating the abnormality and/or the abnormality processing scheme (namely, the target processing strategy) may be found to be inconsistent with the determined processing strategy, the actual reason for generating the abnormality and/or the abnormality processing scheme at this time are recorded.
Specifically, in order to increase the accuracy of the later-stage exception handling, after the exception is handled, the actual handling manner or the cause (namely, the target handling policy) for the exception handling is recorded, the exception index data is used as the exception rule and is added to the preset rule base, and the exception rule is associated with the corresponding target handling policy, so that the exception data and the cause corresponding to the exception data are added to the deep learning machine for cumulative learning, and the subsequent exception analysis and handling capability is improved.
That is, the performance monitoring system of the present application has an intelligent feature, and analyzes performance data (index data) in a deep learning manner. And recording and learning which problems are caused by the existing parameters (index data), and forming output by summarizing experience after learning. The system can automatically analyze and judge a plurality of complex parameters related to the current operation, realize intelligent analysis and judgment and find problems, thereby realizing real-time monitoring and finding the problems at the first time when the problems occur. And the system can continuously perform feedback iteration, so that the analysis and judgment capability of the system is continuously improved and the system becomes more intelligent.
To sum up, in the data processing method provided in the embodiment of the present application, an index data set of at least one operation module is collected, where the index data set includes: at least one piece of index data; under the condition that target index data exist in the index data set, inquiring at least one piece of second index data related to the target index data, and determining the target second index data in the at least one piece of second index data; matching the target index data and the target second index data with abnormal rules in a preset rule base; and determining a processing strategy of a target operation module in the at least one operation module based on a matching result, wherein the target operation module is determined based on the target index data, so that the purpose of analyzing the acquired index data, determining the target index data firstly, and further determining the target second index data associated with the target index data through query is realized, the purpose of determining the index data twice is realized, a large amount of index data is prevented from being acquired or queried at one time, the efficiency of judging the abnormality of the index data is improved, the abnormality is quickly positioned by matching the abnormal rule of the index data, and the efficiency of recovering the abnormality is improved by determining the processing strategy corresponding to the abnormality.
The following describes a data processing method further by taking an application of the data processing method in a server as an example with reference to fig. 3. Fig. 3 shows a flowchart of a data processing method applied to a server according to an embodiment of the present application, which specifically includes the following steps:
In practical application, the scheme can be implemented as a monitoring system for monitoring the abnormality in the server, specifically, the monitoring system may be a network performance monitoring system or a service monitoring system according to different monitoring emphasis points, and is not limited herein.
if yes, indicating that abnormal index data exists, executing the following step 306;
if not, indicating that no abnormal index data exists, and not processing.
And step 310, taking the second index data which does not meet the second preset index condition in the at least one piece of second index data as target second index data.
And step 312, performing similarity matching on the target index data and the target second index data and abnormal rules in a preset rule base to obtain a target matching degree.
And step 316, determining a processing strategy of a target operation module in the at least one operation module based on the matching result, wherein the target operation module is determined based on the target index data.
At step 322, a target processing policy determined based on the processing policy is recorded.
To sum up, in the data processing method provided in the embodiment of the present application, an index data set of at least one operation module in a server is collected, where the index data set includes: at least one piece of index data; under the condition that target index data exist in the index data set, inquiring at least one piece of second index data related to the target index data, and determining the target second index data in the at least one piece of second index data; matching the target index data and the target second index data with abnormal rules in a preset rule base; and determining a processing strategy of a target operation module in the at least one operation module based on a matching result, wherein the target operation module is determined based on the target index data, so that the purpose of analyzing the acquired index data, determining the target index data firstly, and further determining the target second index data associated with the target index data through query is realized, the purpose of determining the index data twice is realized, a large amount of index data is prevented from being acquired or queried at one time, the efficiency of judging the abnormality of the index data is improved, the abnormality is quickly positioned by matching the abnormal rule of the index data, and the efficiency of recovering the abnormality in the server is improved by determining the processing strategy corresponding to the abnormality.
Corresponding to the above data processing method embodiment, the present application further provides an embodiment of a data processing apparatus, and fig. 4 shows a schematic structural diagram of the data processing apparatus provided in the embodiment of the present application. As shown in fig. 4, the apparatus includes:
an acquisition module 402 configured to acquire an index dataset of at least one operating module, wherein the index dataset comprises: at least one piece of index data;
a query module 404 configured to query at least one piece of second index data associated with the target index data in a case where the target index data exists in the index data set, and determine target second index data in the at least one piece of second index data;
a matching module 406 configured to match the target index data and the target second index data with an abnormal rule in a preset rule base;
a determining module 408 configured to determine a processing strategy of a target operation module of the at least one operation module based on the matching result, wherein the target operation module is determined based on the target index data.
Optionally, the query module 404 is further configured to:
judging whether the at least one index data does not meet the corresponding preset index condition;
if so, determining the index data which does not meet the preset index condition as the target index data;
inquiring at least one piece of second index data related to the target index data according to the target index data;
and taking second index data which does not meet a second preset index condition in the at least one piece of second index data as target second index data.
Optionally, the matching module 406 is further configured to:
similarity matching is carried out on the target index data and the target second index data and abnormal rules in a preset rule base, and target matching degree is obtained;
and determining the abnormal rule corresponding to the target matching degree which is greater than the threshold value of the matching degree as the target abnormal rule corresponding to the target index data and the target second index data, and taking the target abnormal rule as a matching result.
Optionally, the data processing apparatus further includes:
a determining mode module configured to determine a reach mode corresponding to the processing policy;
a sending module configured to send an exception notification for the target run module based on the reach style.
Optionally, the exception notification includes: the target metric data, the target second metric data, and the processing policy.
Optionally, the data processing apparatus further includes:
and the processing module is configured to process the target operation module according to the processing strategy under the condition that the processing priority contained in the processing strategy is lower than a preset priority.
Optionally, the data processing apparatus further includes:
a recording module configured to record a target processing policy determined based on the processing policy;
and the adding module is configured to add the target index data and the target second index data to the preset rule base as first abnormal rules under the condition that the processing strategy is inconsistent with the target processing strategy, and take the target processing strategy as a processing strategy corresponding to the first abnormal rules.
To sum up, the data processing apparatus provided in the embodiment of the present application collects an index data set of at least one operation module, where the index data set includes: at least one piece of index data; under the condition that target index data exist in the index data set, inquiring at least one piece of second index data related to the target index data, and determining the target second index data in the at least one piece of second index data; matching the target index data and the target second index data with abnormal rules in a preset rule base; and determining a processing strategy of a target operation module in the at least one operation module based on a matching result, wherein the target operation module is determined based on the target index data, so that the purpose of analyzing the acquired index data, determining the target index data firstly, and further determining the target second index data associated with the target index data through query is realized, the purpose of determining the index data twice is realized, a large amount of index data is prevented from being acquired or queried at one time, the efficiency of judging the abnormality of the index data is improved, the abnormality is quickly positioned by matching the abnormal rule of the index data, and the efficiency of recovering the abnormality is improved by determining the processing strategy corresponding to the abnormality.
The above is a schematic configuration of a data processing apparatus of the present embodiment. It should be noted that the technical solution of the data processing apparatus and the technical solution of the data processing method described above belong to the same concept, and details that are not described in detail in the technical solution of the data processing apparatus can be referred to the description of the technical solution of the data processing method described above.
An embodiment of the present application further provides a computing device, which includes a memory, a processor, and computer instructions stored in the memory and executable on the processor, and when the processor executes the computer instructions, the steps of the data processing method are implemented.
An embodiment of the present application further provides a computer readable storage medium, which stores computer instructions, and the computer instructions, when executed by a processor, implement the steps of the data processing method as described above.
The above is an illustrative scheme of a computer-readable storage medium of the present embodiment. It should be noted that the technical solution of the storage medium belongs to the same concept as the technical solution of the data processing method, and details that are not described in detail in the technical solution of the storage medium can be referred to the description of the technical solution of the data processing method.
The foregoing description of specific embodiments of the present application has been presented. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The computer instructions comprise computer program code which may be in the form of source code, object code, an executable file or some intermediate form, or the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
It should be noted that, for the sake of simplicity, the above-mentioned method embodiments are described as a series of acts or combinations, but those skilled in the art should understand that the present application is not limited by the described order of acts, as some steps may be performed in other orders or simultaneously according to the present application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
The preferred embodiments of the present application disclosed above are intended only to aid in the explanation of the application. Alternative embodiments are not exhaustive and do not limit the invention to the precise embodiments described. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the application and its practical applications, to thereby enable others skilled in the art to best understand and utilize the application. The application is limited only by the claims and their full scope and equivalents.
Claims (11)
1. A data processing method, comprising:
acquiring an index dataset of at least one operating module, wherein the index dataset comprises: at least one piece of index data;
under the condition that target index data exist in the index data set, inquiring at least one piece of second index data related to the target index data, and determining the target second index data in the at least one piece of second index data;
matching the target index data and the target second index data with abnormal rules in a preset rule base;
and determining a processing strategy of a target operation module in the at least one operation module based on the matching result, wherein the target operation module is determined based on the target index data.
2. The data processing method according to claim 1, wherein, in a case where target index data exists in the index data set, querying at least one piece of second index data associated with the target index data, and determining the target second index data in the at least one piece of second index data, comprises:
judging whether the at least one index data does not meet the corresponding preset index condition;
if so, determining the index data which does not meet the preset index condition as the target index data;
inquiring at least one piece of second index data related to the target index data according to the target index data;
and taking second index data which does not meet a second preset index condition in the at least one piece of second index data as target second index data.
3. The data processing method according to claim 1, wherein the matching the target index data and the target second index data with an abnormal rule in a preset rule base comprises:
similarity matching is carried out on the target index data and the target second index data and abnormal rules in a preset rule base, and target matching degree is obtained;
and determining the abnormal rule corresponding to the target matching degree which is greater than the threshold value of the matching degree as the target abnormal rule corresponding to the target index data and the target second index data, and taking the target abnormal rule as a matching result.
4. The data processing method according to claim 1, wherein after determining the processing policy of the target running module of the at least one running module based on the matching result, the method further comprises:
determining a reach mode corresponding to the processing strategy;
and sending an exception notification aiming at the target operation module based on the touch manner.
5. The data processing method of claim 4, wherein the exception notification comprises: the target metric data, the target second metric data, and the processing policy.
6. The data processing method according to claim 1, wherein after determining the processing policy of the target running module of the at least one running module based on the matching result, the method further comprises:
and under the condition that the processing priority contained in the processing strategy is lower than the preset priority, processing the target operation module according to the processing strategy.
7. The data processing method of claim 1, wherein after determining the processing policy for the target run module, further comprising:
recording a target processing strategy determined based on the processing strategy;
and under the condition that the processing strategy is inconsistent with the target processing strategy, adding the target index data and the target second index data serving as first abnormal rules to the preset rule base, and using the target processing strategy as a processing strategy corresponding to the first abnormal rules.
8. A data processing apparatus, comprising:
an acquisition module configured to acquire an index dataset of at least one operating module, wherein the index dataset comprises: at least one piece of index data;
a query module configured to query at least one piece of second index data associated with target index data in a case where the target index data exists in the index data set, and determine target second index data in the at least one piece of second index data;
the matching module is configured to match the target index data and the target second index data with abnormal rules in a preset rule base;
a determination module configured to determine a processing strategy of a target operation module of the at least one operation module based on the matching result, wherein the target operation module is determined based on the target index data.
9. The data processing apparatus of claim 8, further comprising:
a recording module configured to record a target processing policy determined based on the processing policy;
and the adding module is configured to add the target index data and the target second index data to the preset rule base as first abnormal rules under the condition that the processing strategy is inconsistent with the target processing strategy, and take the target processing strategy as a processing strategy corresponding to the first abnormal rules.
10. A computing device comprising a memory, a processor, and computer instructions stored on the memory and executable on the processor, wherein the processor implements the steps of the method of any one of claims 1 to 7 when executing the computer instructions.
11. A computer-readable storage medium storing computer instructions, which when executed by a processor implement the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011600573.XA CN112612929A (en) | 2020-12-29 | 2020-12-29 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011600573.XA CN112612929A (en) | 2020-12-29 | 2020-12-29 | Data processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112612929A true CN112612929A (en) | 2021-04-06 |
Family
ID=75248955
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011600573.XA Pending CN112612929A (en) | 2020-12-29 | 2020-12-29 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112612929A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113762765A (en) * | 2021-09-02 | 2021-12-07 | 南方电网数字电网研究院有限公司 | Index analysis method, index analysis system, electronic device and storage medium |
| CN114493372A (en) * | 2022-03-31 | 2022-05-13 | 一道新能源科技(衢州)有限公司 | Data collection method and system for solar cell big data analysis |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107562556A (en) * | 2017-08-14 | 2018-01-09 | 腾讯科技（深圳）有限公司 | Restoration methods, recovery device and the storage medium of failure |
| CN107707376A (en) * | 2017-06-09 | 2018-02-16 | 贵州白山云科技有限公司 | A kind of method and system for monitoring and alerting |
| CN107957988A (en) * | 2016-10-18 | 2018-04-24 | 阿里巴巴集团控股有限公司 | For determining the method, apparatus and electronic equipment of data exception reason |
| CN111641519A (en) * | 2020-04-30 | 2020-09-08 | 平安科技（深圳）有限公司 | Abnormal root cause positioning method, device and storage medium |
-
2020
- 2020-12-29 CN CN202011600573.XA patent/CN112612929A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107957988A (en) * | 2016-10-18 | 2018-04-24 | 阿里巴巴集团控股有限公司 | For determining the method, apparatus and electronic equipment of data exception reason |
| CN107707376A (en) * | 2017-06-09 | 2018-02-16 | 贵州白山云科技有限公司 | A kind of method and system for monitoring and alerting |
| CN107562556A (en) * | 2017-08-14 | 2018-01-09 | 腾讯科技（深圳）有限公司 | Restoration methods, recovery device and the storage medium of failure |
| CN111641519A (en) * | 2020-04-30 | 2020-09-08 | 平安科技（深圳）有限公司 | Abnormal root cause positioning method, device and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113762765A (en) * | 2021-09-02 | 2021-12-07 | 南方电网数字电网研究院有限公司 | Index analysis method, index analysis system, electronic device and storage medium |
| CN114493372A (en) * | 2022-03-31 | 2022-05-13 | 一道新能源科技(衢州)有限公司 | Data collection method and system for solar cell big data analysis |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR20210019564A (en) | Operation maintenance system and method | |
| CN108521339B (en) | Feedback type node fault processing method and system based on cluster log | |
| CN112612929A (en) | Data processing method and device | |
| CN113313280B (en) | Cloud platform inspection method, electronic equipment and nonvolatile storage medium | |
| CN112214465A (en) | Log storage system and method | |
| US11757701B2 (en) | Method for recommending similar incident, and related device | |
| TW202016805A (en) | System and method of learning-based prediction for anomalies within a base station | |
| CN113516244A (en) | Intelligent operation and maintenance method and device, electronic equipment and storage medium | |
| CN111147306B (en) | Fault analysis method and device of Internet of things equipment and Internet of things platform | |
| CN113297269A (en) | Data query method and device | |
| CN111800290A (en) | Method, system and equipment for determining root cause | |
| CN114610561A (en) | System monitoring method, device, electronic equipment and computer readable storage medium | |
| CN116382224B (en) | Packaging equipment monitoring method and system based on data analysis | |
| CN112565523A (en) | Method and device for automatically processing popup window | |
| JP7339321B2 (en) | Machine learning model update method, computer program and management device | |
| CN110488772B (en) | DCS centralized monitoring method and device and centralized monitoring terminal | |
| CN113778831A (en) | Data application performance analysis method, device, equipment and medium | |
| CN111722977A (en) | System inspection method and device and electronic equipment | |
| CN113610225A (en) | Quality evaluation model training method and device, electronic equipment and storage medium | |
| CN113238888A (en) | Data processing method, system and device | |
| CN105897503A (en) | Hadoop cluster bottleneck detection algorithm based on resource information gain | |
| CN117056663B (en) | Data processing method and device, electronic equipment and storage medium | |
| CN116909848A (en) | Process processing method and device | |
| CN115858324B (en) | AI-based IT equipment fault processing method, apparatus, equipment and medium | |
| CN113296870B (en) | Method and device for predicting Kubernetes cluster configuration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 519000 Room 102, 202, 302 and 402, No. 325, Qiandao Ring Road, Tangjiawan Town, high tech Zone, Zhuhai City, Guangdong Province, Room 102 and 202, No. 327 and Room 302, No. 329 Applicant after: Zhuhai Jinshan Digital Network Technology Co.,Ltd. Applicant after: Zhuhai Xishanju Digital Technology Co.,Ltd. Address before: 519000 Room 102, 202, 302 and 402, No. 325, Qiandao Ring Road, Tangjiawan Town, high tech Zone, Zhuhai City, Guangdong Province, Room 102 and 202, No. 327 and Room 302, No. 329 Applicant before: ZHUHAI KINGSOFT ONLINE GAME TECHNOLOGY Co.,Ltd. Applicant before: ZHUHAI SEASUN MOBILE GAME TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information |