CN110442498B - Abnormal data node positioning method and device, storage medium and computer equipment - Google Patents
Abnormal data node positioning method and device, storage medium and computer equipment Download PDFInfo
- Publication number
- CN110442498B CN110442498B CN201910576944.6A CN201910576944A CN110442498B CN 110442498 B CN110442498 B CN 110442498B CN 201910576944 A CN201910576944 A CN 201910576944A CN 110442498 B CN110442498 B CN 110442498B
- Authority
- CN
- China
- Prior art keywords
- log data
- abnormal data
- abnormal
- code
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3438—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment monitoring of user actions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3604—Software analysis for verifying properties of programs
- G06F11/3612—Software analysis for verifying properties of programs by runtime analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0633—Workflow analysis
Abstract
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for locating an abnormal data node, a storage medium, and a computer device. The positioning method of the abnormal data node comprises the following steps: acquiring log data for recording access behaviors, and binding the access behaviors with the log data according to a timestamp; comparing the operating parameters in the log data with a set threshold, and if the log data exceeds the range of the set threshold, judging the corresponding log data as abnormal data; and acquiring a time stamp corresponding to the abnormal data, acquiring the access behavior according to the time stamp, and determining the position of the abnormal data node according to the access behavior. According to the scheme, the position of the abnormal data node is determined by continuously comparing the operating parameters in the log data with the set threshold value, the problem that the positioning of the abnormal code consumes long time is solved, and the position of the abnormal data node can be rapidly and accurately positioned.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for locating an abnormal data node, a storage medium, and a computer device.
Background
As application systems become more complex, a business process often needs to be completed across multiple business systems. For example, a purchase business process of a consumer on an e-commerce platform may involve a merchandise display business system, an order business system, a payment business system, an inventory management business system, a logistics business system, etc., each business system corresponding to a business involved in the business process. In order to realize comprehensive and efficient monitoring of the business process, a plurality of systems need to be spanned, and the data of the whole business call chain needs to be tracked. The operation mode of the cross-system causes the bottom layer codes to need to be mutually called and jumped, the code data increases suddenly, once a certain section of code in a certain system is abnormal, if the precise positioning of the abnormal code section is needed to be realized, the method is very difficult,
the existing means for positioning the abnormal code segments is to detect each link one by one, and the mode is easy to cause huge economic loss due to the problems of long time consumption for positioning the abnormal code segments and high cost for later-stage system maintenance under the conditions of long system operation time, large code data volume and complex service.
Disclosure of Invention
The application provides a method and a device for positioning an abnormal data node, a computer readable storage medium and computer equipment, which are used for solving the problem that the positioning of an abnormal code segment takes long time and realizing the rapid and accurate positioning of the position of the abnormal data node.
The embodiment of the application firstly provides a method for positioning an abnormal data node, which comprises the following steps:
acquiring log data related to access behaviors, and binding the access behaviors with the log data according to timestamps;
comparing the operating parameters in the log data with a set threshold, and if the log data exceeds the range of the set threshold, judging the corresponding log data as abnormal data;
and acquiring a time stamp corresponding to the abnormal data, acquiring the access behavior according to the time stamp, and determining the position of the abnormal data node according to the access behavior.
In one embodiment, the step of obtaining log data related to access behavior includes:
and continuously monitoring and recording at least one of execution time and execution times of the code corresponding to the access behavior and statistical information generated periodically by the access behavior, wherein the statistical information generated periodically by the access behavior comprises at least one of the utilization rate of a central processing unit, the state of a connection pool and the system load.
In one embodiment, the step of binding the access behavior with the log data according to a timestamp comprises:
segmenting code segments corresponding to the access behaviors and the log data according to the time information to obtain code subsegments and log data segments;
matching according to the code subsegment and the time stamp on the log data segment;
and binding the matched code subsegment and the log data segment for subsequent calling.
In one embodiment, after the step of binding the matched code subsegment and the log data segment, the method further includes:
and packaging each log data segment into an API, establishing a corresponding identification name of each log data segment, and displaying the identification name on a client in a visual form.
In one embodiment, after the step of determining the location of the abnormal data node according to the access behavior, the method further includes:
and removing the occupation of the log data on the storage space.
In one embodiment, after the step of determining the location of the abnormal data node according to the access behavior, the method further includes:
and sending alarm information containing the abnormal data node position to a client.
In one embodiment, the step of sending the alarm information including the abnormal data node location to the client includes:
determining the danger level of the current abnormal data according to the position of the abnormal data node;
and calling an alarm mode corresponding to the danger level according to the danger level of the abnormal data, and sending the alarm information to the client through the determined alarm mode.
Correspondingly, the embodiment of the present application further provides a device for locating an abnormal data node, including:
the log data acquisition module is used for acquiring log data related to access behaviors and binding the access behaviors with the log data according to timestamps;
the abnormal data obtaining module is used for comparing the operating parameters in the log data with a set threshold value, and if the log data exceeds the range of the set threshold value, judging the corresponding log data as abnormal data;
and the abnormal data node position determining module is used for acquiring the timestamp corresponding to the abnormal data, acquiring the access behavior according to the timestamp, and determining the position of the abnormal data node according to the access behavior.
Further, an embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium is used to store computer instructions, and when the computer instructions are run on a computer, the computer is enabled to execute the steps of the method for locating an abnormal data node according to any one of the above technical solutions.
Still further, an embodiment of the present application further provides a computer device, where the computer device includes:
one or more processors;
a storage device for storing one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the abnormal data node locating method according to any one of the above technical solutions.
Compared with the prior art, the scheme provided by the application at least has the following advantages:
according to the abnormal data node positioning method provided by the embodiment of the application, the operation parameters in the log data are continuously compared with the preset threshold value in the access behavior operation process, and once the abnormal data is found, the position of the abnormal data node and the corresponding operation code segment are locked.
According to the positioning method of the abnormal data node, the log data records statistical data generated in the execution process of the code segment corresponding to the access behavior, the data records the operation parameters of at least one code segment, the monitoring of the execution process of the multi-code segment can be realized by adopting the log data, and the monitoring accuracy of the access behavior across system boundaries and the service logic with longer execution time can be improved.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 is an implementation environment diagram of a method for locating an abnormal data node according to an embodiment of the present application;
fig. 2 is a flowchart of a method for locating an abnormal data node according to an embodiment of the present application;
FIG. 3 is a flow diagram that illustrates binding of the access behavior to the log data according to a timestamp, according to an embodiment of the present application;
fig. 4 is a flowchart for sending an alarm message including the location of the abnormal data node to a client according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a device for locating an abnormal data node according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a computer device according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are exemplary only for explaining the present application and are not construed as limiting the present application.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
Fig. 1 is a diagram of an implementation environment of a method for locating an abnormal data node in an embodiment, where the implementation environment includes a user terminal and a server side.
The method for locating the abnormal data node is applicable to the server side, the server side obtains log data related to an access behavior, and the access behavior and the log data are bound according to a timestamp; comparing the operating parameters in the log data with a set threshold, and if the log data exceeds the range of the set threshold, judging the corresponding log data as abnormal data; and acquiring a time stamp corresponding to the abnormal data, acquiring the access behavior according to the time stamp, and determining the position of the abnormal data node according to the access behavior.
It should be noted that the user terminal may be a smart phone, a tablet computer, a notebook computer, a desktop computer, and the like, and the server side may be implemented by a computer device having a processing function, but is not limited thereto. The server and the user terminal may be connected to each other through bluetooth, USB (Universal Serial Bus), or other communication connection methods, which is not limited herein.
In an embodiment, fig. 2 is a flowchart of a method for locating an abnormal data node according to an embodiment of the present application, where the method for locating an abnormal data node may be applied to the server side, and includes the following steps:
step S210, obtaining log data related to access behaviors, and binding the access behaviors with the log data according to timestamps;
step S220, comparing the operation parameters in the log data with a set threshold, and if the log data exceeds the range of the set threshold, judging the corresponding log data as abnormal data;
step S230, obtaining a timestamp corresponding to the abnormal data, obtaining the access behavior according to the timestamp, and determining a position of an abnormal data node according to the access behavior.
The access behavior refers to an access behavior of a certain code, program or application to other code segments, modules, applications, databases, servers, and the like, and includes: remote invocation, database invocation and the like, and the access behaviors can comprise program access behaviors.
The log data is used for recording the operation process of the access behavior and data generated in the process. The log data and the access behavior both have time stamps, the time stamp on the access behavior is the time information for executing a certain program code, and the time stamp on the log data records the time information of the access behavior. And binding the access behavior and the log data corresponding to the same time information according to the time stamp.
The scheme provided by the application aims to obtain the position of an abnormal data node, firstly, an access behavior and log data are bound through a timestamp, namely, the access behavior corresponding to the same time information and the generated log data are bound, the binding also can be an incidence relation, and then, whether the current log data is abnormal data or not is judged by comparing an operation parameter in the log data with a preset threshold value of the operation parameter, wherein the operation parameter in the log data includes but is not limited to: the access behavior corresponds to the execution time of the code, the execution times of the code, the occurrence times of events, statistical information periodically generated by the program and the like, wherein the statistical information periodically generated by the program comprises: CPU, connection pool state, and system load. When the data of any one operation parameter exceeds a preset threshold value, the current log data can be indicated to be abnormal data.
An example is used for explaining how to judge the process of abnormal data according to comparison between an operation parameter in log data and a set threshold, if the log data comprises the execution time of a code segment corresponding to an access behavior, the execution time recorded in the log data is 30 seconds, and the preset threshold range of the execution time of the code segment is between 10 seconds and 20 seconds, and the comparison between the execution times of the code segments shows that the execution time in the log data exceeds the preset threshold range, the code segment is determined to be the abnormal data.
Similarly, the process of judging the abnormal data according to other operating parameters is similar and is not described again.
If the current log data is abnormal data, determining an access behavior corresponding to the current time according to the timestamp of the abnormal data, wherein the access behavior corresponding to the current time is the occurrence position of the abnormal data, and the occurrence position can be an operation code segment corresponding to the abnormal data.
According to the abnormal data node positioning method, the operation parameters and the preset threshold value are continuously compared in the access behavior operation process, abnormal data can be found without waiting for the end of the access behavior, once the abnormal data is found, the position of the abnormal data node and the corresponding operation code segment are locked, the occurrence node of the abnormal data can be rapidly and accurately positioned, and the occurrence node of the abnormal data can be repaired subsequently.
In order to make the positioning scheme of the abnormal data node and the technical effect thereof more clear, the following detailed description is made on the specific scheme of the abnormal data node by using a plurality of embodiments.
In one embodiment, the step of obtaining log data related to access behaviors of step S210 includes:
continuously monitoring and recording at least one of execution time and execution times of codes corresponding to the access behaviors and statistical information generated periodically by the access behaviors; wherein the statistical information generated periodically by the access behavior comprises at least one of Central Processing Unit (CPU) utilization rate, connection pool state and system load.
The log data records statistical data generated by the execution process of the code segment corresponding to the access behavior, such as: the execution time, the execution times and the access behavior of the code are counted to generate statistical information regularly, and the data or the information are all the operation parameters of the recorded code segment or a plurality of code segments to realize the monitoring of the execution process of the multi-segment code segment, so that the positioning scheme of the abnormal data node provided by the application is suitable for recording the access behavior crossing the system boundary and is suitable for the service logic monitoring with longer execution time, such as: remote invocation, database invocation, and multi-system collaboration scenarios.
In one embodiment, the step of binding the access behavior with the log data according to the timestamp in step S210 can be performed as follows, and the flowchart is shown in fig. 3, and includes the following sub-steps:
s310, segmenting code segments corresponding to the access behaviors and the log data according to time information to obtain code subsegments and log data segments;
s320, matching the code subsegments with the time stamps on the log data segments;
s330, binding the matched code subsegment and the log data segment for subsequent calling.
In the embodiment of the application, because the code segment corresponding to the access behavior is likely to be longer and the data volume of the corresponding log data is larger, the log data and the code segment corresponding to the access behavior are segmented according to the time information to form the log data segment and the code subsegment, and the time information for segmenting the log data and the code segment corresponding to the access behavior is preferably the same, for example, in the multi-segment code subsegment and the log data segment, a code subsegment executed in a time period of 20 to 10.
Further, the log data segments can be packaged into an Application Program Interface (API), a corresponding identification name of each log data segment is established, and the log data segments are visually displayed on the client.
And packaging the segmented log data segments into the form of an application program interface, and respectively storing the log data, such as log data generated by the systems 10: the log data segments 20 are displayed on the front-end page, and in such a way, the log data segments are visually displayed on the client, so that a user can conveniently call and view the corresponding log data segments.
The log data segment is packaged into an API form, the data segment can be directly obtained or accessed, source codes do not need to be accessed, or details of an internal working mechanism are understood, and obtaining efficiency is improved.
In the embodiment, the log data is segmented and bound with the code subsections in the same time period or displayed on the client, so that the log data is monitored and managed, abnormal data and generation nodes of the abnormal data are found in time, the maintenance efficiency is improved, and the probability of paralysis of the whole system is reduced.
In an embodiment, before the step of comparing the operating parameter in the log data with the set threshold in step S220, the method further includes: detecting abnormal information or receiving an abnormal analysis request.
In the solution provided in this embodiment, the background receives an exception analysis request, which may be sent by the user side, and executes step S220 to step S230; and returns the location of the anomalous data node.
After the step of determining the location of the abnormal data node according to the access behavior in step S230, the method further includes: sending alarm information containing abnormal data node positions to a client; and once the node position of the abnormal data is determined, the node position is sent to the related client so as to obtain the repair as soon as possible.
Further, the step of sending the alarm information including the abnormal data node location to the client may be performed in the following manner, and a flowchart thereof is shown in fig. 4, and includes the following sub-steps:
s410, determining the danger level of the current abnormal data according to the position of the abnormal data node;
and S420, calling an alarm mode corresponding to the danger level according to the danger level of the abnormal data, and sending the alarm information to the client through the determined alarm mode.
It is known that a project is composed of a large number of code segments, the number of code segments can be divided into a plurality of importance levels according to the functions and positions of the codes, a core code segment has the highest importance level, and the importance level of an optimized code segment is slightly lower, and similarly, the code segments corresponding to the access behavior provided by the present application can also be distinguished by importance levels, the operation starting sections of some code segments are important, and the operation middle sections corresponding to the access behavior are important, so that the importance level corresponding to the code position needs to be set in advance according to the access behavior, after the position of an abnormal data node is determined according to the scheme provided in steps S210 to S230, the position of the abnormal data node, that is, the position of an abnormal code corresponding to the access behavior, an association table of the code position and the importance level is called, the higher the importance level of the code position is, the higher the danger level of the abnormal data appearing at the position is determined according to the position information of the abnormal data node determined in step S230, the association table of the code position and the importance level is combined, the danger level is determined in advance, and an alarm is performed according to the alarm mode corresponding to each preset danger level.
For example, the alarm mode with the lowest risk level may send alarm information indicating that the current access behavior is abnormal to the client, the alarm information with a slightly higher risk level may send alarm information including the position of the abnormal data node to the client in a pop-up window manner, and attract the attention of the user in a blinking display manner, and the alarm information with the highest risk level may call a pre-associated phone number and convey the alarm information including the position of the abnormal data node to the user in a phone manner.
According to the embodiment of the application, the judgment of the danger level is carried out on the current abnormal data by calling the preset danger level standard, corresponding warning measures are executed according to the danger level of the abnormal data, different warning modes are adopted for warning aiming at different danger levels, when the danger level is small, a user cannot be disturbed, warning information can be conveyed, when the danger level is high, the warning information is conveyed through a more direct warning mode, the user is ensured to timely know the warning information, and the abnormal data is repaired as soon as possible.
Further, after the step of determining the location of the abnormal data node according to the access behavior in step S230, the method further includes: and removing the occupation of the log data on the storage space.
After the position of the abnormal data node is determined, deleting the stored log data to reduce the occupation of the log data on the storage space; or deleting the log data except the log data section corresponding to the abnormal data, and reserving the log data section corresponding to the abnormal data so as to be convenient for subsequently calling the log data section to repair the code section, thereby being beneficial to improving the subsequent positioning efficiency of other abnormal data nodes.
The solutions provided in the above embodiments are all that abnormal data occurs during the operation of the access behavior, and in order to reduce memory usage caused by log data, the log data is deleted, after the access behavior is completed, if there is no abnormal data in the log data, the usage of the storage space by the log data is removed, and if there is no abnormal data in the log data during the operation of the access behavior, the data recorded in the log data all conform to the set threshold range, and it is not necessary to recall the access behavior, and the log data corresponding to the access behavior is meaningless to repair the abnormal data, and the usage of the memory data by the partial log data is removed, so as to reduce the usage of excessive intermediate data, which is beneficial to improving the efficiency of obtaining the node position of the abnormal data.
The above embodiments of the method for locating an abnormal data node provided by the present application also provide embodiments of a locating device for an abnormal data node corresponding to the method.
An embodiment of the present application further provides a device for locating an abnormal data node, where a schematic structural diagram of the device is shown in fig. 5, and the device includes: the log data obtaining module 510, the abnormal data obtaining module 520, and the abnormal data node position determining module 530 are specifically as follows:
an obtain log data module 510, configured to obtain log data related to an access behavior, and bind the access behavior with the log data according to a timestamp;
an abnormal data obtaining module 520, configured to compare an operating parameter in the log data with a set threshold, and if the log data exceeds the range of the set threshold, determine that the corresponding log data is abnormal data;
a node location determining module 530, configured to obtain a timestamp corresponding to the abnormal data, obtain the access behavior according to the timestamp, and determine a location of an abnormal data node according to the access behavior.
The log data obtaining module 510 further includes: the device comprises a segmentation unit, a matching unit and a binding unit, wherein the configuration information of each unit is as follows:
the segmenting unit is used for segmenting the code segments corresponding to the access behaviors and the log data according to time information to obtain code subsegments and log data segments;
the matching unit is used for matching the code subsegments with the time stamps on the log data segments;
and the binding unit is used for binding the matched code subsegment and the log data segment so as to facilitate subsequent calling.
With regard to the positioning apparatus for abnormal data nodes in the above embodiments, the specific manner in which each module and unit performs operations has been described in detail in the embodiments related to the method, and will not be elaborated here.
Further, an embodiment of the present application also provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions, when executed by a processor, implement the steps of the abnormal data node positioning method described in any one of the above. The storage medium includes, but is not limited to, any type of disk including floppy disks, hard disks, optical disks, CD-ROMs, and magneto-optical disks, ROMs (Read-Only memories), RAMs (Random AcceSS memories), EPROMs (EraSable Programmable Read-Only memories), EEPROMs (Electrically EraSable Programmable Read-Only memories), flash memories, magnetic cards, or optical cards. That is, a storage medium includes any medium that stores or transmits information in a form readable by a device (e.g., a computer). Which may be a read-only memory, magnetic or optical disk, or the like.
Still further, an embodiment of the present application further provides a computer device, where the computer device includes:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the steps of any of the above-described methods for locating an anomalous data node.
FIG. 6 is a block diagram illustrating a computer device 600 according to an example embodiment. For example, the computer device 600 may be provided as a server. Referring to fig. 6, computer device 600 includes a processing component 622 that further includes one or more processors and memory resources, represented by memory 632, for storing instructions, such as applications, that are executable by processing component 622. The application programs stored in memory 632 may include one or more modules that each correspond to a set of instructions. Further, the processing component 622 is configured to execute instructions to perform the steps of the above-described method of locating an anomalous data node.
The computer device 600 may also include a power component 626 configured to perform power management of the computer device 600, a wired or wireless network interface 650 configured to connect the computer device 600 to a network, and an input/output (I/O) interface 658. The computer device 600 may operate based on an operating system stored in memory 632, such as Windows Server, mac OS XTM, unixTM, linuxTM, freeBSDTM, or the like. It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of execution is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
It should be understood that each functional unit in the embodiments of the present application may be integrated into one processing module, each unit may exist alone physically, or two or more units may be integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
The foregoing is only a partial embodiment of the present application, and it should be noted that, for those skilled in the art, several modifications and decorations can be made without departing from the principle of the present application, and these modifications and decorations should also be regarded as the protection scope of the present application.
Claims (9)
1. A method for locating an abnormal data node is characterized by comprising the following steps:
acquiring log data related to access behaviors, and binding the access behaviors with the log data according to timestamps; the access behavior belongs to cross-system access behavior, and the access behavior is in the running process and is not finished running;
the binding the access behavior with the log data according to the timestamp comprises:
segmenting the code segments corresponding to the access behaviors and the log data according to time information to obtain code subsegments and log data segments;
matching according to the code subsegment and the time stamp on the log data segment;
binding the matched code subsegments with the log data segments;
comparing the operating parameters in the log data with a set threshold, and if the log data exceeds the range of the set threshold, judging the corresponding log data as abnormal data;
and acquiring a time stamp corresponding to the abnormal data, and determining the position of the code subsegment corresponding to the abnormal data node according to the time stamp corresponding to the abnormal data.
2. The method of claim 1, wherein the step of obtaining log data related to access behavior comprises:
and continuously monitoring and recording at least one of execution time and execution times of the code corresponding to the access behavior and statistical information generated periodically by the access behavior, wherein the statistical information generated periodically by the access behavior comprises at least one of the utilization rate of a Central Processing Unit (CPU), the state of a connection pool and the system load.
3. The method for locating an abnormal data node according to claim 1, wherein after the step of binding the matched code subsegment and the log data segment, the method further comprises:
and packaging each log data segment into an API, establishing a corresponding identification name of each log data segment, and displaying the identification name on a client in a visual form.
4. The method according to claim 1, wherein after determining the position of the code sub-segment corresponding to the abnormal data node according to the timestamp corresponding to the abnormal data, the method further comprises:
and removing the occupation of the log data on the storage space.
5. The method according to claim 1, wherein after determining the position of the code sub-segment corresponding to the abnormal data node according to the timestamp corresponding to the abnormal data, the method further comprises:
and sending alarm information containing the abnormal data node position to a client.
6. The method according to claim 5, wherein the step of sending an alarm message including the location of the abnormal data node to the client comprises:
determining the danger level of the current abnormal data according to the position of the abnormal data node;
and calling an alarm mode corresponding to the danger level according to the danger level of the abnormal data, and sending the alarm information to the client through the determined alarm mode.
7. An apparatus for locating an abnormal data node, comprising:
the log data acquisition module is used for acquiring log data for recording access behaviors and binding the access behaviors with the log data according to timestamps; the access behavior belongs to cross-system access behavior, and the access behavior is in the running process and is not finished running;
the binding the access behavior with the log data according to the timestamp comprises:
segmenting the code segments corresponding to the access behaviors and the log data according to time information to obtain code subsegments and log data segments;
matching according to the code subsegment and the time stamp on the log data segment;
binding the matched code subsegments with the log data segments; the access behavior comprises at least one of a remote call behavior, a database call behavior and a program access behavior;
the abnormal data obtaining module is used for comparing the operating parameters in the log data with a set threshold value, and if the log data exceeds the range of the set threshold value, judging the corresponding log data as abnormal data;
and the abnormal data node position determining module is used for acquiring the timestamp corresponding to the abnormal data and determining the position of the corresponding code subsection.
8. A computer-readable storage medium for storing computer instructions which, when executed on a computer, cause the computer to perform the steps of the method for locating an anomalous data node in accordance with any one of the preceding claims 1 to 6.
9. A computer device, characterized in that the computer device comprises:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the steps of the method for locating an anomalous data node in accordance with any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910576944.6A CN110442498B (en) | 2019-06-28 | 2019-06-28 | Abnormal data node positioning method and device, storage medium and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910576944.6A CN110442498B (en) | 2019-06-28 | 2019-06-28 | Abnormal data node positioning method and device, storage medium and computer equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110442498A CN110442498A (en) | 2019-11-12 |
| CN110442498B true CN110442498B (en) | 2022-11-25 |
Family
ID=68428836
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910576944.6A Active CN110442498B (en) | 2019-06-28 | 2019-06-28 | Abnormal data node positioning method and device, storage medium and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110442498B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111045950B (en) * | 2019-12-16 | 2023-06-30 | 上海钧正网络科技有限公司 | Performance problem point determining method, device, data analysis system and storage medium |
| CN111343161B (en) * | 2020-02-14 | 2021-12-10 | 平安科技（深圳）有限公司 | Abnormal information processing node analysis method, abnormal information processing node analysis device, abnormal information processing node analysis medium and electronic equipment |
| CN113472725B (en) * | 2020-03-31 | 2023-04-07 | 阿里巴巴集团控股有限公司 | Data processing method and device |
| WO2021218179A1 (en) * | 2020-04-28 | 2021-11-04 | 平安科技（深圳）有限公司 | Abnormal data query method and apparatus, computer device, and storage medium |
| CN111756579A (en) * | 2020-06-24 | 2020-10-09 | 北京百度网讯科技有限公司 | Abnormity early warning method, device, equipment and storage medium |
| CN112015653A (en) * | 2020-08-31 | 2020-12-01 | 中移（杭州）信息技术有限公司 | Problem positioning method, server and storage medium |
| CN113762569A (en) * | 2020-10-15 | 2021-12-07 | 北京沃东天骏信息技术有限公司 | Data processing method, device, equipment and computer readable storage medium |
| CN112948361B (en) * | 2021-02-02 | 2022-04-01 | 北京三快在线科技有限公司 | Data restoration method and device |
| CN113449159B (en) * | 2021-06-29 | 2024-02-02 | 乐视云网络技术(北京)有限公司 | Node data processing method, device, equipment and computer readable storage medium |
| CN114428822B (en) * | 2022-01-27 | 2022-07-29 | 云启智慧科技有限公司 | Data processing method and device, electronic equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105556482A (en) * | 2013-06-20 | 2016-05-04 | 微软技术许可有限责任公司 | Monitoring mobile application performance |
| CN106557308A (en) * | 2015-09-29 | 2017-04-05 | 腾讯科技（深圳）有限公司 | A kind of software continuous integrated approach and device |
| CN109871307A (en) * | 2018-12-25 | 2019-06-11 | 杭州茂财网络技术有限公司 | A kind of automated testing method being accurately positioned starting matter of time |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7143396B2 (en) * | 2002-11-06 | 2006-11-28 | Sun Microsystems, Inc. | System and method for measuring code segment performance |
| CN107332680A (en) * | 2016-04-28 | 2017-11-07 | 苏宁云商集团股份有限公司 | A kind of system monitoring method and device |
| CN107168845B (en) * | 2017-03-31 | 2020-12-25 | 北京奇艺世纪科技有限公司 | Fault positioning method and device |
| CN107229555B (en) * | 2017-05-04 | 2021-07-20 | 北京星选科技有限公司 | Identification generation method and device |
| CN107273280B (en) * | 2017-06-30 | 2021-02-09 | 百度在线网络技术（北京）有限公司 | Log processing method and device, electronic equipment and storage medium |
| CN109739718A (en) * | 2018-11-30 | 2019-05-10 | 贵阳朗玛信息技术股份有限公司 | Log alarming method and device |
-
2019
- 2019-06-28 CN CN201910576944.6A patent/CN110442498B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105556482A (en) * | 2013-06-20 | 2016-05-04 | 微软技术许可有限责任公司 | Monitoring mobile application performance |
| CN106557308A (en) * | 2015-09-29 | 2017-04-05 | 腾讯科技（深圳）有限公司 | A kind of software continuous integrated approach and device |
| CN109871307A (en) * | 2018-12-25 | 2019-06-11 | 杭州茂财网络技术有限公司 | A kind of automated testing method being accurately positioned starting matter of time |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110442498A (en) | 2019-11-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110442498B (en) | Abnormal data node positioning method and device, storage medium and computer equipment | |
| US8516499B2 (en) | Assistance in performing action responsive to detected event | |
| CN105591821B (en) | Monitoring system and service system | |
| CN107679683B (en) | Software development progress early warning method and device | |
| CN112101803A (en) | Business process monitoring method, device, system, equipment and medium | |
| US11310140B2 (en) | Mitigating failure in request handling | |
| CN113946499A (en) | Micro-service link tracking and performance analysis method, system, equipment and application | |
| CN111651595A (en) | Abnormal log processing method and device | |
| CN111400189A (en) | Code coverage rate monitoring method and device, electronic equipment and storage medium | |
| CN110807050B (en) | Performance analysis method, device, computer equipment and storage medium | |
| CN114978883B (en) | Network wakeup management method and device, electronic equipment and storage medium | |
| CN112328602B (en) | Method, device and equipment for writing data into Kafka | |
| CN113204692A (en) | Method and device for monitoring execution progress of data processing task | |
| CN111124818A (en) | Monitoring method, device and equipment for Expander | |
| CN107193721B (en) | Method and device for generating log | |
| CN104794040A (en) | Service monitoring method, device and system | |
| CN114938339A (en) | Data processing method and related device | |
| CN112860527A (en) | Fault monitoring method and device of application server | |
| CN109669867B (en) | Test apparatus, automated test method, and computer-readable storage medium | |
| CN113761306A (en) | Vehicle-end data processing method and device | |
| CN111078522A (en) | Method and device for identifying to-be-optimized area of page, electronic equipment and storage medium | |
| CN115190008B (en) | Fault processing method, fault processing device, electronic equipment and storage medium | |
| CN116302370A (en) | Method, device, equipment and medium for generating return code abnormality alarm | |
| CN117151726A (en) | Fault repairing method, repairing device, electronic equipment and storage medium | |
| CN116560882A (en) | Fault root cause positioning method, device, equipment and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |