CN112559565A - Abnormity detection method, system and device - Google Patents
Abnormity detection method, system and device Download PDFInfo
- Publication number
- CN112559565A CN112559565A CN202011387050.1A CN202011387050A CN112559565A CN 112559565 A CN112559565 A CN 112559565A CN 202011387050 A CN202011387050 A CN 202011387050A CN 112559565 A CN112559565 A CN 112559565A
- Authority
- CN
- China
- Prior art keywords
- information processing
- information
- record
- unmarked
- processing record
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
- G06F16/24554—Unary operations; Data partitioning operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/219—Managing data history or versioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Abstract
The specification discloses an abnormality detection method, system and device, which are used for sending a query request to an information processing device according to an address of the information processing device obtained from a server node, determining whether an abnormal information processing record exists according to an unmarked information processing record and a marked information processing record which are locally stored by the information processing device in a query result returned by the information processing device, and sending prompt information for prompting that a partition of information obtained by the information processing device changes if the abnormal information processing record exists. And determining whether the abnormal information processing record exists according to the acquired information processing record, so that the change condition of the partition corresponding to the information processing equipment can be monitored in real time, and when the change of the partition corresponding to the information processing equipment is known, the pressure of the information processing equipment can be reduced by adjusting the information processing speed.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a method, a system, and an apparatus for detecting an abnormality.
Background
At present, with the development of computer technology, people have more and more demand for information, and similarly, the requirements of mass data on information systems are gradually increased. Particularly in the field of large data processing, a large amount of data and information needs to be processed, and thus the demand for an information distribution system is high.
In the prior art, Kafka is a high-throughput distributed publish-subscribe messaging system, and is generally used in the field of real-time processing of big data.
In general, the Kafka system serves as a platform for connecting between an information producer and an information processing apparatus, and receives information sent from the producer for the information processing apparatus to acquire, as shown in fig. 1. The basic architecture of Kafka mainly comprises three parts, namely a server node, a producer and information processing equipment. The producer is the party producing the information and sends the information to the server node, and the server node is also called a broker and is used for receiving and caching the information sent by the producer and sending the information to the information processing equipment according to a strategy, or the information processing equipment captures the information from the server node, and the information processing equipment is the party processing the information.
The Kafka system, in order to improve information throughput, generally stores different types of information separately according to the types of information, and also has different partitions provided under each type, and stores the same type of information in the different partitions, and the same information processing side, in order to improve the capability of acquiring information, may also generally provide an information processing group consisting of a plurality of information processing apparatuses, each of which is available for acquiring information, so that the information processing side can acquire information through the plurality of apparatuses. Also, in order to avoid repeated acquisition of information, information of one partition cannot be acquired by a plurality of information processing apparatuses in the same information processing group. Whereas, in order to determine each information processing apparatus in the information processing group, specifically acquire information of that partition, the Kafka system also provides a rebalancing strategy for determining the correspondence relationship between the partitions and the information processing apparatuses.
Disclosure of Invention
The present specification provides an anomaly detection method, system and apparatus to partially solve the above problems in the prior art.
The technical scheme adopted by the specification is as follows:
the abnormality detection method provided by the present specification includes:
acquiring the address of each information processing device in the information processing group from the server node;
for each information processing device in the information processing group, sending a query request to the information processing device according to the address of the information processing device;
receiving a query result returned by the information processing equipment according to the query request, wherein the query result comprises: an unmarked information processing record and a marked information processing record which are locally stored by the information processing device, wherein the information processing record is determined and stored by the information processing device according to the information acquired from the server node;
judging whether an abnormal information processing record exists according to the unmarked information processing record and the marked information processing record;
if so, sending prompt information, wherein the prompt information is used for prompting that the partition of the information acquired by the information processing equipment is changed, and sending a marking request to the information processing equipment, wherein the marking request is used for enabling the information processing equipment to mark the unmarked information processing record returned according to the query request as a marked information processing record.
Optionally, the unmarked information processing record is determined and stored when the characteristics of the information acquired by the information processing apparatus from the server node and the characteristics of the information corresponding to the marked information processing record stored by the information processing apparatus are different;
wherein the information processing record includes at least a time at which the information processing record was generated and a partition identification.
Optionally, the method further comprises:
determining whether an unmarked information processing record exists according to the acquired information processing record;
if the unmarked information processing record exists, determining the unmarked information processing record as an abnormal information processing record and sending prompt information, and marking the unmarked information processing record.
Optionally, the method further comprises:
determining whether an unmarked information processing record exists according to the acquired information processing record;
if the unmarked information processing record exists, judging whether an abnormal information processing record exists according to the unmarked information processing record and the marked information processing record;
if yes, sending prompt information, and marking the unmarked information processing record.
Optionally, sending, by each information processing apparatus in the information processing group, an inquiry request to the information processing apparatus according to the address of the information processing apparatus, specifically including:
and according to a preset monitoring period, aiming at each information processing device in the information processing group, sending a query request to the information processing device according to the address of the information processing device.
Optionally, the method further comprises:
and if the abnormal information processing record does not exist, sending a marking request to the information processing equipment so that the information processing equipment marks the unmarked information processing record.
The present specification provides an anomaly detection system, the system comprising: the monitoring device and the information processing group comprise at least one information processing device; wherein: the information processing device is used for acquiring the characteristic information of the information from the information acquired from the server node every time, judging whether the characteristic information of the information is the same as the preset first characteristic information or not, if the characteristic information of the information is different from the preset first characteristic information, determining that the information is abnormal information, and determining an information processing record and storing the information processing record according to the partition identification and the time point corresponding to the abnormal information;
the monitoring device is configured to obtain an address of each information processing device in an information processing group from a server node, send, according to the address of the information processing device, an inquiry request to the information processing device for each information processing device in the information processing group, and receive an inquiry result returned by the information processing device according to the inquiry request, where the inquiry result includes: the information processing device comprises an unmarked information processing record and a marked information processing record which are locally stored in the information processing device, wherein the information processing record is determined and stored by the information processing device according to information acquired from the server node, whether an abnormal information processing record exists is judged according to the unmarked information processing record and the marked information processing record, if yes, prompt information is sent, the prompt information is used for prompting that the partition of the information acquired by the information processing device changes, and a marking request is sent to the information processing device, and the marking request is used for enabling the information processing device to mark the unmarked information processing record returned according to the query request as the marked information processing record.
The present specification provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the above-described abnormality detection method.
The electronic device provided by the present specification includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the above-mentioned abnormality detection method when executing the program.
The technical scheme adopted by the specification can achieve the following beneficial effects:
in the abnormality detection method provided by the present specification, it is determined whether there is an abnormal information processing record, based on the unmarked information processing record and the marked information processing record stored in each information processing apparatus in each acquired information processing group, and if there is an abnormal information processing record, prompt information for prompting the information processing apparatus that the partition of the acquired information has changed is transmitted.
According to the method, the unmarked information processing records and the marked information processing records generated and stored by the information processing equipment are obtained, whether the abnormal information processing records exist or not is determined according to the obtained information processing records, the change condition of the partition corresponding to the information processing equipment can be monitored in real time, and when the change of the partition corresponding to the information processing equipment is known, the pressure of the information processing equipment can be reduced by adjusting the information processing speed.
Drawings
The accompanying drawings, which are included to provide a further understanding of the specification and are incorporated in and constitute a part of this specification, illustrate embodiments of the specification and together with the description serve to explain the specification and not to limit the specification in a non-limiting sense. In the drawings:
FIG. 1 is a schematic diagram of the infrastructure of a conventional Kafka system;
FIG. 2 is a schematic flow chart of an anomaly detection method provided herein;
FIG. 3 is a schematic diagram of a partition change process provided herein;
FIG. 4 is a schematic flow chart of an anomaly detection method provided herein;
FIG. 5 is a schematic view of an anomaly detection apparatus provided herein;
FIG. 6 is a schematic diagram of an anomaly detection system provided herein;
fig. 7 is a schematic diagram of an electronic device corresponding to fig. 2 provided in the present specification.
Detailed Description
In order to make the objects, technical solutions and advantages of the present disclosure more clear, the technical solutions of the present disclosure will be clearly and completely described below with reference to the specific embodiments of the present disclosure and the accompanying drawings. It is to be understood that the embodiments described are only a few embodiments of the present disclosure, and not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present specification without any creative effort belong to the protection scope of the present specification.
In the Kafka system, the information processing apparatuses in the information processing group are generally apparatuses such as servers and terminals capable of analyzing and processing data, and are also generally referred to as information processors. In order to improve the capability of processing information, the same number of information processing devices can be correspondingly arranged for an information processing group according to the number of partitions arranged in different information types, so that the information of each partition in the information of the type is acquired by one information processing device and processed. Load imbalance of different information processing devices in the information processing group is avoided, so that the information processing group can process information of each subarea in parallel, and the information processing capacity of the information processing group is improved.
In the case where information processing apparatuses correspond to partitions one to one (that is, information of different partitions is acquired and processed by different information processing apparatuses), when the number of information processing apparatuses in an information processing group is reduced (for example, an original information processing apparatus is kicked out of the information processing group or the information processing apparatus is down), a rebalancing mechanism of the Kafka system is triggered, and the corresponding relationship between each partition of the type of information and the information processing apparatus in the information processing group is re-determined. When the partition corresponding to the information processing apparatus that is down is reallocated to another information processing apparatus through the rebalancing mechanism, the other information processing apparatus that reallocates the partition may need to process information of more partitions, and the operating pressure of the other information processing apparatus after rebalancing may increase. The operating pressure increases, which may continue to cause the other information processing apparatus to be down. Further, in order to improve the efficiency of information processing by information processing apparatuses, the processing policy of information processing by information processing apparatuses is usually set as needed, for example, different processing policies are determined according to different partition information acquired by the information processing apparatuses, and in the case where the information stored in the partitions may be inconsistent, the processing policies of information processing apparatuses corresponding to different partitions may also be different. Thus, when the rebalancing mechanism of the Kafka system is triggered, the processing policy of the information processing apparatus may no longer be applied to the information of the partition it acquires, so that the information processing apparatus is reduced in operating efficiency.
In addition, in the prior art, the corresponding relationship between the partitions and the information processing devices can only be obtained through an active query mode, and the information processing devices corresponding to the partitions can only be found out when the information processing devices are down, a rebalancing mechanism is triggered, and the partitions are allocated to other information processing devices and an abnormal condition occurs. Even if the processing policy of the information is newly configured for the information processing apparatus, waste of resources cannot be avoided.
Based on this, the present specification proposes an abnormality detection method capable of detecting in time a change in the correspondence relationship between the partitions and the information processing apparatuses.
The technical solutions provided by the embodiments of the present description are described in detail below with reference to the accompanying drawings.
Fig. 2 is a schematic flow chart of the abnormality detection method provided in this specification, specifically including the following steps:
s100: the address of each information processing apparatus in the information processing group is acquired from the server node.
S102: and sending an inquiry request to each information processing device in the information processing group according to the address of the information processing device.
In the Kafka system, information in different partitions is stored in a server node, and the server node needs to send the information to corresponding information processing equipment for processing according to the subscription relationship between the information processing equipment and the information in the partitions, so that the server node stores the addresses of the information processing equipment in the information processing group.
Thus, in the present specification, a monitoring apparatus that executes the abnormality detection method may send an address acquisition request to a server node. After receiving the address acquisition request sent by the monitoring system, the server node can return the addresses of the information processing devices in the stored information processing group to the monitoring device. After acquiring the address of each information processing device in the information processing group, the monitoring device may further send, for each information processing device in the information processing group, an inquiry request to the information processing device according to the acquired address of the information processing device.
The inquiry request is used for acquiring an information processing record stored by the information processing device.
S104: receiving a query result returned by the information processing equipment according to the query request, wherein the query result comprises: an unmarked information processing record locally stored by the information processing apparatus and a marked information processing record determined and stored by the information processing apparatus based on the information acquired from the server node.
In one or more embodiments of the present specification, for each information processing apparatus, the information processing apparatus, upon receiving an inquiry request sent by a monitoring apparatus, may determine a stored information processing record as an inquiry result, and return the inquiry result to the monitoring apparatus.
The information processing record is determined when the information processing equipment acquires the information sent by the server node. In the Kafka system, the information of different partitions stored in the server node carries the partition identification, and therefore, when the information processing apparatus acquires information from the server node, the information processing record can be determined according to the partition identification carried by the acquired information and the time of acquiring the information. The content of the information processing record may include a partition identifier of the acquired information and time of acquiring the information, and may specifically include what content is set as required, which is not limited in this specification.
Determining, at the information processing device, a query result according to the query request, and the monitoring device may further receive a query result returned by the monitoring device, where the query result includes: the information processing apparatus stores locally unmarked information processing records and marked information processing records. Wherein an unmarked information processing record refers to an information processing record that the monitoring device has not acquired from the information processing device, and a marked information processing record refers to an information processing record that the monitoring device has acquired.
In one or more embodiments provided herein, the monitoring device may be a single server or may be composed of multiple servers. The functions of acquiring information from the information processing apparatus and performing analysis processing on the acquired information are realized by code running on the server. When the monitoring device is composed of a plurality of servers, which server monitors which information processing device in which information processing group is specifically set as required, one server may monitor one or a plurality of information processing devices, and the plurality of monitored information processing devices may also belong to different information processing groups. The specific setting can be according to needs, and this specification does not limit this.
Further, in order to save server resources, the query result returned by the information processing device to the monitoring device according to the query request may include an unmarked information processing record and a marked information processing record closest to the current acquisition time.
S106: and judging whether an abnormal information processing record exists according to the unmarked information processing record and the marked information processing record, and if so, executing the step S108.
After the monitoring device obtains the information processing records returned by the information processing devices, the partition identifier corresponding to the unmarked information processing record and the partition identifier corresponding to the marked information processing record corresponding to each information processing device can be determined according to the unmarked information processing records and the marked information processing records.
After determining the partition identifier corresponding to the unmarked information processing record and the partition identifier corresponding to the marked information processing record, the monitoring device may further determine whether the partition identifier corresponding to the unmarked information processing record is different from the partition identifier corresponding to the marked information processing record according to the partition identifier corresponding to the determined unmarked information processing record and the partition identifier corresponding to the marked information processing record.
If so, determining that an abnormal information processing record exists according to the partition identification corresponding to the information processing record different from the marked information processing record
S108: sending prompt information for prompting that the partition of the information acquired by the information processing device changes, and sending a marking request to the information processing device, wherein the marking request is used for enabling the information processing device to mark the unmarked information processing record returned according to the inquiry request as a marked information processing record.
After determining that the abnormal information processing record exists, the monitoring device may send a prompt message according to the abnormal information processing record, where the prompt message is used to prompt that the partition of the information acquired by the information processing device changes.
Further, the monitoring apparatus is also capable of determining that there is an unmarked information processing record other than the partition identification corresponding to the marked information processing record in the unmarked information processing record according to the partition identification corresponding to the determined information processing record other than the marked information processing record, and determining that there is an unmarked information processing record other than the partition identification corresponding to the marked information processing record as the abnormal information processing record.
Furthermore, the monitoring device may further determine, according to the abnormal information processing record, a partition identifier of the information acquired by the information processing device, a time when the partition of the information processing device changes, and a process when the partition identifier corresponding to the information processing device changes, and generate a graph facilitating more intuitive observation of the partition changing process and send prompt information for prompting that the partition corresponding to the information processing device changes according to the determined time and process when the partition changes. The chart can be used for adjusting the information processing strategies of the machines corresponding to the rebalanced partitions in time and preventing the condition that the information processing equipment corresponding to the rebalanced partitions is shut down due to overlarge operating pressure.
Fig. 3 is a schematic diagram of a partition change process provided in this specification, and is divided into two parts, namely a server node and an information processing group, where a solid line part indicates a correspondence relationship between a partition and an information processing apparatus before a balancing mechanism is triggered, a shaded part indicates that the correspondence relationship between the partition and the information processing apparatus has changed, and a dotted line part indicates a specific information processing apparatus with which the partition whose correspondence relationship between the partition and the information processing apparatus has changed has reestablished a correspondence relationship after rebalancing, and a time for reestablishing the correspondence relationship.
In addition, the monitoring device may also mark the obtained unmarked information processing record, where the marking mode may be that the information processing device marks the unmarked information processing record when sending the unmarked information processing record to the monitoring device after receiving the query request, or that the monitoring device marks the unmarked information processing record after determining the unmarked information processing record, or that the monitoring device sends a marking request to the information processing device, and the information processing device marks the unmarked information processing device according to an identifier of the unmarked information processing record carried in the marking request, and may set the unmarked information processing device as needed, which is not limited in this specification.
Further, in order to improve the sensitivity of the monitoring device, a monitoring period may be preset, such as: every minute is a monitoring cycle, and within every minute, the monitoring device needs to acquire the information processing record stored by the information processing device once, and perform abnormality detection and marking according to the acquired information processing record. Different monitoring periods can be preset for different information processing groups, and the specific period can be set as required, which is not limited in the specification.
Based on the anomaly detection method provided in fig. 2, an inquiry request is sent to the information processing device according to the address of the information processing device acquired from the server node, so as to determine whether an anomalous information processing record exists according to an unmarked information processing record and a marked information processing record locally stored by the information processing device in the inquiry result returned by the information processing device, and if an anomalous information processing record exists, prompt information for prompting that the partition of the information acquired by the information processing device changes is sent. According to the method, the unmarked information processing records and the marked information processing records generated and stored by the information processing equipment are obtained, whether the abnormal information processing records exist or not is determined according to the obtained information processing records, the change condition of the partition corresponding to the information processing equipment can be monitored in real time, and when the change of the partition corresponding to the information processing equipment is known, the pressure of the information processing equipment can be reduced by adjusting the processing speed.
Further, based on the schematic flow chart of the abnormality detection method shown in fig. 2, the present specification also provides a schematic flow chart of another abnormality detection method, as shown in fig. 4.
Fig. 4 is a schematic flowchart of the abnormality detection method provided in this specification, and the contents of steps S100 to S108 included in fig. 4 refer to the contents described in fig. 2, and if the monitoring device determines that there is no abnormality information processing record in step S106, step S110 may be executed.
Step S110: a marking request is sent to the information processing apparatus to cause the information processing apparatus to mark the unmarked information processing record.
If no abnormal information processing record exists, that is, if the partition identifier corresponding to the unmarked information processing record does not have a partition identifier different from the partition identifier corresponding to the marked information processing record, the monitoring device may mark the unmarked information processing record.
In addition, in this specification, in order to save resources and improve resource utilization, for each information processing apparatus, the information processing apparatus may regenerate and store an information processing record when it is determined that a change has occurred in the partition to which the acquired information belongs.
Specifically, when the information processing apparatus acquires information from a server node, the information processing apparatus may first determine a feature corresponding to the information sent by the server node, determine a partition identifier corresponding to the information based on the feature, that is, determine a partition in which the information is stored in the server node, then determine whether a partition identifier corresponding to the information sent by the server node is the same as a partition identifier corresponding to a marked information processing record stored in the information processing apparatus based on a partition identifier in the feature corresponding to the marked information processing record stored locally by the information processing apparatus and a partition identifier corresponding to the determined information, if the partition identifiers are the same, it is determined that the correspondence between the information processing apparatus and the partition has not changed, the information processing apparatus does not need to generate an information processing record, and if the partition identifiers are different, it is determined that the correspondence between the information processing apparatus and the partition has changed, an information processing record may be generated and stored according to the obtained partition identifier of the information sent by the server node and the time of obtaining the information, and it should be noted that the generated information processing record is unmarked.
Further, when the monitoring device receives the information processing record sent by the information processing device, the monitoring device can firstly judge whether the unmarked information processing record exists, if the unmarked information processing record does not exist, the operation condition of the information processing device is normal, and if the unmarked information processing record exists, the monitoring device can determine that the unmarked information processing record is the abnormal information processing record
Furthermore, in order to improve the reliability of the monitoring device, after the monitoring device determines that an unmarked information processing record exists, the monitoring device may further compare the partition identifier corresponding to the marked information processing record with the unmarked information processing record, when the partition identifier corresponding to the marked information processing record corresponding to the information processing device is different from the unmarked information processing record, determine that the unmarked information processing record is an abnormal information processing record, and send a prompt message according to the partition identifier of the abnormal information processing record to prompt the information processing device that the corresponding partition is changed, and mark the unmarked information processing record.
In addition, in order to improve monitoring efficiency, connection with the monitoring device is required to be established every time a new information processing device is added, and after the unmarked information processing record and the marked information processing record are determined, the unmarked information processing record and the marked information processing record can be sent to and stored in one information processing device in the information processing group.
Based on the same idea, the abnormality detection method provided in one or more embodiments of the present specification further provides a corresponding abnormality detection apparatus, as shown in fig. 5.
Fig. 5 is a schematic diagram of an abnormality detection apparatus provided in the present specification, which specifically includes:
an address acquisition module 200 for acquiring an address of each information processing apparatus in the information processing group from the server node.
And the query module 202 is configured to send, to each information processing apparatus in the information processing group, a query request according to the address of the information processing apparatus to the information processing apparatus.
A receiving module 204, configured to receive a query result returned by the information processing apparatus according to the query request, where the query result includes: an unmarked information processing record locally stored by the information processing apparatus and a marked information processing record determined and stored by the information processing apparatus based on the information acquired from the server node.
The determining module 206 determines whether an abnormal information processing record exists according to the unmarked information processing record and the marked information processing record, and if so, sends a prompt message for prompting that the partition of the information acquired by the information processing apparatus changes, and sends a marking request to the information processing apparatus, where the marking request is used for enabling the information processing apparatus to mark the unmarked information processing record returned according to the query request as the marked information processing record.
Optionally, the query module 202 is configured to send, according to a preset monitoring period, a query request to each information processing device in the information processing group according to an address of the information processing device.
Optionally, the unmarked information processing record is determined and stored when the characteristics of the information acquired by the information processing apparatus from the server node and the characteristics of the information corresponding to the marked information processing record stored by the information processing apparatus are different, wherein the information processing record at least comprises the time and the partition identification for generating the information processing record.
Optionally, the determining module 206 is configured to determine whether an unmarked information processing record exists according to the obtained information processing record, determine, if an unmarked information processing record exists, that the unmarked information processing record is an abnormal information processing record and send a prompt message, and mark the unmarked information processing record.
Optionally, the determining module 206 is configured to determine whether an unmarked information processing record exists according to the obtained information processing record, if an unmarked information processing record exists, determine whether an abnormal information processing record exists according to the unmarked information processing record and the marked information processing record, if so, send a prompt message, and mark the unmarked information processing record.
Optionally, the determining module 206 is configured to send a marking request to the information processing apparatus if there is no abnormal information processing record, so that the information processing apparatus marks the unmarked information processing record.
The present specification also provides a computer-readable storage medium storing a computer program operable to execute the abnormality detection method provided in fig. 2 described above.
Fig. 6 is a schematic structural diagram of an anomaly detection system provided in the present specification, wherein the system includes: the monitoring device comprises a monitoring device and an information processing group, wherein the information processing group comprises at least one information processing device.
The information processing device is used for acquiring the characteristic information of the information from the information acquired from the server node every time, judging whether the characteristic information of the information is the same as the preset characteristic information or not, if the characteristic information of the information is different from the preset characteristic information, determining that the information is abnormal information, and determining an information processing record and storing the information according to the partition identification and the time point corresponding to the abnormal information.
Specifically, the information processing apparatus may perform the process of determining the information processing record in step S104 in the abnormality detection method shown in fig. 2, and the process of marking the obtained unmarked information processing record in step S108, details of which have been described in the above method steps may refer to the description of the above contents in the specification, and the description of the above contents is not repeated here.
The monitoring device is configured to acquire an address of each information processing device in an information processing group from a server node, send, according to the address of the information processing device, an inquiry request to the information processing device for each information processing device in the information processing group, and receive an inquiry result returned by the information processing device according to the inquiry request, where the inquiry result includes: the information processing device comprises an unmarked information processing record and a marked information processing record which are locally stored in the information processing device, wherein the information processing record is determined and stored by the information processing device according to information acquired from the server node, whether an abnormal information processing record exists is judged according to the unmarked information processing record and the marked information processing record, if yes, prompt information is sent, the prompt information is used for prompting that the partition of the information acquired by the information processing device changes, and a marking request is sent to the information processing device, and the marking request is used for enabling the information processing device to mark the unmarked information processing record returned according to the query request as the marked information processing record.
Specific contents executed by the monitoring device during the anomaly detection can refer to the description of the steps S100 to S108 in the anomaly detection method of fig. 2 for executing the operation of the monitoring device, and are not described herein again.
Specifically, when the monitoring device obtains the information processing record, the monitoring device may first send an inquiry request to the information processing device, and after receiving the inquiry request, the information processing device may send the unmarked information processing record and the marked information processing record to the monitoring device, so that the monitoring device determines whether an abnormal information processing record exists according to the unmarked information processing record and the marked information processing record, and the information processing device marks the unmarked information processing record.
Fig. 6 shows 2 information processing groups, where the information processing group 1 includes an information processing device 1-1, an information processing device 1-2, an information processing device 1-3, and an information processing device 1-4, the information processing group 2 includes an information processing device 2-1 and an information processing device 2-2, and the monitoring device includes two servers that respectively obtain information processing records of the information processing device 1-1, the information processing device 1-2, the information processing device 2-1, the information processing device 2-2, and the information processing device 1-3, and the information processing device 1-4. Of course, fig. 6 is only a schematic diagram of the detection system, wherein the monitoring device may be one device or a distributed system consisting of a plurality of devices, and the number of information processing devices included in each information processing group may also be set as required.
This specification also provides a schematic block diagram of the electronic device shown in fig. 7. As shown in fig. 7, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, but may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs the computer program to implement the anomaly detection method described in fig. 2 above. Of course, besides the software implementation, the present specification does not exclude other implementations, such as logic devices or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, and may be hardware or logic devices.
In the 90 s of the 20 th century, improvements in a technology could clearly distinguish between improvements in hardware (e.g., improvements in circuit structures such as diodes, transistors, switches, etc.) and improvements in software (improvements in process flow). However, as technology advances, many of today's process flow improvements have been seen as direct improvements in hardware circuit architecture. Designers almost always obtain the corresponding hardware circuit structure by programming an improved method flow into the hardware circuit. Thus, it cannot be said that an improvement in the process flow cannot be realized by hardware physical modules. For example, a Programmable Logic Device (PLD), such as a Field Programmable Gate Array (FPGA), is an integrated circuit whose Logic functions are determined by programming the Device by a user. A digital system is "integrated" on a PLD by the designer's own programming without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Furthermore, nowadays, instead of manually making an Integrated Circuit chip, such Programming is often implemented by "logic compiler" software, which is similar to a software compiler used in program development and writing, but the original code before compiling is also written by a specific Programming Language, which is called Hardware Description Language (HDL), and HDL is not only one but many, such as abel (advanced Boolean Expression Language), ahdl (alternate Hardware Description Language), traffic, pl (core universal Programming Language), HDCal (jhdware Description Language), lang, Lola, HDL, laspam, hardward Description Language (vhr Description Language), vhal (Hardware Description Language), and vhigh-Language, which are currently used in most common. It will also be apparent to those skilled in the art that hardware circuitry that implements the logical method flows can be readily obtained by merely slightly programming the method flows into an integrated circuit using the hardware description languages described above.
The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, an Application Specific Integrated Circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic for the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller as pure computer readable program code, the same functionality can be implemented by logically programming method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Such a controller may thus be considered a hardware component, and the means included therein for performing the various functions may also be considered as a structure within the hardware component. Or even means for performing the functions may be regarded as being both a software module for performing the method and a structure within a hardware component.
The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. One typical implementation device is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functions of the various elements may be implemented in the same one or more software and/or hardware implementations of the present description.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, the description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the description may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
This description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The above description is only an example of the present specification, and is not intended to limit the present specification. Various modifications and alterations to this description will become apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification should be included in the scope of the claims of the present specification.
Claims (10)
1. An abnormality detection method characterized by comprising:
acquiring the address of each information processing device in the information processing group from the server node;
for each information processing device in the information processing group, sending a query request to the information processing device according to the address of the information processing device;
receiving a query result returned by the information processing equipment according to the query request, wherein the query result comprises: an unmarked information processing record and a marked information processing record which are locally stored by the information processing device, wherein the information processing record is determined and stored by the information processing device according to the information acquired from the server node;
judging whether an abnormal information processing record exists according to the unmarked information processing record and the marked information processing record;
if so, sending prompt information, wherein the prompt information is used for prompting that the partition of the information acquired by the information processing equipment is changed, and sending a marking request to the information processing equipment, wherein the marking request is used for enabling the information processing equipment to mark the unmarked information processing record returned according to the query request as a marked information processing record.
2. The method according to claim 1, wherein the unmarked information processing record is determined and stored when a characteristic of information acquired by the information processing apparatus from a server node and a characteristic of information corresponding to the marked information processing record stored by the information processing apparatus are different;
wherein the information processing record includes at least a time at which the information processing record was generated and a partition identification.
3. The method of claim 2, wherein the method further comprises:
determining whether an unmarked information processing record exists according to the received information processing record;
if the unmarked information processing record exists, determining the unmarked information processing record as an abnormal information processing record and sending prompt information, and marking the unmarked information processing record.
4. The method of claim 2, wherein the method further comprises:
determining whether an unmarked information processing record exists according to the received information processing record;
if the unmarked information processing record exists, judging whether an abnormal information processing record exists according to the unmarked information processing record and the marked information processing record;
if yes, sending prompt information, and marking the unmarked information processing record.
5. The method according to claim 1, wherein sending, for each information processing apparatus in the information processing group, an inquiry request to the information processing apparatus according to the address of the information processing apparatus specifically includes:
and according to a preset monitoring period, aiming at each information processing device in the information processing group, sending a query request to the information processing device according to the address of the information processing device.
6. The method of claim 1, wherein the method further comprises:
and if the abnormal information processing record does not exist, sending a marking request to the information processing equipment so that the information processing equipment marks the unmarked information processing record.
7. An anomaly detection system, said system comprising: the monitoring device and the information processing group comprise at least one information processing device; wherein:
the information processing device is used for acquiring the characteristic information of the information from the information acquired from the server node every time, judging whether the characteristic information of the information is the same as the preset first characteristic information or not, if the characteristic information of the information is different from the preset first characteristic information, determining that the information is abnormal information, and determining an information processing record and storing the information processing record according to the partition identification and the time point corresponding to the abnormal information;
the monitoring device is configured to obtain an address of each information processing device in an information processing group from a server node, send, according to the address of the information processing device, an inquiry request to the information processing device for each information processing device in the information processing group, and receive an inquiry result returned by the information processing device according to the inquiry request, where the inquiry result includes: the information processing device comprises an unmarked information processing record and a marked information processing record which are locally stored in the information processing device, wherein the information processing record is determined and stored by the information processing device according to information acquired from the server node, whether an abnormal information processing record exists is judged according to the unmarked information processing record and the marked information processing record, if yes, prompt information is sent, the prompt information is used for prompting that the partition of the information acquired by the information processing device changes, and a marking request is sent to the information processing device, and the marking request is used for enabling the information processing device to mark the unmarked information processing record returned according to the query request as the marked information processing record.
8. An abnormality detection device characterized by comprising:
the address acquisition module is used for acquiring the address of each information processing device in the information processing group from the server node;
the query module is used for sending a query request to each information processing device in the information processing group according to the address of the information processing device;
a receiving module, configured to receive a query result returned by the information processing device according to the query request, where the query result includes: an unmarked information processing record and a marked information processing record which are locally stored by the information processing device, wherein the information processing record is determined and stored by the information processing device according to the information acquired from the server node;
and the determining module is used for judging whether an abnormal information processing record exists according to the unmarked information processing record and the marked information processing record, if so, sending prompt information which is used for prompting that the partition of the information acquired by the information processing equipment is changed, and sending a marking request to the information processing equipment, wherein the marking request is used for enabling the information processing equipment to mark the unmarked information processing record returned according to the inquiry request as the marked information processing record.
9. A computer-readable storage medium, characterized in that the storage medium stores a computer program which, when executed by a processor, implements the method of any of the preceding claims 1 to 6.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any of claims 1 to 6 when executing the program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011387050.1A CN112559565A (en) | 2020-12-01 | 2020-12-01 | Abnormity detection method, system and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011387050.1A CN112559565A (en) | 2020-12-01 | 2020-12-01 | Abnormity detection method, system and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112559565A true CN112559565A (en) | 2021-03-26 |
Family
ID=75047088
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011387050.1A Pending CN112559565A (en) | 2020-12-01 | 2020-12-01 | Abnormity detection method, system and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112559565A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113344454A (en) * | 2021-07-05 | 2021-09-03 | 湖南快乐阳光互动娱乐传媒有限公司 | Pressure measurement data processing method and device |
-
2020
- 2020-12-01 CN CN202011387050.1A patent/CN112559565A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113344454A (en) * | 2021-07-05 | 2021-09-03 | 湖南快乐阳光互动娱乐传媒有限公司 | Pressure measurement data processing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107450981B (en) | Block chain consensus method and equipment | |
| EP3425524A1 (en) | Cloud platform-based client application data calculation method and device | |
| CN107066519B (en) | Task detection method and device | |
| CN110046187B (en) | Data processing system, method and device | |
| CN111258851B (en) | Cluster alarm method, device, setting and storage medium | |
| CN109391512A (en) | A kind of service issuing method, device and electronic equipment | |
| CN113067875A (en) | Access method, device and equipment based on dynamic flow control of micro-service gateway | |
| CN115185777A (en) | Abnormity detection method and device, readable storage medium and electronic equipment | |
| CN113595773B (en) | Information processing method, device, electronic equipment and storage medium | |
| CN109002348B (en) | Load balancing method and device in virtualization system | |
| CN112965882B (en) | Data fault analysis method and device | |
| CN112559565A (en) | Abnormity detection method, system and device | |
| CN113704117A (en) | Algorithm testing system, method and device | |
| CN116737345A (en) | Distributed task processing system, distributed task processing method, distributed task processing device, storage medium and storage device | |
| CN110908824A (en) | Fault identification method, device and equipment | |
| CN111221651B (en) | Load balancing configuration method, device, server and readable storage medium | |
| CN114860432A (en) | Method and device for determining information of memory fault | |
| CN114691445A (en) | Cluster fault processing method and device, electronic equipment and readable storage medium | |
| CN108255820B (en) | Method and device for data storage in distributed system and electronic equipment | |
| CN110659105A (en) | Virtualized resource processing method, device, equipment and storage medium | |
| CN116743550B (en) | Processing method of fault storage nodes of distributed storage cluster | |
| CN111404709B (en) | Method and device for operating policy rules under network function virtualization | |
| CN117041980B (en) | Network element management method and device, storage medium and electronic equipment | |
| CN110968370B (en) | Data processing method and device | |
| CN113271235B (en) | Fuzzy test method and device for network traffic, storage medium and processor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |