WO2021190659A1

WO2021190659A1 - System data acquisition method and apparatus, and medium and electronic device

Info

Publication number: WO2021190659A1
Application number: PCT/CN2021/084170
Authority: WO
Inventors: 吴超勇
Original assignee: 平安科技（深圳）有限公司
Priority date: 2020-10-29
Filing date: 2021-03-30
Publication date: 2021-09-30
Also published as: CN112306802A

Abstract

A system data acquisition method and apparatus, and a medium and an electronic device, which relate to the technical fields of data management and artificial intelligence. The method comprises: acquiring pre-constructed intelligent collectors, wherein the intelligent collectors are used for collecting operation data of host nodes, and the operation data of the host nodes are different from each other (210); distributing the intelligent collectors and control instructions to the host nodes, so that the host nodes automatically install the intelligent collectors, wherein the control instructions are used for periodically triggering the intelligent collectors to collect the operation data in the host nodes (230); for each host node, monitoring whether the intelligent collector has collected the operation data of the host node (250); and when it is detected that the intelligent collector has collected the operation data of the host node, acquiring the operation data by means of a data transmission channel established with the intelligent collector (270). By means of the method, the consumption of resources required for acquiring system data can be reduced.

Description

System data acquisition method, device, medium and electronic equipment

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on October 29, 2020, the application number is 202011181177.8, and the application name is "systematic data acquisition methods, devices, media and electronic equipment", the entire content of which is incorporated by reference Incorporated in this application.

Technical field

This application belongs to the field of artificial intelligence technology, and particularly relates to a system data acquisition method, device, medium, and electronic equipment.

Background technique

At present, in the scenario of monitoring the system, for example, in the scenario of monitoring the operating status of a complex IT system, it is usually necessary to collect multi-dimensional operating data indicators of various components in the system. The inventor realizes that in the prior art, the acquisition of system data is usually aimed at the specificity of each component, and the corresponding operating data indicators are collected manually/automatically. However, how to reduce the resources required to obtain system data is a technical problem to be solved urgently.

technical problem

The purpose of this application is to provide a system data acquisition method and device, computer readable storage medium, and electronic equipment, so as to reduce the resources required to acquire system data.

Technical solutions

According to one aspect of the embodiments of the present application, there is provided a data acquisition method of a system, the system includes at least two host nodes, and the method includes: acquiring a pre-built intelligent collector, the intelligent collector being used for collecting The operating data of each host node, wherein the operating data of each host node is different from each other; the intelligent collector and control instructions are distributed to each host node, so that each host node automatically installs the intelligent collector, and the control The instruction is used to periodically trigger the smart collector to collect the operating data in the host node; for each host node, monitor whether the smart collector has collected the operating data of the host node; When the operating data of the host node is reached, the operating data is acquired through the data transmission channel established with the intelligent collector.

According to another aspect of the embodiments of the present application, there is provided a data acquisition device of a system, characterized in that the system includes at least two host nodes, and the device includes: a first acquisition unit configured to acquire The intelligent collector is constructed, the intelligent collector is used to collect the operating data of each host node, wherein the operating data of each host node is different from each other; the distribution unit is used to distribute the intelligent collector and control instructions to each Host node, so that each host node automatically installs the smart collector, and the control instruction is used to periodically trigger the smart collector to collect operating data in the host node; the monitoring unit is used to monitor each host node Whether the smart collector collects the operating data of the host node; the second acquisition unit is used to communicate with the smart collector when the operating data of the host node is monitored by the smart collector The established data transmission channel obtains the operating data.

According to another aspect of the embodiments of the present application, there is provided an electronic device for data acquisition of the system, wherein the electronic device includes a memory, a processor, and a computer that is stored in the memory and can run on the processor. Reading instructions, the processor executes the following steps when executing the computer-readable instructions: acquiring a pre-built intelligent collector, the intelligent collector is used to collect the operating data of each host node, wherein the operating data of each host node is mutually exchanged Not the same; distribute the smart collector and control instructions to each host node so that each host node automatically installs the smart collector, and the control instruction is used to periodically trigger the smart collector to collect the operation in the host node Data; For each host node, monitor whether the smart collector collects the operating data of the host node; when the smart collector collects the operating data of the host node, through the smart collector The data transmission channel established by the device obtains the operating data.

According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores at least one instruction, and the following steps are executed when the at least one instruction is executed by a processor: A pre-built intelligent collector, the intelligent collector is used to collect the operating data of each host node, wherein the operating data of each host node is different from each other; the intelligent collector and control instructions are distributed to each host node, so that Each host node automatically installs the smart collector, and the control instruction is used to periodically trigger the smart collector to collect operating data in the host node; for each host node, monitor whether the smart collector collects the host The operating data of the node; when the operating data of the host node collected by the intelligent collector is monitored, the operating data is acquired through the data transmission channel established with the intelligent collector.

Beneficial effect

This application distributes pre-built intelligent collectors and control instructions to each host node and installs them on each host node, so that the intelligent collector in each host node can collect the operating data in the host node where it is located. The proposed technical scheme can uniformly issue collectors for different monitoring objects, saving the time and cost of manually distinguishing monitoring objects, while also simplifying the installation and configuration process, reducing operation and maintenance costs, and reducing the cost of acquiring system data H.

Description of the drawings

Fig. 1 shows a schematic diagram of an application scenario of a data acquisition method of a system according to an embodiment of the present application;

Fig. 2 shows a flow chart of a data acquisition method of the system according to an embodiment of the present application;

Figure 3 shows a detailed flow chart of the smart collector collecting operating data according to an embodiment of the present application;

Fig. 4 shows a flow chart of a method for the smart collector according to an embodiment of the present application before identifying the identity of the application installed in the host node;

FIG. 5 shows a detailed flowchart of the smart collector identifying the identification of the application installed in the host node according to an embodiment of the present application;

Fig. 6 shows a flowchart of a method after the smart collector collects the running data of the installed application according to an embodiment of the present application;

FIG. 7 shows a flowchart of a method after acquiring the operating data through a data transmission channel established with the smart collector according to an embodiment of the present application;

Fig. 8 shows a block diagram of a data acquisition device of the system according to an embodiment of the present application;

FIG. 9 shows a computer-readable storage medium for implementing a data acquisition method of the system according to an embodiment of the present application;

Fig. 10 shows an example block diagram of an electronic device implementing a data acquisition method of a system according to an embodiment of the present application.

Embodiments of the present invention

Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms, and should not be construed as being limited to the examples set forth herein; on the contrary, the provision of these embodiments makes this application more comprehensive and complete, and fully conveys the concept of the example embodiments To those skilled in the art.

In addition, the described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to give a sufficient understanding of the embodiments of the present application. However, those skilled in the art will realize that the technical solutions of the present application can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be used. In other cases, well-known methods, devices, implementations or operations are not shown or described in detail in order to avoid obscuring various aspects of the present application.

The block diagrams shown in the drawings are merely functional entities, and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in the form of software, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and/or processor devices and/or microcontroller devices. entity.

The flowchart shown in the drawings is only an exemplary description, and does not necessarily include all contents and operations/steps, nor does it have to be performed in the described order. For example, some operations/steps can be decomposed, and some operations/steps can be combined or partially combined, so the actual execution order may be changed according to actual conditions.

First, this application briefly describes the application scenarios of the system's data acquisition method.

In an embodiment of the present application, the data acquisition method of the system may be implemented in the scenario shown in FIG. 1. Referring to FIG. 1, a schematic diagram of the application scenario of the system data acquisition method according to an embodiment of the present application is shown. .

First of all, it should be noted that the data acquisition method of the system can be implemented based on the server 101 shown in Figure 1. Of course, the server shown in the figure can also be an electronic device with a touch screen such as a tablet computer or a notebook computer. To replace.

In addition, the data acquisition method of the system can also be executed by a cloud server with cloud computing function. Specifically, the cloud computing is a computing mode that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space, and information services as required. The network that provides resources is called the "cloud." In the eyes of users, the resources in the "cloud" are infinitely expandable, and can be obtained at any time, used on demand, and expanded at any time. Through the establishment of a cloud computing resource pool (referred to as cloud platform, generally referred to as IaaS (Infrastructure As a Service (Infrastructure as a Service) platform, multiple types of virtual resources are deployed in the resource pool for external customers to choose and use. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices.

In the scenario illustrated in FIG. 1, the collection object of operating data includes at least one host node 103 in the IT system 104, for example, including host node 1, host node 2, host node 3, and host node 4. Among them, each host node is responsible for different system operation tasks in the IT system. For example, host node 1 can be used to store system data, and a database is installed in it, and host node 2 can be used to monitor system network security. There is firewall software.

It should be noted that multiple components can also be installed in a host node at the same time to undertake multiple system operation tasks in the IT system.

In an embodiment of the present application, the server 101 distributes the intelligent collector 102 and control instructions for collecting operating data of each host node to each host node 103, so that each host node 103 automatically installs the intelligent collector 102, Among them, the control instruction is used to periodically trigger the smart collector 102 to collect the operating data in the host node 104. For each host node 103, the server 101 monitors whether the smart collector 102 has collected the operating data of the host node, When the intelligent collector collects the operating data of the host node, the server 101 obtains the operating data through the data transmission channel established with the intelligent collector 102.

The implementation details of the technical solution of the system data acquisition method of the embodiment of the present application will be described in detail below with reference to FIG. 2.

Referring to Fig. 2, there is shown a flow chart of data acquisition of the illustrated system according to an embodiment of the present application. The system includes at least one host node, and the data acquisition method of the system can be executed by a physical device or a virtual device with computing and processing functions.

As shown in Figure 2, the data acquisition method of the system at least includes step 210 to step 270:

In step 210, a pre-built smart collector is acquired, and the smart collector is used to collect operating data of each host node, where the operating data of each host node is different from each other.

In this application, storage disk cabinets, operating systems, databases, various middleware and runtime libraries, various application software, etc. are installed in the host node. Each host node is responsible for different system operation tasks in the system, for example, writing documents, making forms, sending emails, managing inventory, managing customers, and so on.

Based on this, when each master node is in a working state, the running data in each host node is different from each other.

In step 230, the smart collector and control instructions are distributed to each host node, so that each host node automatically installs the smart collector, and the control instruction is used to periodically trigger the smart collector to collect data in the host node. Operating data.

Specifically, in this application, the intelligent collector can be sent to a fixed folder on the host node by calling the automatic scheduling platform, and a control command can be added. For example, the control command can be a crond command, and the crond command is a Linux regular The command to execute the program, the crond command will periodically check whether there is a work to be performed, and if there is a work to be performed, the work will be executed automatically.

In this application, each host node can be bound by IP. Each host node corresponds to an IP, and each host node can have multiple data ports. The intelligent collector can determine how many data ports there are by reading the IP port. Applications.

In one case, if a smart collector is installed in the master node, there is no need to distribute the smart collector to the master node.

In another case, if a smart collector is installed in the master node and the smart collector needs to be updated, only the update package data of the smart collector needs to be distributed to each host node for the smart collector to update.

Continuing to refer to FIG. 2, in step 250, for each host node, it is monitored whether the smart collector has collected the operating data of the host node.

In this application, the intelligent collector installed in the host node collects operating data generated by one or more applications in the host node during operation.

In this application, the smart collector includes at least one collection program, wherein each collection program is used to collect operating data of one application.

In an embodiment of the present application, the smart collector can perform the steps shown in FIG. 3 to collect operating data in the host node:

Referring to FIG. 3, it shows the detailed process of collecting operating data by the smart collector according to an embodiment of the present application. Specifically, it includes step 320 to step 340:

In step 320, the smart collector recognizes the identifier of the installed application in the host node.

In this embodiment, before the smart collector recognizes the identifier of the installed application in the host node, the smart collector may also perform the steps shown in FIG. 4.

Referring to FIG. 4, it shows a flow chart of a method for the smart collector according to an embodiment of the present application before identifying the identifier of the installed application in the host node. Specifically, it includes steps 311 to 313:

Step 311: The smart collector recognizes the operating state of the host node.

Step 312: When the host node is in a shutdown state, the intelligent collector collects the shutdown state information of the host node.

Step 313: When the host node is in the power-on state, the smart collector collects the power-on state information of the host node.

Specifically, for example, the intelligent collector detects the running status of the host node through the preset host node status detection command. If the host is in a shutdown state, it collects the shutdown status information of the host node, and if the host node is in the on state, it collects the host node. Operating parameters,

Specifically, the operating parameters include but are not limited to data such as cpu, mem, disk, and host temperature.

In this embodiment, the smart collector includes at least one detection program, and the host node runs at least one type of application, wherein each type of detection program is used to detect a corresponding identification of a type of application.

In this embodiment, the smart collector recognizes the identifier of the installed application in the host node, which can be executed according to the steps shown in FIG. 5.

Referring to FIG. 5, it shows a detailed flowchart of the smart collector recognizing the identifier of the application installed in the host node according to an embodiment of the present application. Specifically, it includes step 321 to step 323:

Step 321: For each detection program, the smart collector determines whether an application of the type detected by the detection program is installed in the host node through the detection program.

Step 322: If yes, the smart collector recognizes the identifier of the installed application belonging to the application type through the detection program.

Step 323, if not, the smart collector returns an empty result of the corresponding detection of the detection program.

Specifically, for example, the smart collector can synchronously start the host node including tomcat, weblogic, zookeeper, nginx and other middleware detection commands to detect whether there is an application of the type detected by the detection program installed in the host node, If so, the ps command is used to identify the application identifier of the middleware application type contained in the process of the host node. If not, return the empty result of the corresponding detection of the detection program.

For another example, the smart collector can synchronously start database detection commands including oracle, mongdb, PostgreSQL, redis, etc. on the host node to detect whether there is an application of the type detected by the detection program installed in the host node. If so, The ps command is used to identify the application identifier of the database application type contained in the process of the host node. If not, return the empty result of the corresponding detection of the detection program.

3, in step 340, based on the identifier of the installed application, the smart collector executes a collection program corresponding to the identifier of the installed application to collect operating data of the installed application.

Specifically, for example, when a middleware application is included in the process of the host node, based on the identified middleware application identifier, the data collection switch and the timing task crond timing task switch of the corresponding middleware application are triggered, such as every 3 minutes. Trigger a collection task to collect the running data of the corresponding middleware application in the host node.

For another example, when a database application is included in the process of the host node, based on the identified database application identifier, the data collection switch and the timed task crond timed task switch of the corresponding database application are triggered, and the corresponding data is collected and applied in the host node The operating data of the database, such as the indicator information of the database instance.

In this embodiment, after the smart collector collects the operating data of the installed application, the smart collector may also perform the steps shown in FIG. 6.

Referring to FIG. 6, it shows a flowchart of a method after the smart collector collects the running data of the installation application according to an embodiment of the present application. Specifically, it includes step 351 to step 352:

Step 351: The smart collector determines the log path in the host node.

Step 352: The smart collector collects log information in the log based on the log path.

Specifically, the smart collector synchronously initiates the path search command of the log on the application host, detects the corresponding application log path, and collects the log information in the host node based on the log path.

Continuing to refer to FIG. 2, in step 270, when the operation data of the host node collected by the intelligent collector is monitored, the operation data is obtained through the data transmission channel established with the intelligent collector.

Specifically, the data transmission channel may be a data transmission channel based on the http protocol.

In an embodiment of the present application, after the operating data is acquired through the data transmission channel established with the smart collector, the steps shown in FIG. 7 may also be performed.

Referring to FIG. 7, it shows a flowchart of a method after acquiring the operating data through the data transmission channel established with the smart collector according to an embodiment of the present application. Specifically, it includes step 281 to step 283:

Step 281: After obtaining the operating data in all host nodes, feature extraction is performed on the operating data in each host node to obtain feature index data.

In this application, the operating data in the host node generally exists in the form of a data stream, which includes parameter information of various monitoring indicators. The feature extraction of the operating data in each host node is actually performed by identifying keywords Segment, extract the parameter data of various indicators in the data stream.

Step 282, based on the characteristic index data and the characteristic index data reference interval, respectively check whether the operation of each host node is abnormal.

In this application, the reference interval of each indicator will be pre-defined according to the actual situation, and then according to whether the parameter of a certain indicator falls within the reference interval corresponding to the indicator, it is judged whether the operation of the host node is abnormal.

Specifically, in step 282, for each host node, when the characteristic index data does not belong to the characteristic index data reference interval, the operation of the host node is abnormal. For example, the data transmission speed of a certain middleware application running in the host node should be between 100 bits per second (bps) and 100 bits per second (bps), however, when the middleware application is actually running in the host node If the data transmission speed is 28 bits per second (bps), it indicates that the middleware application is operating abnormally, and further that the host node where the middleware application is located is operating abnormally.

Step 283: When there is an abnormality in the operation of the host node, an abnormality alarm is issued to the abnormal host node in the system.

In the technical solutions of some embodiments of the present application, pre-built smart collectors and control instructions are distributed to each host node and installed in each host node, so that the intelligent collector in each host node can collect the host node where it is located. Because the technical solution proposed in this application can uniformly issue collectors for different monitoring objects, it saves the time and cost of manually distinguishing monitoring objects. It also simplifies the installation and configuration process and reduces operation and maintenance costs. Reduce the resources required to obtain system data.

The following describes the device embodiments of the present application, which can be used to implement the data acquisition method of the system in the foregoing embodiments of the present application. For details that are not disclosed in the device embodiment of this application, please refer to the embodiment of the data acquisition method of the above-mentioned system of this application.

Fig. 8 shows a block diagram of a data acquisition device of the system according to an embodiment of the present application.

Referring to FIG. 8, the data acquisition device 800 of the system according to an embodiment of the present application includes: a first acquisition unit 801, a distribution unit 802, a monitoring unit 803, and a second acquisition unit 804.

Among them, the first obtaining unit 801 is used to obtain a pre-built smart collector, which is used to collect the operating data of each host node, where the operating data of each host node is different from each other; the distribution unit 802 is used to Used for distributing the smart collector and control instructions to each host node, so that each host node automatically installs the smart collector, and the control instruction is used to periodically trigger the smart collector to collect operating data in the host node The monitoring unit 803 is used for each host node to monitor whether the smart collector has collected the operating data of the host node; the second obtaining unit 804 is used to monitor the collection of the smart collector When operating data of the host node, the operating data is acquired through a data transmission channel established with the intelligent collector.

In some embodiments of the present application, based on the foregoing solution, the smart collector includes at least one collection program, wherein each collection program is used to collect the operating data of an application corresponding to each host node. The smart collector is configured to: the smart collector recognizes the identifier of the installed application in the host node; based on the identifier of the installed application, the smart collector performs collection corresponding to the identifier of the installed application Program to collect the running data of the installed application.

In some embodiments of the present application, based on the foregoing solution, the smart collector is configured to: before the smart collector recognizes the identifier of the installed application in the host node, the smart collector recognizes the host The operating state of the node; when the host node is in a shutdown state, the smart collector collects the shutdown state information of the host node; when the host node is in the on state, the smart collector collects the host node Power-on status information.

In some embodiments of the present application, based on the foregoing solution, the smart collector includes at least one detection program, and the host node runs at least one type of application, wherein each detection program is used to detect a corresponding type of Identification of the application, the smart collector is configured to: for each detection program, the smart collector determines through the detection program whether an application of the type detected by the detection program is installed in the host node; if so, then The smart collector recognizes the identifier of the installed application belonging to the application type through the detection program; if not, the smart collector returns an empty result of the detection corresponding to the detection program.

In some embodiments of the present application, based on the foregoing solution, the smart collector is configured to: after the smart collector collects the operating data of the installation application, the smart collector determines the log in the host node Path; the smart collector collects log information in the log based on the log path.

In some embodiments of the present application, based on the foregoing solution, the device further includes: an extraction unit, which is used to obtain the operating data through the data transmission channel established with the smart collector, after obtaining all the hosts After the operating data in the node, feature extraction is performed on the operating data in each host node to obtain the feature index data; the verification unit is used to verify each of the feature index data and the reference interval of the feature index data. Whether there is an abnormality in the operation of the host node; the alarm unit is used to issue an abnormal alarm to the abnormal host node in the system when the operation of the host node is abnormal.

In some embodiments of the present application, based on the foregoing solution, the verification unit is configured to: for each host node, when the characteristic index data does not belong to the characteristic index data reference interval, the host node's There is an abnormal operation.

It should be noted that although the data acquisition method of the system and several units of the data acquisition device of the system are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, two or more units and functions described above may be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied. A component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the present application. Those of ordinary skill in the art can understand and implement without creative work.

As another aspect, the present application also provides a computer-readable storage medium. The computer-readable storage medium may be non-volatile or volatile, and a program capable of implementing the above-mentioned method in this specification is stored thereon. product. In some possible implementation manners, various aspects of the present application can also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to make the The terminal device executes the steps according to various exemplary embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.

Referring to FIG. 9, a program product 900 for implementing the above method according to an embodiment of the present application is described. It can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be installed in a terminal device, For example, running on a personal computer. However, the program product of this application is not limited to this. In this document, the readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, device, or device.

The program product can use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or a combination of any of the above. More specific examples (non-exhaustive list) of readable storage media include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Type programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

The computer-readable signal medium may include a data signal propagated in baseband or as a part of a carrier wave, and readable program code is carried therein. This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The readable signal medium may also be any readable medium other than a readable storage medium, and the readable medium may send, propagate, or transmit a program for use by or in combination with the instruction execution system, apparatus, or device.

The program code contained on the readable medium can be transmitted by any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

The program code used to perform the operations of the present application can be written in any combination of one or more programming languages. The programming languages include object-oriented programming languages—such as Java, C++, etc., as well as conventional procedural programming languages. Programming language-such as "C" language or similar programming language. The program code can be executed entirely on the user's computing device, partly on the user's device, executed as an independent software package, partly on the user's computing device and partly executed on the remote computing device, or entirely on the remote computing device or server Executed on. In the case of a remote computing device, the remote computing device can be connected to a user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (for example, using Internet service providers). Business to connect via the Internet).

As another aspect, the present application also provides an electronic device capable of implementing the above method.

Those skilled in the art can understand that various aspects of the present application can be implemented as a system, a method, or a program product. Therefore, each aspect of the present application can be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which can be collectively referred to herein as "Circuit", "Module" or "System".

The electronic device 1000 according to this embodiment of the present application will be described below with reference to FIG. 10. The electronic device 1000 shown in FIG. 10 is only an example, and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

As shown in FIG. 10, the electronic device 1000 is represented in the form of a general-purpose computing device. The components of the electronic device 1000 may include, but are not limited to: the aforementioned at least one processing unit 1010, the aforementioned at least one storage unit 1020, and a bus 1030 connecting different system components (including the storage unit 1020 and the processing unit 1010).

Wherein, the storage unit stores program code, and the program code can be executed by the processing unit 1010, so that the processing unit 1010 executes the various exemplary methods described in the above-mentioned "Methods of Embodiments" section of this specification. Steps of implementation.

The storage unit 1020 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 1021 and/or a cache storage unit 1022, and may further include a read-only storage unit (ROM) 1023.

The storage unit 1020 may also include a program/utility tool 1024 having a set of (at least one) program modules 1025. Such program modules 1025 include but are not limited to: an operating system, one or more application programs, other program modules, and program data, Each of these examples or some combination may include the implementation of a network environment.

The bus 1030 may represent one or more of several types of bus structures, including a storage unit bus or a storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local area using any bus structure among multiple bus structures. bus.

The electronic device 1000 may also communicate with one or more external devices 1100 (such as keyboards, pointing devices, Bluetooth devices, etc.), and may also communicate with one or more devices that enable a user to interact with the electronic device 1000, and/or communicate with Any device (such as a router, modem, etc.) that enables the electronic device 1000 to communicate with one or more other computing devices. This communication can be performed through an input/output (I/O) interface 1050. In addition, the electronic device 1000 may also communicate with one or more networks (for example, a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet) through the network adapter 1060. As shown in the figure, the network adapter 1060 communicates with other modules of the electronic device 1000 through the bus 1030. It should be understood that although not shown in the figure, other hardware and/or software modules can be used in conjunction with the electronic device 1000, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives And data backup storage system, etc.

Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described here can be implemented by software, or can be implemented by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, U disk, mobile hard disk, etc.) or on the network , Including several instructions to make a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) execute the method according to the embodiment of the present application.

In addition, the above-mentioned drawings are merely schematic illustrations of the processing included in the method according to the exemplary embodiments of the present application, and are not intended for limitation. It is easy to understand that the processing shown in the above drawings does not indicate or limit the time sequence of these processings. In addition, it is easy to understand that these processes can be executed synchronously or asynchronously in multiple modules, for example.

It should be understood that the present application is not limited to the precise structure that has been described above and shown in the drawings, and various modifications and changes can be performed without departing from its scope. The scope of the application is only limited by the appended claims.

Claims

A method for acquiring data in a system, the system includes at least two host nodes, and the method includes:

Acquiring a pre-built intelligent collector, which is used to collect the operating data of each host node, where the operating data of each host node is different from each other;

Distributing the smart collector and control instructions to each host node, so that each host node automatically installs the smart collector, and the control instruction is used to periodically trigger the smart collector to collect operating data in the host node;

For each host node, monitor whether the intelligent collector has collected the operating data of the host node;

When the operation data of the host node collected by the intelligent collector is monitored, the operation data is obtained through the data transmission channel established with the intelligent collector.
The method according to claim 1, wherein the smart collector includes at least one collection program, wherein each collection program is used to collect operating data corresponding to one type of application, and for each host node, the smart collector The collector collects operating data in the following ways:

The smart collector recognizes the identifier of the installed application in the host node;

Based on the identifier of the installed application, the smart collector executes a collection program corresponding to the identifier of the installed application to collect operating data of the installed application.
The method according to claim 2, wherein, before the smart collector recognizes the identification of the installed application in the host node, the method further comprises:

The smart collector recognizes the operating state of the host node;

When the host node is in a shutdown state, the intelligent collector collects the shutdown state information of the host node;

When the host node is in the power-on state, the smart collector collects power-on state information of the host node.
The method according to claim 2, wherein the smart collector includes at least one detection program, and the host node runs at least one type of application, wherein each detection program is used to detect one type of application. Identification, the identification of the smart collector identifying the installed application in the host node includes:

For each detection program, the smart collector determines through the detection program whether an application of the type detected by the detection program is installed in the host node;

If yes, the smart collector recognizes the identifier of the installed application belonging to the application type through the detection program;

If not, the smart collector returns an empty result corresponding to the detection by the detection program.
The method according to claim 2, wherein, after the smart collector collects operating data of the installed application, the method further comprises:

The smart collector determines the log path in the host node;

The smart collector collects log information in the log based on the log path.
The method according to claim 1, wherein, after acquiring the operating data through the data transmission channel established with the smart collector, the method further comprises:

After obtaining the operating data in all host nodes, perform feature extraction on the operating data in each host node to obtain feature index data;

Based on the characteristic index data and the characteristic index data reference interval, respectively verify whether the operation of each host node is abnormal;

When there is an abnormality in the operation of the host node, an abnormal alarm is issued to the abnormal host node in the system.
7. The method according to claim 6, wherein the step of separately verifying whether the operation of each host node is abnormal based on the characteristic index data and the characteristic index data reference interval comprises:

For each host node, when the characteristic index data does not belong to the characteristic index data reference interval, the operation of the host node is abnormal.
A data acquisition device of a system, wherein the system includes at least two host nodes, and the device includes:

The first acquiring unit is used to acquire a pre-built intelligent collector, which is used to collect operating data of each host node, where the operating data of each host node is different from each other;

The distribution unit is used to distribute the smart collector and control instructions to each host node, so that each host node automatically installs the smart collector, and the control instruction is used to trigger the smart collector to collect the host node regularly Operating data in;

The monitoring unit is used for each host node to monitor whether the intelligent collector has collected the operating data of the host node;

The second acquiring unit is used to acquire the operating data through the data transmission channel established with the intelligent collector when monitoring the operating data of the host node collected by the intelligent collector.
An electronic device for data acquisition of a system, wherein the electronic device includes a memory, a processor, and computer-readable instructions stored in the memory and running on the processor, and the processor executes the Perform the following steps when computer-readable instructions:

Acquiring a pre-built intelligent collector, which is used to collect the operating data of each host node, where the operating data of each host node is different from each other;

Distributing the smart collector and control instructions to each host node, so that each host node automatically installs the smart collector, and the control instruction is used to periodically trigger the smart collector to collect operating data in the host node;

For each host node, monitor whether the intelligent collector has collected the operating data of the host node;

When the operation data of the host node collected by the intelligent collector is monitored, the operation data is obtained through the data transmission channel established with the intelligent collector.
The electronic device according to claim 9, wherein the smart collector includes at least one collection program, wherein each collection program is used to collect operating data of one application, and for each host node, the The smart collector collects operating data in the following ways:

The smart collector recognizes the identifier of the installed application in the host node;

Based on the identifier of the installed application, the smart collector executes a collection program corresponding to the identifier of the installed application to collect operating data of the installed application.
The electronic device according to claim 10, wherein, before the smart collector recognizes the identification of the installed application in the host node, the processor further executes the following steps when executing the computer-readable instruction:

The smart collector recognizes the operating state of the host node;

When the host node is in a shutdown state, the intelligent collector collects the shutdown state information of the host node;

When the host node is in the power-on state, the smart collector collects power-on state information of the host node.
The electronic device according to claim 10, wherein the smart collector includes at least one detection program, and the host node runs at least one type of application, wherein each detection program is used to detect a corresponding type of application The identification of the smart collector identifying the installation application in the host node includes:

For each detection program, the smart collector determines through the detection program whether an application of the type detected by the detection program is installed in the host node;

If yes, the smart collector recognizes the identifier of the installed application belonging to the application type through the detection program;

If not, the smart collector returns an empty result corresponding to the detection by the detection program.
The electronic device according to claim 10, wherein, after the smart collector collects the running data of the installed application, the processor further executes the following steps when executing the computer-readable instruction:

The smart collector determines the log path in the host node;

The smart collector collects log information in the log based on the log path.
The electronic device according to claim 9, wherein, after acquiring the operating data through the data transmission channel established with the smart collector, the processor further executes the following steps when executing the computer-readable instruction:

After obtaining the operating data in all host nodes, perform feature extraction on the operating data in each host node to obtain feature index data;

Based on the characteristic index data and the characteristic index data reference interval, respectively verify whether the operation of each host node is abnormal;

When there is an abnormality in the operation of the host node, an abnormal alarm is issued to the abnormal host node in the system.
14. The electronic device according to claim 14, wherein said separately verifying whether the operation of each host node is abnormal based on the characteristic index data and the characteristic index data reference interval comprises:

For each host node, when the characteristic index data does not belong to the characteristic index data reference interval, the operation of the host node is abnormal.
A computer-readable storage medium, wherein the computer-readable storage medium stores at least one instruction, and the following steps are executed when the at least one instruction is executed by a processor:

Acquiring a pre-built intelligent collector, which is used to collect the operating data of each host node, where the operating data of each host node is different from each other;

Distributing the smart collector and control instructions to each host node, so that each host node automatically installs the smart collector, and the control instruction is used to periodically trigger the smart collector to collect operating data in the host node;

For each host node, monitor whether the intelligent collector has collected the operating data of the host node;

When the operation data of the host node collected by the intelligent collector is monitored, the operation data is obtained through the data transmission channel established with the intelligent collector.
The computer-readable storage medium according to claim 16, wherein the smart collector includes at least one collection program, wherein each collection program is used to collect the operating data of an application corresponding to each host node , The smart collector collects operating data in the following ways:

The smart collector recognizes the identifier of the installed application in the host node;

Based on the identifier of the installed application, the smart collector executes a collection program corresponding to the identifier of the installed application to collect operating data of the installed application.
The computer-readable storage medium according to claim 17, wherein, before the smart collector recognizes the identification of the installed application in the host node, the following is executed when the at least one instruction is executed by the processor step:

The smart collector recognizes the operating state of the host node;

When the host node is in a shutdown state, the intelligent collector collects the shutdown state information of the host node;

When the host node is in the power-on state, the smart collector collects power-on state information of the host node.
The computer-readable storage medium according to claim 17, wherein the smart collector includes at least one detection program, and the host node runs at least one type of application, wherein each detection program is used to detect one The identification of the type of application, and the identification of the installed application on the host node by the smart collector includes:

For each detection program, the smart collector determines through the detection program whether an application of the type detected by the detection program is installed in the host node;

If yes, the smart collector recognizes the identifier of the installed application belonging to the application type through the detection program;

If not, the smart collector returns an empty result corresponding to the detection by the detection program.
18. The computer-readable storage medium according to claim 17, wherein, after the smart collector collects the running data of the installed application, the following steps are performed when the at least one instruction is executed by the processor:

The smart collector determines the log path in the host node;

The smart collector collects log information in the log based on the log path.