WO2021114812A1 - 数据采集方法、装置、计算机设备及计算机可读介质 - Google Patents

数据采集方法、装置、计算机设备及计算机可读介质 Download PDF

Info

Publication number
WO2021114812A1
WO2021114812A1 PCT/CN2020/117255 CN2020117255W WO2021114812A1 WO 2021114812 A1 WO2021114812 A1 WO 2021114812A1 CN 2020117255 W CN2020117255 W CN 2020117255W WO 2021114812 A1 WO2021114812 A1 WO 2021114812A1
Authority
WO
WIPO (PCT)
Prior art keywords
collection
model file
module
data
parameters
Prior art date
Application number
PCT/CN2020/117255
Other languages
English (en)
French (fr)
Inventor
陈波
唐文彬
全锐
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Priority to EP20898040.9A priority Critical patent/EP3975481B1/en
Publication of WO2021114812A1 publication Critical patent/WO2021114812A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08—Configuration management of networks or network elements
    • H04L41/0803—Configuration setting
    • H04L41/084—Configuration by using pre-existing information, e.g. using templates or copying from other elements
    • H04L41/0843—Configuration by using pre-existing information, e.g. using templates or copying from other elements based on generic templates
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06—Management of faults, events, alarms or notifications
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08—Configuration management of networks or network elements
    • H04L41/0803—Configuration setting
    • H04L41/0806—Configuration setting for initial configuration or provisioning, e.g. plug-and-play
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/34—Signalling channels for network management communication
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/02—Standardisation; Integration
    • H04L41/0213—Standardised network management protocols, e.g. simple network management protocol [SNMP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/02—Standardisation; Integration
    • H04L41/0246—Exchanging or transporting network management information using the Internet; Embedding network management web servers in network elements; Web-services-based protocols
    • H04L41/0273—Exchanging or transporting network management information using the Internet; Embedding network management web servers in network elements; Web-services-based protocols using web services for network management, e.g. simple object access protocol [SOAP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters

Definitions

  • the embodiments of the present disclosure relate to the field of communication technology, and in particular to a data collection method, device, computer equipment, and computer-readable medium.
  • the monitoring system is an automation system based on computer technology. It can collect data from network-related equipment and systems, so as to realize functions such as alarm monitoring and performance collection.
  • the monitoring system has many drawbacks: for new collection objects, it needs to be hard-coded and deployed to the existing monitoring system, the development and joint debugging cycle is long, and the monitoring system needs to be restarted after the deployment is completed, which affects the business run. For existing collection objects, if they need to be deployed to other monitoring systems, they also need to be coded and deployed again, which is inefficient. With the development of the network, various new demands continue to increase, and it becomes more and more difficult to update or repair the functions of the monitoring system, and it is also difficult for the monitoring system to quickly respond to demands.
  • the embodiments of the present disclosure provide a data collection method, device, computer equipment, and computer readable medium.
  • an embodiment of the present disclosure provides a data collection method, the method includes: receiving a model file, the model file is used to define data collection objects, collection parameters and alarm description information; sending the model file to the collection Device, so that the collection device collects data according to the model file.
  • an embodiment of the present disclosure also provides a data collection method, the method includes: receiving a model file sent by a processing device, the model file is used to define data collection objects, collection parameters, and alarm description information; The model file collects data; sends the collected data to the designated device.
  • embodiments of the present disclosure also provide a data collection method, the method comprising: in response to receiving collection object parameters, collection parameters, and alarm description information, generating according to the collection object parameters, collection parameters, and alarm description information A model file, the model file is used to define data collection objects, collection parameters and alarm description information; the model file is sent to the processing device.
  • an embodiment of the present disclosure also provides a processing device, including: a first receiving module and a first sending module, wherein the first receiving module is used to receive a model file, and the model file is used to define data Collection objects, collection parameters, and alarm description information; the first sending module is configured to send the model file to a collection device, so that the collection device collects data according to the model file.
  • an embodiment of the present disclosure also provides a collection device, including: a second receiving module, a collection module, and a second sending module, wherein the second receiving module is used to receive the model file sent by the processing device, so The model file is used to define data collection objects, collection parameters and alarm description information; the collection module is used to collect data according to the model file; the second sending module is used to send the collected data to a designated device .
  • an embodiment of the present disclosure also provides a model definition device, including a generating module and a sending module, wherein the generating module is configured to respond to receiving collection object parameters, collection parameters, and alarm description information, according to the The collection object parameters, collection parameters, and alarm description information generate a model file, and the model file is used to define the data collection object, collection parameters, and alarm description information.
  • the sending module is used for sending the model file to a processing device.
  • the embodiments of the present disclosure also provide a computer device, including: one or more processors and a storage device; wherein, one or more programs are stored on the storage device, and when the above one or more programs are used by the above one When executed by the or multiple processors, the foregoing one or more processors implement the data collection methods provided in the foregoing embodiments.
  • the embodiments of the present disclosure also provide a computer-readable medium storing a computer program, where the computer program implements the data collection method provided in the foregoing embodiments when the computer program is executed.
  • FIG. 1 is a system structure diagram of the application of the data collection method provided by an embodiment of the disclosure
  • FIG. 2 is a flowchart of a data collection method provided by an embodiment of the disclosure
  • FIG. 3 is another flowchart of the data collection method provided by an embodiment of the present disclosure.
  • FIG. 5 is a flowchart of collecting data according to a model file according to another embodiment of the present disclosure.
  • Fig. 6a is a schematic diagram of a deployment method provided by another embodiment of the present disclosure.
  • Figure 6b is a schematic diagram of a deployment method provided by yet another embodiment of the present disclosure.
  • FIG. 6c is a schematic diagram of a deployment method provided by another embodiment of the present disclosure.
  • FIG. 7 is a schematic structural diagram of a processing device provided by another embodiment of the present disclosure.
  • FIG. 8 is a schematic structural diagram of a collection device provided by still another embodiment of the present disclosure.
  • FIG. 9 is a schematic structural diagram of a model definition device provided by another embodiment of the present disclosure.
  • FIG. 10 is a flowchart of a data collection method provided by an embodiment of the present disclosure.
  • FIG. 11 is a schematic structural diagram of a computer device provided by an embodiment of the disclosure.
  • the embodiments of the present disclosure provide a data collection method. As shown in FIG. 1, the method is applied to a system including a monitoring system, a collection device, and a collection object.
  • the monitoring system includes a processing device and General business module.
  • the collection object can be hardware or software that can be managed by the monitoring system through a certain communication protocol, such as routers, switches, firewalls, servers, racks/frames, MySQL (My Structure Quest Language, relational database management system) And PostgreSQL (Postgre Structured Query Language, Object Relational Database Management System), Ubuntu (Ubantu Operating System) and Windows (Windows Operating System), Kafka (Apache Kafka, Distributed Publish and Subscribe Message System) and FTP (File Transfer Protocol) , File Transfer Protocol) and other public service software, OpenStack (cloud computing service component) and VMware (a kind of virtual machine software) and other cloud platforms.
  • the collection device can collect resource, performance, and alarm data from the collection object.
  • Performance collection refers to the use of SNMP (Simple Network Management Protocol, Simple Network Management Protocol), SSH (Secure Shell, Secure Shell Protocol), JDBC (Java Database Connectivity Java, database connection) and other protocols obtain performance data from collection objects.
  • Alarm monitoring refers to monitoring alarms for collection objects that support the SNMP protocol.
  • the processing device can process the data uploaded by the collection device and is responsible for transmitting the data to the network management, cloud management or third-party application.
  • the processing device can usually be deployed as a service, microservice, or plug-in in the network management, cloud management or third-party application platform .
  • the general service module may be a service module commonly used in the monitoring system of the network management or cloud management.
  • the data collection method provided by the embodiment of the present disclosure may include the following steps:
  • Step 21 Receive a model file, which is used to define data collection objects, collection parameters, and alarm description information.
  • the processing device can receive the model file uploaded by the user in the form of a compressed file.
  • the compressed file can include multiple model files, which are used to define data collection objects, collection parameters, and alarm description information.
  • the data collection object can be defined by defining the network element type in the resource (for example, the field of the storage network element) and the component type in the resource (for example, the field of the storage component instance), and the collection can be defined by defining related performance information such as measurement counters. Parameters, for example, define collection parameters as CPU (Central Processing Unit/Processor) usage, CPU temperature, etc.
  • the alarm description information can be defined by defining alarm codes corresponding to various abnormal conditions.
  • the collection objects, collection parameters and alarm description information are all necessary information for the collection device of the monitoring system to collect the data of the equipment or system.
  • Step 22 Send the model file to the collection device, so that the collection device collects data according to the model file.
  • the processing device can send the model file directly to the collection device in the form of a compressed file, and the collection device parses each model file from the compressed file to obtain data collection objects, collection parameters and alarm description information, which can be further based on this information Data collection.
  • the collection device can collect data according to the model file, if there is a need to increase collection requirements (for example, adding a new type of collection parameter), it is only necessary to add corresponding information in the model file. If you need to monitor the collection object in other monitoring systems of the same type, you can reuse the model file of the collection object in other monitoring systems. If you need to support the collection of data from new equipment, customized equipment or special equipment, you can do so by modifying or adding data collection objects, collection parameters, and alarm description information in the model file.
  • the data collection method receives a model file used to define data collection objects, collection parameters, and alarm description information, and sends the model file to the collection device so that the collection device Collect data according to the model file.
  • the embodiment of the present disclosure can use the model file to define the collection object.
  • the monitoring system and the collection object are in a decoupled relationship. In this way, the model file can take effect after the model file is uploaded to the monitoring system. Therefore, the joint debugging cycle can be shortened and the monitoring does not need to be restarted The system will not affect business operations.
  • the embodiment of the present disclosure can also reuse the model file definition in other monitoring systems without recoding and deployment.
  • the embodiments of the present disclosure can also update or repair the function of the monitoring system by redefining the model file, and quickly respond to changes in demand.
  • the embodiments of the present disclosure can be applied to collection objects of types such as central data centers and edge clouds, and can monitor cloud resources at various levels of IaaS, PaaS, and SaaS in various scenarios.
  • Step 31 Obtain the access parameters of the collection object to be added.
  • the general service module can include a user interface (UI) module.
  • UI user interface
  • the user can upload the model file to the processing device through the human-computer interaction interface module.
  • the human-computer interaction interface module can dynamically generate parameters based on the description of the resource field in the model file.
  • the input field is convenient for the user to fill in the access parameters of the collection object to be added, and send the access parameters to the processing device.
  • Step 32 Send the access parameters to the general service module, so that the general service module persists the access parameters, and the general service module includes a resource module.
  • the access parameters can be stored in the database, so that there is no need to import or input the access parameters when subsequently collecting data of the collection object.
  • Step 33 Send the access parameters to the collection device, so that the collection device performs data collection on the to-be-added collection objects according to the access parameters and the model file.
  • the processing device sends the access parameters to the collection device so that the collection device can automatically analyze the collection logic according to the access parameters and cooperate with the model file sent by the processing device to intelligently collect the data of the collection object.
  • step 21 after receiving the model file (ie, step 21), it may further include: sending the model file to the general service module, so that the general service module loads the model file.
  • the processing device receives the model file uploaded by the user in the form of a compressed file, it will decompress the compressed file while sending the model file to the collection device to obtain data collection objects, collection parameters and alarm descriptions.
  • Information model files, and each model file is sent to the general business module, so that the general business module loads the model file, so as to achieve business monitoring.
  • the model file may include: a resource model file used to define data collection objects, a performance model file used to define collection parameters, and an alarm model file used to define alarm description information.
  • General service modules may include: resource modules, performance modules, and alarm modules.
  • the sending the model file to the general service module so that the general service module loads the model file may include: sending the resource model file to the resource module, so that the resource module loads the resource model file, and the performance model file Send to the performance module so that the performance module loads the performance model file, and sends the alarm model file to the alarm module, so that the alarm module loads the alarm model file.
  • the compressed file can include three model files: resource model files, performance model files, and alarm resource model files.
  • General business modules can also include resource modules, performance modules, and alarm modules.
  • Resource modules can be used to manage collection objects.
  • the performance module can be used to provide performance-related services, and the alarm module can be used to provide alarm-related services.
  • the processing device decompresses the compressed file, it can respectively send the three model files to the corresponding modules, and each module dynamically loads the corresponding model file to display resources, performance, alarms and other related information to the user.
  • the database storing resources and performance data can dynamically generate different storage tables based on the description of the resources and performance fields in the model.
  • the model file may also include one or any combination of the following: a script file, a protocol configuration file, and a management information library file.
  • the script file can include one or any combination of the following: the execution script of the related protocol used to connect the device and collect data, the custom script for the resource, that is, the access parameter field, the special processing script for the alarm analysis, and the performance data Customized special logic scripts such as scripts for secondary calculations and calling external services to collect data.
  • the protocol configuration file may include a configuration file for analyzing the results returned by the protocol, and so on.
  • the model file may also include a Management Information Base (MIB) file.
  • MIB Management Information Base
  • the user can package the above-mentioned script file, protocol configuration file, and management information library file with the resource model file, performance model file, and alarm model file to generate a compressed package, and send the compressed package to the processing device.
  • the sending the model file to the collection device may include: sending the model file to the collection device through the expressive state transfer interface; and/or sending the model file to the general service module , May include: sending the model file to the general service module through the expressive state transfer interface.
  • the processing device and the collection device, as well as the processing device and the general service module, can be connected through a uniformly defined Restful (Representational State Transfer) interface to form a microservice architecture.
  • the processing device can connect model files or access parameters Send to the collection device or general business module through the Restful interface.
  • the processing device, the acquisition device and the general business module are independent microservice modules, and each microservice module is loosely coupled, and each microservice module only focuses on completing its own task. When new requirements are added to the monitoring system, only the corresponding microservice modules need to be added or modified. It becomes easy and simple to update or repair the functions of the monitoring system.
  • the microservice architecture is very suitable for cloud deployment, it can be compatible with applications in the cloud development model.
  • the embodiment of the present disclosure also provides a data collection method. As shown in FIG. 4, the method may include the following steps:
  • Step 41 Receive a model file sent by the processing device, where the model file is used to define data collection objects, collection parameters, and alarm description information.
  • the processing device can send the model file to the acquisition device in the form of a compressed file.
  • Step 42 Collect data according to the model file.
  • Step 43 Send the collected data to the designated device.
  • the collection device After the collection device receives the model file sent by the processing device in the form of a compressed file, it can directly collect data according to the collection object, collection parameters and alarm description information in the model file, and send the collected data to the processing device or management system, etc. Designated equipment.
  • the collecting data according to the model file may include the following steps:
  • Step 51 Determine the collection object, collection parameters, and alarm description information according to the model file.
  • the collection device can determine the collection object according to the resource model file, determine the collection parameters according to the performance model file, and determine the alarm description information according to the alarm model file.
  • the alarm description information can be parameters for alarm filtering or adaptation.
  • Step 52 Obtain data of the collection object, and receive alarm information sent by the collection object, where the data is performance data corresponding to the collection parameters of the collection object.
  • Step 53 Process the alarm information according to the alarm description information.
  • the received alarm information is filtered and adapted according to the parameters of alarm filtering or adaptation, and converted into alarm data in a unified model and a unified format.
  • the collection device can collect the collection parameters of the collection object according to the collection cycle, collect the collection objects through different docking protocols, and obtain different forms of original data.
  • the collection device can also receive alarm information sent by the collection object.
  • step 42 after collecting data according to the model file (i.e. step 42) and before sending the collected data to the designated device (i.e. step 43), it may also include: abstract processing of the collected data to unify The format of the data.
  • the collection device After the data is collected by the collection device, the original data of different forms is abstracted into monitoring data in a unified format through the data abstraction capability, which is convenient for the designated equipment to perform other applications based on the monitoring data. Because the collection device has universal data abstraction capabilities and protocol conversion capabilities, it can also be connected to various monitoring systems.
  • the embodiment of the present disclosure also provides a data collection method. As shown in FIG. 10, the method may include the following steps:
  • Step 101 in response to receiving the collection object parameters, collection parameters and alarm description information, generate a model file according to the collection object parameters, collection parameters and alarm description information, and the model file is used to define the data collection object, collection parameters and alarm description information; step 102. Send the model file to the processing device.
  • the user can define the model file, and upload the model file to the processing device through the manual interactive interface module.
  • the model file can include information necessary for collection such as data collection objects, collection parameters, and alarm description information.
  • collection device and the processing device provided in the embodiments of the present disclosure can be deployed in the same location or in different locations.
  • the collection device and the processing device can be deployed on the same cloud platform.
  • the collection device is connected to the collection object through a dedicated network.
  • the resource module, performance module, alarm module and UI module are also used as independent microservices. Deployed in the same cloud platform. In this way, the deployment and management of microservices are more convenient.
  • the collection device and the processing device can also be deployed on different cloud platforms.
  • the collection device is connected to the collection object through a dedicated network.
  • the resource module, performance module, alarm module, and UI module are also independent
  • the microservices and the adaptation/processing microservices are deployed on the same cloud platform, and the cloud platform where the collection microservices are located is also deployed with service module 1, service module 2, and other modules.
  • the collection device and the processing device can be operated and managed separately, and the resources of the two cloud platforms are used respectively, and the running status of the collection device and the running status of the processing device do not interfere with each other.
  • the collection device can also be deployed outside the cloud and deployed in the same network as the collection object, while the processing device is deployed on the cloud platform.
  • the resource module, performance module, alarm module, and UI module are also used as independent modules.
  • Microservices and adaptation/processing microservices are deployed on the same cloud platform. In this way, the deployment is relatively flexible and can adapt to scenarios with more complex network conditions.
  • the data collection method and monitoring system involved in the embodiments of the present disclosure are designed through a microservice architecture, using model definition files, script files, etc., to describe device characteristics, provide model driving capabilities, and define models for different collection protocols and collection logic, encapsulating each
  • the complexity of this collection protocol avoids coding to support new cloud resource objects. It can be based on bare equipment, IaaS (Infrastructure as a Service, infrastructure as a service), PaaS (Platform as a Service, platform as a service), etc.
  • Scenarios using model-driven data collection for servers, magnetic arrays, routers, switches, firewalls, load balancers and other hardware at the hardware layer, data collection for servers, distributed storage, and database software at the software layer, on the cloud platform Data collection for OpenStack (cloud computing service component) and VMware (virtual machine software). It can perform alarm monitoring and performance collection on related equipment and systems of the 5G cloud network in the complex 5G networking environment. Perform data abstraction on multiple types of collected data, and provide data processing templates, which can extract unified modeling data from multiple types of data.
  • the complex docking protocol can be converted into a simple and unified Restful interface for use by the monitoring system, which is convenient for docking with the monitoring system or analysis system, and the method of dynamically loading model files reduces the time and cost required for system customization or upgrade.
  • the model file can be uploaded online to make it effective dynamically, avoiding system restarts caused by deployment operations. It can support multiple deployment methods, which can be deployed as a collection service in the cloud or as a collection service outside the cloud, suitable for PaaS/ Various application scenarios such as IaaS/stand-alone.
  • an embodiment of the present disclosure further provides a processing device, including: a receiving module 701 and a sending module 702.
  • the receiving module 701 is used to receive a model file, and the model file is used to define data collection. Objects, collection parameters, and alarm description information.
  • the sending module 702 is configured to send the model file to the collection device, so that the collection device collects data according to the model file.
  • the processing device may further include an obtaining module, which is used to obtain access parameters of the collection object to be added.
  • the sending module 702 is further configured to send the access parameters to the general service module, so that the general service module persists the access parameters, and the general service module includes a resource module;
  • the access parameters are sent to the collection device, so that the collection device performs data collection on the collection objects to be added according to the access parameters and the model file.
  • the sending module 702 is also used to send the model file to the general service module, so that the general service module loads the model file.
  • the model file may include: a resource model file used to define data collection objects, a performance model file used to define collection parameters, and an alarm model file used to define alarm description information.
  • General service modules may include: resources Module, performance module and alarm module.
  • the sending module 702 is also used to send the resource model file to the resource module, so that the resource module loads the resource model file, and sends the performance model file to the performance module, so that the performance module loads the performance model file, and sends the alarm model file to Alarm module, so that the alarm module loads the alarm model file.
  • the model file further includes one or any combination of the following: a script file, a protocol configuration file, and a management information library file.
  • the sending module 702 is further configured to send the model file to the collection device through the expressive state transfer interface, and/or send the model file to the general service module through the expressive state transfer interface.
  • an embodiment of the present disclosure also provides a collection device, including: a receiving module 801, a collection module 802, and a sending module 803.
  • the receiving module 801 is used for receiving a model file sent by the processing device. Define collection objects, collection parameters, and alarm description information.
  • the collection module 802 is used to collect data according to the model file.
  • the sending module 803 is used to send the collected data to a designated device.
  • the collection module 802 is used to determine the collection object, collection parameters, and alarm description information according to the model file; obtain the data of the collection object, and receive the alarm information sent by the collection object, the data being the collection parameter of the collection object Corresponding performance data; processing alarm information according to alarm description information.
  • the collection device may further include a processing module, which is used to perform abstract processing on the collected data in a unified data format.
  • the embodiment of the present disclosure also provides a model definition device, including a generating module 901 and a sending module 902.
  • the generating module 901 is used to respond to receiving collection object parameters, collection parameters, and alarm description information, according to The collection object parameters, collection parameters, and alarm description information generate a model file.
  • the model file is used to define the data collection object, collection parameters, and alarm description information.
  • the sending module 902 is used to send the model file to the processing device.
  • the embodiment of the present disclosure also provides a computer device.
  • the computer device includes: one or more processors 1101 and a storage device 1102; wherein, the storage device 1102 stores one or more programs, when When the foregoing one or more programs are executed by the foregoing one or more processors 1101, the foregoing one or more processors 1101 implement the data collection methods provided in the foregoing embodiments.
  • the embodiments of the present disclosure also provide a computer-readable medium storing a computer program, where the computer program implements the data collection method provided in the foregoing embodiments when the computer program is executed.
  • the data collection method provided by the embodiment of the present disclosure receives a model file used to define data collection objects, collection parameters, and alarm description information, and sends the model file to the collection device, so that the collection device collects data according to the model file.
  • the embodiment of the present disclosure can use the model file to define the collection object.
  • the monitoring system and the collection object are in a decoupled relationship. In this way, the model file can take effect after the model file is uploaded to the monitoring system. Therefore, the joint debugging cycle can be shortened and the monitoring does not need to be restarted The system will not affect business operations.
  • the embodiment of the present disclosure can also reuse the model file definition in other monitoring systems without recoding and deployment.
  • the embodiments of the present disclosure can also update or repair the function of the monitoring system by redefining the model file, and quickly respond to changes in demand.
  • the embodiments of the present disclosure can be applied to collection objects of types such as central data centers and edge clouds, and can monitor cloud resources at various levels of IaaS, PaaS, and SaaS in various scenarios.
  • the functional modules/units in the device can be implemented as software, firmware, hardware, and appropriate combinations thereof.
  • the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may consist of several physical components.
  • the components are executed cooperatively.
  • Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit .
  • Such software may be distributed on a computer-readable medium
  • the computer-readable medium may include a computer storage medium (or non-transitory medium) and a communication medium (or transitory medium).
  • the term computer storage medium includes volatile and non-volatile data implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data).
  • Information such as computer-readable instructions, data structures, program modules, or other data.
  • Computer storage media include but are not limited to RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or Any other medium used to store desired information and that can be accessed by a computer.
  • communication media usually contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as carrier waves or other transmission mechanisms, and may include any information delivery media. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Environmental & Geological Engineering (AREA)
  • Debugging And Monitoring (AREA)
  • Computer And Data Communications (AREA)

Abstract

一种数据采集方法、装置、计算机设备及计算机可读介质，所述方法包括：接收用于定义数据采集对象、采集参数和告警描述信息的模型文件(S21)，并将模型文件发送给采集装置，以使采集装置根据模型文件采集数据(S22)。

Description

数据采集方法、装置、计算机设备及计算机可读介质
相关申请的交叉引用
本申请基于申请号为201911288997.4、申请日为2019年12月12日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本申请作为参考。
技术领域
本公开实施例涉及通信技术领域，具体涉及一种数据采集方法、装置、计算机设备及计算机可读介质。
背景技术
在网络环境下，监控系统是一种以计算机技术为基础的自动化系统，它可以对网络相关设备和系统进行数据采集，从而实现告警监控和性能采集等功能。
传统模式下，监控系统存在着很多弊端：对于新的采集对象，需要通过硬编码部署到现有的监控系统中，开发、联调周期较长，且部署完成后还需要重启监控系统，影响业务运行。对于现有的采集对象，如果需要部署到其他监控系统中使用，也需要重新对其进行编码和部署，效率低下。随着网络的发展，各种新需求不断增加，更新或修复监控系统的功能愈发困难，监控系统也难以快速响应需求。
随着5G(5th generation wireless systems，第五代移动通信技术)时代和云时代的到来，引入了不同层级的云化部署和网络部署，硬件、软件设备飞速发展，传统的监控系统更加难以及时支持对5G云化网络的设备和系统的监控。
发明内容
有鉴于此，本公开实施例提供一种数据采集方法、装置、计算机设备及计算机可读介质。
第一方面，本公开实施例提供一种数据采集方法，所述方法包括：接收模型文件，所述模型文件用于定义数据采集对象、采集参数和告警描述信息；将所述模型文件发送给采集装置，以使所述采集装置根据所述模型文件采集数据。
又一方面，本公开实施例还提供一种数据采集方法，所述方法包括：接收处理装置发送的模型文件，所述模型文件用于定义数据采集对象、采集参数和告警描述信息；根据所述模型文件采集数据；将采集到的数据发送给指定设备。
又一方面，本公开实施例还提供一种数据采集方法，所述方法包括：响应于接收到采 集对象参数、采集参数和告警描述信息，根据所述采集对象参数、采集参数和告警描述信息生成模型文件，所述模型文件用于定义数据采集对象、采集参数和告警描述信息；将所述模型文件发送给处理装置。
又一方面，本公开实施例还提供一种处理装置，包括：第一接收模块和第一发送模块，其中，所述第一接收模块用于，接收模型文件，所述模型文件用于定义数据采集对象、采集参数和告警描述信息；所述第一发送模块用于，将所述模型文件发送给采集装置，以使所述采集装置根据所述模型文件采集数据。
又一方面，本公开实施例还提供一种采集装置，包括：第二接收模块、采集模块和第二发送模块，其中，所述第二接收模块用于，接收处理装置发送的模型文件，所述模型文件用于定义数据采集对象、采集参数和告警描述信息；所述采集模块用于，根据所述模型文件采集数据；所述第二发送模块用于，将采集到的数据发送给指定设备。
又一方面，本公开实施例还提供一种模型定义装置，包括生成模块和发送模块，其中，所述生成模块用于，响应于接收到采集对象参数、采集参数和告警描述信息，根据所述采集对象参数、采集参数和告警描述信息生成模型文件，所述模型文件用于定义数据采集对象、采集参数和告警描述信息。所述发送模块用于，将所述模型文件发送给处理装置。
又一方面，本公开实施例还提供一种计算机设备，包括：一个或多个处理器以及存储装置；其中，存储装置上存储有一个或多个程序，当上述一个或多个程序被上述一个或多个处理器执行时，使得上述一个或多个处理器实现如前述各实施例所提供的数据采集方法。
本公开实施例还提供了一种计算机可读介质，存储有计算机程序，其中，该计算机程序被执行时实现如前述各实施例所提供的数据采集方法。
附图说明
图1为本公开一实施例提供的数据采集方法应用的系统结构图；
图2为本公开一实施例提供的数据采集方法的流程图；
图3为本公开一实施例提供的数据采集方法的又一流程图；
图4为本公开又一实施例提供的数据采集方法的流程图；
图5为本公开再一实施例提供的根据模型文件采集数据的流程图；
图6a为本公开又一实施例提供的部署方式示意图；
图6b为本公开再一实施例提供的部署方式示意图；
图6c为本公开又一实施例提供的部署方式示意图；
图7为本公开又一实施例提供的处理装置的结构示意图；
图8为本公开再一实施例提供的采集装置的结构示意图；
图9为本公开又一实施例提供的模型定义装置的结构示意图；
图10为本公开一实施例提供的数据采集方法的流程图；
图11为本公开一实施例提供的计算机设备的结构示意图。
具体实施方式
在下文中将参考附图更充分地描述本公开的一些实施例，但是所述实施例可以以不同形式来体现且不应当被解释为限于本文阐述的实施例。反之，提供这些实施例的目的在于使本公开透彻和完整，并将使本领域技术人员充分理解本公开的范围。
如本文所使用的，术语“和/或”包括一个或多个相关列举条目的任何和所有组合。
本文所使用的术语仅用于描述特定实施例，且不意欲限制本公开。如本文所使用的，单数形式“一个”和“该”也意欲包括复数形式，除非上下文另外清楚指出。还将理解的是，当本说明书中使用术语“包括”和/或“由……制成”时，指定存在所述特征、整体、步骤、操作、元件和/或组件，但不排除存在或添加一个或多个其他特征、整体、步骤、操作、元件、组件和/或其群组。
本文所述实施例可借助本公开的理想示意图而参考平面图和/或截面图进行描述。因此，可根据制造技术和/或容限来修改示例中的图示。因此，实施例不限于附图中所示的实施例，而是包括基于制造工艺而形成的配置的修改。因此，附图中例示的区具有示意性属性，并且图中所示区的形状例示了元件的区的具体形状，但并不旨在是限制性的。
除非另外限定，否则本文所用的所有术语(包括技术和科学术语)的含义与本领域普通技术人员通常理解的含义相同。还将理解，诸如那些在常用字典中限定的那些术语应当被解释为具有与其在相关技术以及本公开的背景下的含义一致的含义，且将不解释为具有理想化或过度形式上的含义，除非本文明确如此限定。
有鉴于现有的上述问题，本公开实施例提供了一种数据采集方法，如图1所示，所述方法应用于包括监控系统、采集装置和采集对象的系统中，监控系统包括处理装置和通用业务模块。其中，采集对象可以是通过某种通信协议能够被监控系统纳管的硬件或软件，例如路由器、交换机、防火墙、服务器、机架/机框、MySQL(My Structure Quest Language，关系型数据库管理系统)和PostgreSQL(Postgre Structured Query Language，对象关系型数据库管理系统)、Ubuntu(乌班图操作系统)和Windows(视窗操作系统)、Kafka(Apache Kafka，分布式发布订阅消息系统)和FTP(File Transfer Protocol，文件传输协议)等公共服务软件、OpenStack(云端计算服务组件)和VMware(一种虚拟机软件)等云平台。采集装置能够从采集对象处采集资源、性能和告警等数据，主要提供性能采集和告警监控这两种采集功能，性能采集是指通过SNMP(Simple Network Management Protocol，简单网 络管理协议)、SSH(Secure Shell，安全外壳协议)、JDBC(Java DataBase Connectivity java，数据库连接)等协议从采集对象获取性能数据，告警监听是指对于支持SNMP协议的采集对象监听告警。处理装置能够处理采集装置上传的数据，并负责将数据传递给网管、云管或第三方应用，处理装置通常可以作为网管、云管或第三方应用平台中的一个服务、微服务、插件来部署。通用业务模块可以是在网管或云管的监控系统中常用的业务模块。
如图2所示，本公开实施例提供的数据采集方法，可以包括以下步骤：
步骤21，接收模型文件，模型文件用于定义数据采集对象、采集参数和告警描述信息。
在本步骤中，处理装置可以接收用户以压缩文件形式上传的模型文件，压缩文件中可以包括多个模型文件，分别用来定义数据采集对象、采集参数和告警描述信息。可以通过定义资源中的网元类型(例如存储网元的字段)和资源中的部件类型(例如存储部件实例的字段)来定义数据采集对象，可以通过定义测量计数器等相关的性能信息来定义采集参数，例如定义采集参数为CPU(Central Processing Unit/Processor，中央处理器)使用率、CPU温度等等，可以通过定义各种异常情况对应的告警码来定义告警描述信息。
采集对象、采集参数和告警描述信息都是监控系统的采集装置在采集设备或系统的数据时所必需的信息，使用模型文件来定义数据采集对象、采集参数和告警描述信息，当需要修改或添加这三种信息时，可以直接在模型文件中进行修改或添加，因此，当这三种信息发生变化时，不再需要对采集对象进行重新编码和部署。
步骤22，将模型文件发送给采集装置，以使采集装置根据模型文件采集数据。
在本步骤中，处理装置可以将模型文件直接以压缩文件的形式发送给采集装置，采集装置从压缩文件中解析各模型文件，获取数据采集对象、采集参数和告警描述信息，进一步可以根据这些信息采集数据。
由于采集装置可以根据模型文件采集数据，如果需要增加采集需求(例如增加新类型的采集参数)，只需要在模型文件中增加相应信息。如果需要在其他同类型监控系统中监控采集对象，可以将该采集对象的模型文件复用其他监控系统中。如果需要支持采集新设备、定制设备或特殊设备的数据，可以通过修改或添加模型文件中的数据采集对象、采集参数和告警描述信息来实现。本公开实施例通过灵活定义模型文件、在线部署的方式，对各种设备、系统的采集更加快捷，并能方便地进行定制、在线部署即时生效。在接入新设备类型时，取得了效率上的进步，达到了快速开发、快速接入的效果，节省了开发、联调和部署的时间，提高了易用性。
通过上述步骤21-22可以发现，本公开实施例提供的数据采集方法，接收用于定义数据采集对象、采集参数和告警描述信息的模型文件，并将模型文件发送给采集装置，以使 采集装置根据模型文件采集数据。本公开实施例可以利用模型文件定义采集对象，监控系统与采集对象是解耦合关系，这样，将模型文件上传至监控系统后模型文件即可生效，因此可以缩短联调周期，并且不需要重启监控系统，也不会影响业务运行。本公开实施例还可以在其他监控系统中复用模型文件定义，不需要重新进行编码和部署。本公开实施例还可以通过重新定义模型文件来更新或修复监控系统的功能，快速响应需求变化。本公开实施例可以适用于中央数据中心和边缘云等类型的采集对象，能够在各种场景下对IaaS、PaaS、SaaS各个层次的云资源进行监控。
在一些实施例中，如图3所示，在将模型文件发送给采集装置之后，还可以包括以下步骤：
步骤31，获取待增加的采集对象的接入参数。
通用业务模块可以包括人机交互界面(User Interface，UI)模块，当第一次采集新增加的采集对象的数据时，可以从人机交互界面模块导入或输入待增加的采集对象的接入参数，处理装置通过人机交互界面模块获取待增加的采集对象的接入参数。
具体的，用户可以通过人机交互界面模块将模型文件上传至处理装置，在处理装置将模型文件发送给采集装置之后，人机交互界面模块可以根据模型文件中对资源字段的描述，动态生成参数输入字段，便于用户填写待增加的采集对象的接入参数，并将该接入参数发送给处理装置。
步骤32，将接入参数发送给通用业务模块，以使通用业务模块持久化接入参数，通用业务模块包括资源模块。
资源模块接收处理装置发送的待增加的采集对象的接入参数后，可以将该接入参数存储至数据库中，以便后续再采集该采集对象的数据时无需导入或输入接入参数。
步骤33，将接入参数发送给采集装置，以使采集装置根据接入参数和模型文件对待增加的采集对象进行数据采集。
在本步骤中，处理装置将接入参数发送给采集装置，以便采集装置可以根据该接入参数，配合处理装置发送的模型文件，自动解析采集逻辑，智能化采集该采集对象的数据。
在一些实施例中，在接收模型文件(即步骤21)之后，还可以包括：将模型文件发送给通用业务模块，以使通用业务模块加载模型文件。在本步骤中，处理装置接收到用户以压缩文件形式上传的模型文件之后，在将模型文件发送给采集装置的同时，将压缩文件进行解压，获得用于定义数据采集对象、采集参数和告警描述信息的模型文件，并将各模型文件发送给通用业务模块，以使通用业务模块加载模型文件，从而实现业务监控。
在一些实施例中，模型文件可以包括：用于定义数据采集对象的资源模型文件、用于 定义采集参数的性能模型文件和用于定义告警描述信息的告警模型文件。通用业务模块可以包括：资源模块、性能模块和告警模块。所述将模型文件发送给通用业务模块，以使通用业务模块加载模型文件(即步骤32)，可以包括：将资源模型文件发送给资源模块，以使资源模块加载资源模型文件，将性能模型文件发送给性能模块，以使性能模块加载性能模型文件，并将告警模型文件发送给告警模块，以使告警模块加载告警模型文件。
压缩文件中可以包括资源模型文件、性能模型文件和告警资源模型文件这三种模型文件，通用业务模块还可以包括资源模块、性能模块和告警模块这几种模块，资源模块可用于管理采集对象，性能模块可用于提供性能相关服务，告警模块可用于提供告警相关服务。处理装置在对压缩文件解压后，可以分别将三种模型文件发送给对应的模块，各模块动态加载对应的模型文件，以便向用户显示资源、性能和告警等相关信息。同时，存储资源、性能数据的数据库可以根据模型中对资源、性能字段的描述来动态生成不同的存储表。
在一些实施例中，模型文件还可以包括以下其中之一或任意组合：脚本文件、协议配置文件、管理信息库文件。
其中，脚本文件可以包括以下其中之一或任意组合：连接设备、采集数据使用的相关协议的执行脚本，对资源即接入参数字段的自定义脚本，对告警解析的特殊处理脚本，对性能数据的二次计算脚本，调用外部服务采集数据的脚本等自定义的特殊逻辑脚本。协议配置文件可以包括解析协议返回结果的配置文件等。模型文件还可以包括管理信息库(Management Information Base，MIB)文件。
需要说明的是，用户可以将上述脚本文件、协议配置文件、管理信息库文件可以与资源模型文件、性能模型文件和告警模型文件打包生成压缩包，将该压缩包发送给处理装置。
在一些实施例中，所述将模型文件发送给采集装置(即步骤22)，可以包括：通过表述性状态传递接口将模型文件发送给采集装置；和/或，将模型文件发送给通用业务模块，可以包括：通过表述性状态传递接口将模型文件发送给通用业务模块。
处理装置与采集装置以及处理装置与通用业务模块之间，可以通过统一定义的Restful(Representational State Transfer，表述性状态传递)接口连接，从而形成微服务架构，处理装置可以将模型文件或接入参数通过Restful接口发送给采集装置或通用业务模块。处理装置、采集装置和通用业务模块为独立的微服务模块，各微服务模块之间是松耦合关系，每个微服务模块仅专注于完成各自的任务。当监控系统增加新需求时，只需要添加或修改对应的微服务模块即可，更新或修复监控系统的功能变得轻松简单。并且，由于微服务架构非常适合云部署，因此能够与云端开发模式下的应用兼容。
本公开实施例还提供了一种数据采集方法，如图4所示，该方法可以包括以下步骤：
步骤41，接收处理装置发送的模型文件，模型文件用于定义数据采集对象、采集参数和告警描述信息。
在本步骤中，处理装置可以通过压缩文件形式将模型文件发送给采集装置。
步骤42，根据模型文件采集数据。
步骤43，将采集到的数据发送给指定设备。
采集装置接收处理装置以压缩文件形式发送的模型文件后，可直接根据模型文件中的采集对象、采集参数和告警描述信息来采集数据，并将采集到的数据发送给处理装置或管理系统等任意的指定设备。
在一些实施例中，如图5所示，所述根据模型文件采集数据(即步骤42)，可以包括以下步骤：
步骤51，根据模型文件确定采集对象、采集参数和告警描述信息。
在本步骤中，采集装置可以根据资源模型文件确定采集对象，根据性能模型文件确定采集参数，根据告警模型文件确定告警描述信息。告警描述信息可以是告警过滤或适配的参数。
步骤52，获取采集对象的数据，并接收采集对象发送的告警信息，所述数据为采集对象的采集参数对应的性能数据。
步骤53，根据告警描述信息处理告警信息。
在本步骤中，根据告警过滤或适配的参数，对接收到的告警信息进行过滤和适配等处理，将其转换为统一模型、统一格式的告警数据。
在本步骤中，采集装置可以按照采集周期采集采集对象的采集参数，通过不同的对接协议，对采集对象进行采集，获取到不同形式的原始数据。当有异常情况发生时，采集装置还可以接收采集对象发送的告警信息。
在一些实施例中，在根据模型文件采集数据(即步骤42)之后、将采集到的数据发送给指定设备(即步骤43)之前，还可以包括：对采集到的数据进行抽象处理，以统一数据的格式。
采集装置采集到数据之后，通过数据抽象能力，将不同形式的原始数据抽象为统一格式的监控数据，便于指定设备后续根据监控数据做其他应用。由于采集装置具备通用的数据抽象能力和协议转换能力，因此也可以对接各种监控系统。
本公开实施例还提供了一种数据采集方法，如图10所示，该方法可以包括以下步骤：
步骤101，响应于接收到采集对象参数、采集参数和告警描述信息，根据采集对象参数、采集参数和告警描述信息生成模型文件，模型文件用于定义数据采集对象、采集参数 和告警描述信息；步骤102，将模型文件发送给处理装置。
具体的，可以由用户定义模型文件，并将模型文件通过人工交互界面模块上传到处理装置。模型文件中可以包括数据采集对象、采集参数和告警描述信息等采集所必需的信息。
需要说明的是，本公开实施例提供的采集装置和处理装置可以部署在相同位置，也可以部署在不同位置。
如图6a所示，采集装置可以与处理装置部署在同一云平台中，采集装置通过专用网络与采集对象连接，同时，资源模块、性能模块、告警模块和UI模块也分别作为一个独立的微服务部署在同一云平台中。这种方式下，微服务的部署和管理都比较方便。
如图6b所示，采集装置也可以与处理装置分别部署在不同的云平台中，采集装置通过专用网络与采集对象连接，同时，资源模块、性能模块、告警模块和UI模块也分别作为一个独立的微服务与适配/处理微服务部署在同一云平台中，采集微服务所在的云平台还部署了服务模块1、服务模块2等等模块。这种方式下，可以分开运行和管理采集装置和处理装置，分别使用两个云平台的资源，采集装置的运行状态和处理装置的运行状态互不干扰。
如图6c所示，采集装置还可以部署在云外，与采集对象部署在同一网络中，而处理装置部署在云平台，同时，资源模块、性能模块、告警模块和UI模块也分别作为一个独立的微服务与适配/处理微服务部署在同一云平台中。这种方式下，部署相对灵活，可以适应网络情况较为复杂的场景。
本公开实施例涉及的数据采集方法和监控系统，通过微服务架构设计，采用模型定义文件、脚本文件等形式描述设备特征，提供模型驱动能力，针对不同采集协议和采集逻辑定义模型，封装了各种采集协议的复杂性，避免通过编码才能支持新的云资源对象，能够基于裸设备、IaaS(Infrastructure as a Service，基础设施即服务)、PaaS(Platform as a Service，平台即服务)等各种场景，利用模型驱动，在硬件层对服务器、磁阵、路由器、交换机、防火墙、负载均衡器等硬件进行数据采集，在软件层对服务器、分布式存储、数据库的软件进行数据采集，在云平台对OpenStack(云端计算服务组件)和VMware(虚拟机软件)等进行数据采集。能够在5G复杂的组网环境下，对5G云化网络的相关设备和系统进行告警监控、性能采集。对多种类型的采集数据进行数据抽象，并提供数据处理模板，能够从多种类型的数据中提取统一建模的数据。能够将复杂的对接协议转换为简单统一的Restful接口提供给监控系统使用，方便与监控系统或者分析系统对接，采用动态加载模型文件的方式降低系统定制或升级所需的时间和成本。能够通过在线上传模型文件，使之动态生效，避免部署操作带来的系统重启能够支持多种部署方式，即可部署为云中的采集服务，也可 部署为云外的采集服务，适合PaaS/IaaS/单机等各种应用场景。
基于相同的技术构思，如图7所示，本公开实施例还提供一种处理装置，包括：接收模块701和发送模块702，接收模块701用于，接收模型文件，模型文件用于定义数据采集对象、采集参数和告警描述信息。
发送模块702用于，将模型文件发送给采集装置，以使采集装置根据模型文件采集数据。
在一些实施例中，该处理装置还可以包括获取模块，获取模块用于，获取待增加的采集对象的接入参数。
发送模块702还用于，将接入参数发送给通用业务模块，以使通用业务模块持久化接入参数，通用业务模块包括资源模块；
将接入参数发送给采集装置，以使采集装置根据接入参数和模型文件对待增加的采集对象进行数据采集。
在一些实施例中，发送模块702还用于，将模型文件发送给通用业务模块，以使通用业务模块加载模型文件。
在一些实施例中，模型文件可以包括：用于定义数据采集对象的资源模型文件、用于定义采集参数的性能模型文件和用于定义告警描述信息的告警模型文件，通用业务模块可以包括：资源模块、性能模块和告警模块。
发送模块702还用于，将资源模型文件发送给资源模块，以使资源模块加载资源模型文件，将性能模型文件发送给性能模块，以使性能模块加载性能模型文件，并将告警模型文件发送给告警模块，以使告警模块加载告警模型文件。
在一些实施例中，模型文件还包括以下其中之一或任意组合：脚本文件、协议配置文件、管理信息库文件。
在一些实施例中，发送模块702还用于，通过表述性状态传递接口将模型文件发送给采集装置，和/或，通过表述性状态传递接口将模型文件发送给通用业务模块。
如图8所示，本公开实施例还提供了一种采集装置，包括：接收模块801、采集模块802和发送模块803，接收模块801用于，接收处理装置发送的模型文件，模型文件用于定义采集对象、采集参数和告警描述信息。
采集模块802用于，根据模型文件采集数据。
发送模块803用于，将采集到的数据发送给指定设备。
在一些实施例中，采集模块802用于，根据模型文件确定采集对象、采集参数和告警描述信息；获取采集对象的数据，并接收采集对象发送的告警信息，所述数据为采集对象 的采集参数对应的性能数据；根据告警描述信息处理告警信息。
在一些实施例中，该采集装置还可以包括处理模块，处理模块用于，对采集到的数据进行抽象处理，以统一数据的格式。
如图9所示，本公开实施例还提供了一种模型定义装置，包括生成模块901和发送模块902，生成模块901用于，响应于接收到采集对象参数、采集参数和告警描述信息，根据采集对象参数、采集参数和告警描述信息生成模型文件，模型文件用于定义数据采集对象、采集参数和告警描述信息。
发送模块902用于，将模型文件发送给处理装置。
本公开实施例还提供了一种计算机设备，如图11所述，该计算机设备包括：一个或多个处理器1101以及存储装置1102；其中，存储装置1102上存储有一个或多个程序，当上述一个或多个程序被上述一个或多个处理器1101执行时，使得上述一个或多个处理器1101实现如前述各实施例所提供的数据采集方法。
本公开实施例还提供了一种计算机可读介质，存储有计算机程序，其中，该计算机程序被执行时实现如前述各实施例所提供的数据采集方法。
本公开实施例提供的数据采集方法，接收用于定义数据采集对象、采集参数和告警描述信息的模型文件，并将模型文件发送给采集装置，以使采集装置根据模型文件采集数据。本公开实施例可以利用模型文件定义采集对象，监控系统与采集对象是解耦合关系，这样，将模型文件上传至监控系统后模型文件即可生效，因此可以缩短联调周期，并且不需要重启监控系统，也不会影响业务运行。本公开实施例还可以在其他监控系统中复用模型文件定义，不需要重新进行编码和部署。本公开实施例还可以通过重新定义模型文件来更新或修复监控系统的功能，快速响应需求变化。本公开实施例可以适用于中央数据中心和边缘云等类型的采集对象，能够在各种场景下对IaaS、PaaS、SaaS各个层次的云资源进行监控。
本领域普通技术人员可以理解，上文中所公开方法中的全部或某些步骤、装置中的功能模块/单元可以被实施为软件、固件、硬件及其适当的组合。在硬件实施方式中，在以上描述中提及的功能模块/单元之间的划分不一定对应于物理组件的划分；例如，一个物理组件可以具有多个功能，或者一个功能或步骤可以由若干物理组件合作执行。某些物理组件或所有物理组件可以被实施为由处理器，如中央处理器、数字信号处理器或微处理器执行的软件，或者被实施为硬件，或者被实施为集成电路，如专用集成电路。这样的软件可以分布在计算机可读介质上，计算机可读介质可以包括计算机存储介质(或非暂时性介质)和通信介质(或暂时性介质)。如本领域普通技术人员公知的，术语计算机存储介质包括在用于存储信息(诸如计算机可读指令、数据结构、程序模块或其他数据)的任何方法或 技术中实施的易失性和非易失性、可移除和不可移除介质。计算机存储介质包括但不限于RAM、ROM、EEPROM、闪存或其他存储器技术、CD-ROM、数字多功能盘(DVD)或其他光盘存储、磁盒、磁带、磁盘存储或其他磁存储装置、或者可以用于存储期望的信息并且可以被计算机访问的任何其他的介质。此外，本领域普通技术人员公知的是，通信介质通常包含计算机可读指令、数据结构、程序模块或者诸如载波或其他传输机制之类的调制数据信号中的其他数据，并且可包括任何信息递送介质。
本文已经公开了一些实施例，并且虽然采用了具体术语，但它们仅用于并仅应当被解释为一般说明性含义，并且不用于限制的目的。在一些实例中，对本领域技术人员显而易见的是，除非另外明确指出，否则可单独使用与特定实施例相结合描述的特征、特性和/或元素，或可与其他实施例相结合描述的特征、特性和/或元件组合使用。因此，本领域技术人员将理解，在不脱离由所附的权利要求阐明的本公开的范围的情况下，可进行各种形式和细节上的改变。

Claims (15)

  1. 一种数据采集方法，包括：
    接收模型文件，所述模型文件用于定义数据采集对象、采集参数和告警描述信息；
    将所述模型文件发送给采集装置，以使所述采集装置根据所述模型文件采集数据。
  2. 如权利要求1所述的方法，在所述将所述模型文件发送给采集装置之后，还包括：
    获取待增加的采集对象的接入参数；
    将所述接入参数发送给通用业务模块，以使所述通用业务模块持久化所述接入参数，所述通用业务模块包括资源模块；
    将所述接入参数发送给所述采集装置，以使所述采集装置根据所述接入参数和所述模型文件对所述待增加的采集对象进行数据采集。
  3. 如权利要求1所述的方法，在所述接收模型文件之后，还包括：
    将所述模型文件发送给通用业务模块，以使所述通用业务模块加载所述模型文件。
  4. 如权利要求3所述的方法，其中，所述模型文件包括：用于定义数据采集对象的资源模型文件、用于定义采集参数的性能模型文件和用于定义告警描述信息的告警模型文件，所述通用业务模块包括：资源模块、性能模块和告警模块；
    所述将所述模型文件发送给通用业务模块，以使所述通用业务模块加载所述模型文件，包括：
    将所述资源模型文件发送给资源模块，以使所述资源模块加载所述资源模型文件，将所述性能模型文件发送给性能模块，以使所述性能模块加载所述性能模型文件，并将所述告警模型文件发送给告警模块，以使所述告警模块加载所述告警模型文件。
  5. 如权利要求4所述的方法，其中，所述模型文件还包括以下其中之一或任意组合：脚本文件、协议配置文件、管理信息库文件。
  6. 如权利要求2-5任一项所述的方法，其中，所述将所述模型文件发送给采集装置，包括：通过表述性状态传递接口将所述模型文件发送给采集装置；和/或，所述将所述模型文件发送给通用业务模块，包括：通过表述性状态传递接口将所述模型文件发送给通用业务模块。
  7. 一种数据采集方法，包括：
    接收处理装置发送的模型文件，所述模型文件用于定义数据采集对象、采集参数和告警描述信息；
    根据所述模型文件采集数据；
    将采集到的数据发送给指定设备。
  8. 如权利要求7所述的方法，其中，所述根据所述模型文件采集数据，包括：
    根据所述模型文件确定采集对象、采集参数和告警描述信息；
    获取所述采集对象的数据，并接收所述采集对象发送的告警信息，所述数据为所述采集对象的采集参数对应的性能数据；
    根据所述告警描述信息处理所述告警信息。
  9. 如权利要求7或8所述的方法，在所述根据所述模型文件采集数据之后、将采集到的数据发送给指定设备之前，还包括：
    对所述采集到的数据进行抽象处理，以统一所述数据的格式。
  10. 一种数据采集方法，包括：
    响应于接收到采集对象参数、采集参数和告警描述信息，根据所述采集对象参数、采集参数和告警描述信息生成模型文件，所述模型文件用于定义数据采集对象、采集参数和告警描述信息；
    将所述模型文件发送给处理装置。
  11. 一种处理装置，包括：接收模块和发送模块，其中，所述接收模块用于，接收模型文件，所述模型文件用于定义数据采集对象、采集参数和告警描述信息；
    所述发送模块用于，将所述模型文件发送给采集装置，以使所述采集装置根据所述模型文件采集数据。
  12. 一种采集装置，包括：接收模块、采集模块和发送模块，其中，所述接收模块用于，接收处理装置发送的模型文件，所述模型文件用于定义数据采集对象、采集参数和告警描述信息；
    所述采集模块用于，根据所述模型文件采集数据；
    所述发送模块用于，将采集到的数据发送给指定设备。
  13. 一种模型定义装置，包括生成模块和发送模块，其中，所述生成模块用于，响应于接收到采集对象参数、采集参数和告警描述信息，根据所述采集对象参数、采集参数和告警描述信息生成模型文件，所述模型文件用于定义数据采集对象、采集参数和告警描述信息；
    所述发送模块用于，将所述模型文件发送给处理装置。
  14. 一种计算机设备，包括：
    一个或多个处理器；
    存储装置，存储有一个或多个程序；其中，
    当所述一个或多个程序被所述一个或多个处理器执行时，使得所述一个或多个处理器实现如权利要求1-10任一项所述的数据采集方法。
  15. 一种计算机可读介质，存储有计算机程序，其中，所述程序被执行时实现如权利 要求1-10任一项所述的数据采集方法。
PCT/CN2020/117255 2019-12-12 2020-09-23 数据采集方法、装置、计算机设备及计算机可读介质 WO2021114812A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP20898040.9A EP3975481B1 (en) 2019-12-12 2020-09-23 Data acquisition method and apparatus, computer device, and computer-readable medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911288997.4A CN112968789B (zh) 2019-12-12 2019-12-12 数据采集方法、装置、计算机设备及计算机可读介质
CN201911288997.4 2019-12-12

Publications (1)

Publication Number Publication Date
WO2021114812A1 true WO2021114812A1 (zh) 2021-06-17

Family

ID=76270862

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/117255 WO2021114812A1 (zh) 2019-12-12 2020-09-23 数据采集方法、装置、计算机设备及计算机可读介质

Country Status (3)

Country Link
EP (1) EP3975481B1 (zh)
CN (1) CN112968789B (zh)
WO (1) WO2021114812A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113688100A (zh) * 2021-09-06 2021-11-23 北京普睿德利科技有限公司 一种气象数据的处理方法、装置、终端及存储介质
CN114124643A (zh) * 2021-11-10 2022-03-01 中盈优创资讯科技有限公司 一种基于PaaS的网络设备流量采集方法及装置
CN114598602A (zh) * 2022-03-09 2022-06-07 树根互联股份有限公司 数据获取方法、装置、计算机设备及计算机可读存储介质

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114244678B (zh) * 2021-12-15 2024-03-12 浪潮通信信息系统有限公司 多协议告警设备接入方法及系统
CN113992514B (zh) * 2021-12-29 2022-04-01 北京万维盈创科技发展有限公司 一种零代码快速实现数采仪协议的方法及数采仪终端
CN114389960B (zh) * 2022-01-04 2023-11-28 烽火通信科技股份有限公司 一种网络业务性能采集上报的方法和系统
CN114338800B (zh) * 2022-03-08 2022-06-17 杭州义益钛迪信息技术有限公司 数据流组态方法、装置、电子设备及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130093592A1 (en) * 2011-10-14 2013-04-18 Zehua Lan Internet of Things Based Farm Greenhouse Monitor and Alarm Management System
CN105610648A (zh) * 2016-01-11 2016-05-25 飞天诚信科技股份有限公司 一种运维监控数据的采集方法及服务器
CN105763352A (zh) * 2014-12-16 2016-07-13 中兴通讯股份有限公司 一种数据的采集方法、装置及网管管理通信设备
CN106444662A (zh) * 2016-09-23 2017-02-22 东莞团诚自动化设备有限公司 一种用于物联网的数据采集装置及方法
CN107528870A (zh) * 2016-06-22 2017-12-29 腾讯科技（深圳）有限公司 一种数据采集方法及其设备
CN108763040A (zh) * 2018-04-25 2018-11-06 郑州易通众联电子科技有限公司 数据采集方法及数据采集装置
CN110543409A (zh) * 2019-08-29 2019-12-06 南方电网数字电网研究院有限公司 硬件数据采集方法、装置、计算机设备和存储介质

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105306243A (zh) * 2014-07-22 2016-02-03 中兴通讯股份有限公司 过期数据采集方法、装置及系统
US9948521B2 (en) * 2016-01-11 2018-04-17 Equinix, Inc. Architecture for data center infrastructure monitoring
JP6326705B2 (ja) * 2016-11-21 2018-05-23 インテル・コーポレーション テスト、検証及びデバッグアーキテクチャのプログラム及び方法
US10904173B2 (en) * 2017-06-09 2021-01-26 Equinix, Inc. Near real-time messaging service for data center infrastructure monitoring data
CN107276813A (zh) * 2017-06-30 2017-10-20 郑州云海信息技术有限公司 一种snmp‑trap告警方法及装置
CN108880867B (zh) * 2018-05-31 2020-09-04 腾讯科技（深圳）有限公司 一种网络设备数据采集方法及系统

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130093592A1 (en) * 2011-10-14 2013-04-18 Zehua Lan Internet of Things Based Farm Greenhouse Monitor and Alarm Management System
CN105763352A (zh) * 2014-12-16 2016-07-13 中兴通讯股份有限公司 一种数据的采集方法、装置及网管管理通信设备
CN105610648A (zh) * 2016-01-11 2016-05-25 飞天诚信科技股份有限公司 一种运维监控数据的采集方法及服务器
CN107528870A (zh) * 2016-06-22 2017-12-29 腾讯科技（深圳）有限公司 一种数据采集方法及其设备
CN106444662A (zh) * 2016-09-23 2017-02-22 东莞团诚自动化设备有限公司 一种用于物联网的数据采集装置及方法
CN108763040A (zh) * 2018-04-25 2018-11-06 郑州易通众联电子科技有限公司 数据采集方法及数据采集装置
CN110543409A (zh) * 2019-08-29 2019-12-06 南方电网数字电网研究院有限公司 硬件数据采集方法、装置、计算机设备和存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3975481A4 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113688100A (zh) * 2021-09-06 2021-11-23 北京普睿德利科技有限公司 一种气象数据的处理方法、装置、终端及存储介质
CN113688100B (zh) * 2021-09-06 2023-07-18 北京普睿德利科技有限公司 一种气象数据的处理方法、装置、终端及存储介质
CN114124643A (zh) * 2021-11-10 2022-03-01 中盈优创资讯科技有限公司 一种基于PaaS的网络设备流量采集方法及装置
CN114124643B (zh) * 2021-11-10 2023-08-01 中盈优创资讯科技有限公司 一种基于PaaS的网络设备流量采集方法及装置
CN114598602A (zh) * 2022-03-09 2022-06-07 树根互联股份有限公司 数据获取方法、装置、计算机设备及计算机可读存储介质

Also Published As

Publication number Publication date
EP3975481A4 (en) 2022-09-07
CN112968789B (zh) 2024-02-27
EP3975481B1 (en) 2024-06-26
CN112968789A (zh) 2021-06-15
EP3975481A1 (en) 2022-03-30

Similar Documents

Publication Publication Date Title
WO2021114812A1 (zh) 数据采集方法、装置、计算机设备及计算机可读介质
US20230244454A1 (en) Software defined network controller
EP3886367B1 (en) Automating 5g slices using real-time analytics
US9311161B2 (en) Automatically configured management service payloads for cloud IT services delivery
WO2017161984A1 (zh) 数据集群的部署方法、装置、系统及计算机存储介质
CN110413346B (zh) 一种参数更新方法及装置
US8874704B2 (en) Semi-automatic discovery and generation of useful service blueprints
US9876703B1 (en) Computing resource testing
US10284418B2 (en) Network switch management via embedded management controller using management information base (MIB) to JSON parser
CN110532020A (zh) 一种微服务编排的数据处理方法、装置及系统
JPWO2016117697A1 (ja) ネットワーク機能仮想化管理およびオーケストレーション方法と装置とプログラム
KR102328938B1 (ko) 전자 시스템들에서의 로그 데이터의 관리
CN107864053B (zh) 基于zookeeper的动态配置方法和装置
Brandón et al. Fmone: A flexible monitoring solution at the edge
CN105052076B (zh) 一种基于云计算的网元管理系统及网元管理方法
CN111064626B (zh) 配置更新方法、装置、服务器及可读存储介质
US20220012097A1 (en) Resource determination based on resource definition data
WO2016115896A1 (zh) 虚拟网络功能vnf的管理方法及装置
US20240205809A1 (en) Multi-component configurations in a ran system
US12035156B2 (en) Communication method and apparatus for plurality of administrative domains
US20170091351A1 (en) Device, System, and Method for Adaptive Simulation
WO2017016386A1 (zh) 一种实现射频拉远单元故障分析的方法和装置
US20240276188A1 (en) Apparatus and method for implementing r1-o1 data models for o1 related services within a telecommunications network
US10067955B1 (en) Custom video metrics management platform
WO2016086704A1 (zh) 实现光传送网业务的管理平台、系统、方法及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20898040

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020898040

Country of ref document: EP

Effective date: 20211223

NENP Non-entry into the national phase

Ref country code: DE