CN111385365A - Processing method and device for reported data, computer equipment and storage medium - Google Patents

Processing method and device for reported data, computer equipment and storage medium Download PDF

Info

Publication number
CN111385365A
CN111385365A CN202010209894.0A CN202010209894A CN111385365A CN 111385365 A CN111385365 A CN 111385365A CN 202010209894 A CN202010209894 A CN 202010209894A CN 111385365 A CN111385365 A CN 111385365A
Authority
CN
China
Prior art keywords
data
current
service
information
access service
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010209894.0A
Other languages
Chinese (zh)
Other versions
CN111385365B (en
Inventor
刘舟
郭文杰
朱育华
李奕臻
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Jisheng Network Technology Co ltd
Original Assignee
Guangzhou Jisheng Network Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Jisheng Network Technology Co ltd filed Critical Guangzhou Jisheng Network Technology Co ltd
Priority to CN202010209894.0A priority Critical patent/CN111385365B/en
Publication of CN111385365A publication Critical patent/CN111385365A/en
Application granted granted Critical
Publication of CN111385365B publication Critical patent/CN111385365B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/2866Architectures; Arrangements
    • H04L67/30Profiles
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1097Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/14Session management
    • H04L67/146Markers for unambiguous identification of a particular session, e.g. session cookie or URL-encoding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/56Provisioning of proxy services
    • H04L67/565Conversion or adaptation of application format or content

Abstract

The application relates to a processing method, a device, computer equipment and a storage medium of reported data, wherein the method comprises the following steps: receiving current reported data of a current access service, wherein the current access service is any one of a plurality of access services accessed to a monitoring system, the current reported data is configured in a data format according to unified configuration information of the plurality of access services, and the current reported data contains identification information of the current access service; determining a pre-configured data storage format, wherein the data storage format is set according to the unified configuration information of a plurality of access services; extracting identification information of the current access service from the current reported data; and performing data storage on the current reported data according to the identification information and the data storage format. The method can carry out unified storage management on the reported data reported by each service providing device.

Description

Processing method and device for reported data, computer equipment and storage medium
Technical Field
The present application relates to the technical field of service data processing, and in particular, to a method and an apparatus for processing reported data, a computer device, and a storage medium.
Background
In the process of providing the service, the service providing device may have some abnormal conditions in the operation stage due to some factors which cannot be controlled and predicted, so that the normal service cannot be provided. Research and development personnel like these situations all need to grasp in time so as to check and solve problems in time, so data monitoring needs to be carried out on the service providing process of the service providing equipment.
Currently, various service data need to be monitored in many application scenarios. Such as service data monitoring of operating systems, service data monitoring of game processes, service data monitoring of background processes, service data monitoring of databases, etc. However, since the number of services is large and the number of services is large, it is impossible to individually process monitoring by each service providing apparatus. Therefore, a unified monitoring apparatus is needed to perform unified monitoring management on the service data of each service providing device.
However, the reported data reported by each service providing device may be different, and the data format of the reported data may also be diversified, so that it is difficult for the unified monitoring apparatus to perform storage management on the reported data of multiple service providing devices, which will also result in that it is impossible to implement unified monitoring management on the service data of each service providing device by the unified monitoring apparatus.
Disclosure of Invention
Therefore, in order to solve the above technical problems, it is necessary to provide a method and an apparatus for processing reported data, a computer device, and a storage medium, which are capable of performing unified storage management on the reported data reported by each service providing device.
A processing method for reporting data comprises the following steps: receiving current reported data of a current access service, wherein the current access service is any one of a plurality of access services accessed to a monitoring system, the current reported data is configured in a data format according to unified configuration information of the plurality of access services, and the current reported data contains identification information of the current access service; determining a pre-configured data storage format, wherein the data storage format is set according to the unified configuration information of a plurality of access services; extracting identification information of the current access service from the current reported data; and performing data storage on the current reported data according to the identification information and the data storage format.
In one embodiment, the method for processing the reported data further includes: setting a data storage format in a relational database according to the unified configuration information of a plurality of access services; and performing data storage on the current reported data according to the identification information and the data storage format, wherein the data storage method comprises the following steps: acquiring service retrieval information of the current access service according to the identification information; if the relation type database is detected to contain first historical report data of the current access service stored according to the data storage format according to the service retrieval information, storing the current report data into the relation type database in a mode of replacing the first historical report data with the current report data; and if the relational database does not contain the first historical reported data of the current access service according to the service retrieval information, storing the current reported data into the relational database according to the data storage format.
In one embodiment, the method for processing the reported data further includes: acquiring identification information of a current access service; acquiring first display configuration information of the current access service according to the identification information; acquiring current reported data in the relational database according to the identification information; and performing data display on the current reported data according to the first display configuration information.
In one embodiment, the method for processing the reported data further includes: setting a data storage format in a time sequence database according to the unified configuration information of a plurality of access services; and performing data storage on the current reported data according to the identification information and the data storage format, wherein the data storage method comprises the following steps: determining a target storage table corresponding to a data storage format contained in the time sequence database according to the identification information; acquiring current time information; associating the current reported data with the current time information and then storing the current reported data and the current time information into a target storage table according to a data storage format;
preferably, the processing method of the reported data further includes: determining second historical reported data of the current access service in a preset time period from the current time in the target storage table; and deleting the historical data of the current access service except the second historical reported data in the target storage table.
In one embodiment, the method for processing the reported data further includes: acquiring identification information of a current access service; acquiring second display configuration information of the current access service according to the identification information; acquiring third history report data of the current access service stored in a target storage table according to the identification information; and displaying the data of the third history reported data according to the second display configuration information.
In one embodiment, the method for processing the reported data further includes: reading unified configuration information of a plurality of access services, wherein the unified configuration information comprises service identification information, service retrieval information and service business information; and setting a data storage format according to the identification information, the service retrieval information and the service business information of the service.
In one embodiment, receiving currently reported data of a current access service includes: reading data reporting configuration information of the current access service from the unified configuration information of the plurality of access services; if the data reporting mode of the current access service is determined to be the active reporting mode according to the data reporting configuration information, receiving the current reporting data of the current access service when the data reporting request of the current access service is determined to be received; and if the data reporting mode of the current access service is determined to be the passive reporting mode according to the data reporting configuration information, acquiring interface information of the data providing equipment corresponding to the current access service, and acquiring the current reporting data of the current access service from the data providing equipment according to the interface information so as to receive the current reporting data of the current access service.
A processing apparatus for reporting data, the apparatus comprising: a receiving module, configured to receive currently reported data of a current access service, where the current access service is any one of multiple access services accessed to a monitoring system, the currently reported data is configured in a data format according to unified configuration information of the multiple access services, and the currently reported data includes identification information of the current access service; the determining module is used for determining a pre-configured data storage format, and the data storage format is set according to the unified configuration information of the plurality of access services; the extraction module is used for extracting the identification information of the current access service from the current reported data; and the storage module is used for storing the data of the current reported data according to the identification information and the data storage format.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method of any of the above embodiments when executing the computer program.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of any of the above embodiments.
According to the processing method and device of the reported data, the computer equipment and the storage medium, the monitoring system receives the current reported data of the current access service. The current access service is any one of a plurality of access services accessed to the monitoring system, the current reported data is configured in a data format according to the unified configuration information of the plurality of access services, and the current reported data contains the identification information of the current access service. Furthermore, the monitoring system determines a pre-configured data storage format, the data storage format is set by the monitoring system according to the unified configuration information of the multiple access services, meanwhile, the identification information of the current access service is extracted from the current reported data, and finally, the data storage is carried out on the current reported data according to the identification information and the data storage format. Therefore, the monitoring system stores the reported data of any one of the plurality of access services according to the data storage format obtained by setting the unified configuration information according to the identification information of the any one of the plurality of access services, so that the monitoring system can carry out unified storage management on the reported data of the plurality of access services, the format unification of the reported data reported to the monitoring system is ensured, and the unified storage management on the reported data of the access services of various types of services is realized.
Drawings
Fig. 1 is an application environment diagram of a processing method for reporting data according to an embodiment;
fig. 2 is a schematic flow chart illustrating a method for processing reported data according to an embodiment;
fig. 3 is a schematic flow chart illustrating a method for processing reported data according to another embodiment;
fig. 4 is a schematic flowchart illustrating a processing method for reporting data according to yet another embodiment;
fig. 5 is a schematic flowchart illustrating a method for processing reported data according to another embodiment;
FIG. 6 is a schematic flow chart of S102 in one embodiment;
FIG. 7 is a schematic illustration of a display of a data monitoring interface of the monitoring system in one embodiment;
FIG. 8 is a schematic illustration of a display of a data monitoring interface of a monitoring system in accordance with another embodiment;
fig. 9 is a block diagram of a processing device for reporting data according to an embodiment;
FIG. 10 is a diagram showing an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The method for processing the reported data is applied to the application environment shown in fig. 1. As shown in fig. 1, a monitoring system 102 performs unified data monitoring on reported data reported by a plurality of service providing devices. The plurality of service providing apparatuses may include the service providing apparatus 202, the service providing apparatus 204, the service providing apparatus 206, the service providing apparatus 208, and the like, each of which provides a different service. For example, the service providing device 202 is a device providing game process services, the service providing device 204 is a device providing background process services, the service providing device 206 is a device providing game operating system services, and the service providing device 208 is a device providing database services. The service providing device reports the service data to be monitored locally to the monitoring system 102, so that the monitoring system 102 performs monitoring management on the service of each service providing device.
However, before performing unified data monitoring on each service providing device, the monitoring system 102 needs to perform data management on the reported data reported by each service providing device, for example, perform unified storage management on the reported data reported by each service providing device. The monitoring system 102 performs data processing on the reported data reported by each service providing device by using the reported data processing method of the present application. Specifically, the monitoring system 102 is in communication connection with each service providing device to receive reported data corresponding to the access service of each service providing device. The monitoring system 102 sets unified configuration information for each access service, and sets a data storage format according to the unified configuration information, and the reported data corresponding to each access service can be stored according to the data storage format. However, the data formats of the reported data corresponding to different access services may be different, and at this time, the monitoring system 102 identifies the corresponding access service according to the identification information of the access service in the reported data, and then performs data storage on the reported data according to the identification information and the data storage format, so that the monitoring system 102 can perform data unified storage management on the reported data reported by the access services of various types.
In an embodiment, as shown in fig. 2, a processing method for reporting data is provided, which is described by taking the example that the method is applied to the monitoring system in fig. 1, and includes the following steps:
s102, receiving current reported data of a current access service, wherein the current access service is any one of a plurality of access services accessed to a monitoring system, the current reported data is configured in a data format according to unified configuration information of the plurality of access services, and the current reported data contains identification information of the current access service.
In this embodiment, the monitoring system is connected to a plurality of service providing devices to access a plurality of access services. Meanwhile, the monitoring system is provided with unified configuration information aiming at the plurality of access services, and the monitoring system carries out unified management on the plurality of access services according to the unified configuration information. The unified configuration information may include one or more items of information in the service identification information, the service retrieval information, the service information, and the service data reporting configuration information. The identification information of the service may be a service name or a service internal code or any service description information. The service retrieval information can be service key retrieval fields which are used for distinguishing differences among services, all the service key retrieval fields can be positioned to a unique service business field of one service, and the service key retrieval fields are also used for retrieval optimization and storage of subsequent data. The service business information may be a service business field, and the service business field is used for collecting various difference indexes among services by the monitoring system, and the indexes are not generally used for determining the unique service but only used for data viewing and data range retrieval. The data reporting configuration information may be a service data reporting mode. The service data reporting mode comprises an active reporting mode and a passive reporting mode. The active reporting mode requires that the sending information of the service providing device is serialized into a URL (uniform resource Locator) specified by a receiving module of the monitoring system, and the passive reporting mode requires that the monitoring system provides an interface which is open to the outside and is used for a scanning module of the monitoring system to perform active data acquisition on the service providing device.
After the monitoring system sets the uniform configuration information of the multiple access services, the service providing device also needs to perform data format configuration on the reported data to be reported according to the uniform configuration information. Meanwhile, in order to identify the access service corresponding to the reported data, the service providing equipment encapsulates the identification information representing the access service into the reported data. The monitoring system receives the current reported data of the current access service, and can read the identification information from the current reported data so as to identify the current access service according to the identification information.
S104, determining a preset data storage format, wherein the data storage format is set according to the unified configuration information of the plurality of access services.
In this embodiment, the monitoring system is preconfigured with a data storage format for a plurality of access services. The data storage format is set according to the unified configuration information of the plurality of access services, and the reported data of any access service can be stored according to the data storage format, so that the monitoring system can realize the unified storage management of the reported data of the plurality of access services.
Specifically, after the monitoring system sets the uniform configuration information of the plurality of access services, the monitoring system creates a corresponding data storage format in the storage module according to the uniform configuration information. For example, when the unified configuration information includes a service internal code, a service key retrieval field, and a service business field, the created data storage format may be < service internal code, service key retrieval field 1, 2, 3 … … N, service business field 1, 2, 3 … … N >. Specifically, a table containing a data storage format of < service internal code, service key retrieval field 1.. n, service business field 1.. n > is created in the storage module. In addition, the service key retrieval field 1.. n is used as an index table in the storage module. Thus, the pre-configuration of the data storage format is completed. When the monitoring system receives the current reported data of the current access service, the preset data storage format can be directly determined.
S106, extracting the identification information of the current access service from the current reported data.
In this embodiment, after receiving the currently reported data of the current access service, the monitoring system analyzes the identifier information of the current access service from the currently reported data. The identification information is used for uniquely determining the current access service. Therefore, the monitoring system can identify the current access service according to the identification information, so as to perform corresponding data processing on the current reported data. The identification information of the current access service may be a service internal code defined by the monitoring system, or a service name of the current access service, or other description information for describing the current access service.
And S108, performing data storage on the current reported data according to the identification information and the data storage format.
In this embodiment, the service types of the multiple access services in the monitoring system may be different, and the data formats of the reported data corresponding to the access services may also be different. When the monitoring system uses a uniform data storage format to store data of reported data, corresponding storage adjustment needs to be performed according to different access services. Therefore, the monitoring system can perform data storage on the currently reported data according to the identification information and the data storage format of the currently accessed service, so that the storage mode of the currently reported data of the currently accessed service not only meets the unified management requirement of the monitoring system on the data storage, but also can reserve the personalized storage requirement of the currently reported data of the currently accessed service.
Specifically, the monitoring system can determine the data storage mode of the current access service according to the identification information and the data storage format, and then the current reported data of the current access service is stored according to the data storage mode of the current access service. For example, the unified data storage format includes a service internal code field, a service key retrieval field, and a service traffic field. And after identifying the current access service according to the identification information, the monitoring system determines a first number of service key retrieval fields and a second number of service business fields of the current access service according to the data storage format. And determining a data storage mode of the current access service according to the service internal code field, the first quantity of service key retrieval fields and the second quantity of service business fields, and further performing data storage on the current reported data of the current access service according to the data storage mode. For example, the data storage manner may be < service internal code, a quantity value of a first number of service key retrieval fields 1, 2, 3 … …, a quantity value of a second number of service traffic fields 1, 2, 3 … … >.
In the method for processing the reported data, the monitoring system receives the current reported data of the current access service. The current access service is any one of a plurality of access services accessed to the monitoring system, the current reported data is configured in a data format according to the unified configuration information of the plurality of access services, and the current reported data contains the identification information of the current access service. Furthermore, the monitoring system determines a pre-configured data storage format, the data storage format is set by the monitoring system according to the unified configuration information of the multiple access services, meanwhile, the identification information of the current access service is extracted from the current reported data, and finally, the data storage is carried out on the current reported data according to the identification information and the data storage format. Therefore, the monitoring system stores the reported data of any access service according to the data storage format obtained by setting the uniform configuration information according to the identification information of any access service, so that the monitoring system can perform uniform storage management on the reported data of a plurality of access services, the uniform format of the reported data reported to the monitoring system is ensured, and the uniform storage management on the reported data of the access services of various types of services is realized.
In one embodiment, as shown in fig. 3, the step of S104 further comprises:
s1032, setting a data storage format in the relational database according to the unified configuration information of the plurality of access services.
At this time, S108 includes the steps of:
s1082, service retrieval information of the current access service is obtained according to the identification information.
S1084, if it is detected that the relational database already contains the first historical report data of the current access service stored according to the data storage format according to the service retrieval information, storing the current report data into the relational database by replacing the first historical report data with the current report data.
And S1086, if it is detected that the relational database does not contain the first historical reported data of the current access service according to the service retrieval information, storing the current reported data into the relational database according to the data storage format.
In this embodiment, the monitoring system uses a relational database to perform data storage on the reported data of the access service. At this time, the monitoring system sets a data storage format in the relational database according to the unified configuration information of the plurality of access services, and performs data storage management on the reported data of the plurality of access services by using the data storage format. Specifically, after the monitoring system configures service information that needs to be accessed, that is, configures uniform configuration information of a plurality of access services, a storage module in the monitoring system automatically sets a data storage format in a corresponding relational database MySQL. For example, the data storage format is < service internal code, service key search field 1, 2, 3 … … N, service business field 1, 2, 3 … … N >, at this time, a table of < service internal code, service key search field 1, 2, 3 … … N, service business field 1, 2, 3 … … N > may be created in the relational database MySQL, for subsequent storage of the reported data. In addition, the service key search field 1.. n is used as an index table of the relational database MySQL.
Further, when the monitoring system receives the current reported data of the current access service, the service retrieval information of the current access service is obtained according to the identification information in the current reported data, so that whether the first historical reported data of the current access service is contained in the relational database or not is judged according to the service retrieval information. If so, storing the current reporting data into the relational database in a mode of replacing the first historical reporting data with the current reporting data. If not, storing the current reported data into the relational database according to the data storage format.
Specifically, when the data format of the reported data is < service internal code, service key search field 1, service key search field 2.. service key search field N, service field 1, service field 2.. service field N >, the service internal code is extracted from the reported data, and the service internal code is the identification information. And further, positioning the service internal code into a table of a corresponding relational database MySQL. And searching the records in the MySQL table of the relational database according to the query statement with the service key search field 1 and the service key search field 2. And if the corresponding record can be searched, performing updating operation in the relational database MySQL, wherein the updating conditions are a service key retrieval field 1 and a service key retrieval field 2. And if the corresponding record cannot be searched, performing data insertion operation in the relational database MySQL, wherein the insertion conditions are a service key retrieval field 1 and a service key retrieval field 2. The service key retrieval field and the service business field are in one-to-one corresponding relation.
Because the relational database can quickly retrieve the latest data of the corresponding service, the monitoring system adopts the relational database to store the current reported data, and can quickly read the current reported data of the current access service to perform subsequent data monitoring, thereby improving the data management efficiency of the monitoring system.
In one embodiment, as shown in fig. 4, after S1084 or S1086, the method further includes the steps of:
s110, receiving a first display request of the current access service, wherein the first display request comprises identification information of the current access service, and extracting the identification information of the current access service from the first display request.
And S112, acquiring first display configuration information of the current access service according to the identification information.
And S114, acquiring the current reported data in the relational database according to the identification information.
And S116, performing data display on the current reported data according to the first display configuration information.
In this embodiment, after the currently reported data of the currently accessed service is stored in the relational database, the monitoring system needs to display the currently reported data. At this time, the monitoring system receives a first display request of the current access service, the first display request includes identification information of the current access service, first display configuration information is obtained through the identification information of the current access service, and the first display configuration information is used for indicating the monitoring system to perform data display on current reported data according to a specified display form. Meanwhile, the monitoring system acquires the current reported data in the relational database according to the identification information, and then displays the current reported data according to the first display configuration information. Therefore, the current reported data can be displayed in the monitoring system, and the data monitoring of the current access service is realized.
Wherein the first display configuration information may include data display logic configuration information and/or configuration information of a resource on which data depends. The data display logic configuration information may include configuration information whether data is displayed and custom function information for configuring the data display logic. The configuration information of the data-dependent resource may be a resource table on which the configuration service depends. Specifically, the service key search field, the service field, and the like have a method for configuring and displaying, and each field can configure the display mode thereof. For example, the first display configuration information may include whether a field is displayed, whether a custom function is included (in this custom function, there are attribute values of the input field, all data of the row where the input field is located, processing logic, output, where the processing logic may be written by customization), and a resource table of dependencies (some display logic may have a correspondence, such as prop id: 1- > glory handguard). In the specific implementation process, the resource table can be preloaded, and then the resource table can be quickly used in a user-defined function in an interface mode, so that the efficiency can be improved in the preloading process, and the cost for repeatedly preloading the resource table is reduced. Further, the time gap of the update time of the dependency table may also be configured, for example, the time gap of the update time is 5 minutes.
Specifically, in the implementation process of displaying the currently reported data, the service internal code is the identification information of the currently accessed service. And acquiring a corresponding table of service internal codes from a relational database MySQL, traversing all the entries in the corresponding table, and displaying the entries in descending order of reporting time, wherein the display content comprises a service name, a service key search field 1, a service key search field 2 … …, a service key search field N, a service field 1 and a service field 2 … …, and the service field N >. And a display module of the monitoring system performs content rendering display on the current reported data. When rendering the display content, traversing the service key fields 1, 2 and 3 … … N and the service business fields 1, 2 and 3 … … N, and performing display rendering operation. And in the process of rendering, judging whether the current field of the current reported data is displayed or not, and if not, jumping to the next field. If the self-defined function does not exist in the previous field, the field value of the field is directly displayed. If the custom function exists, whether the dependent resource table is configured or not is judged. If configuration exists, the configured URL is accessed, the configured URL is compared with all the memory cache hash tables, if the memory cache hash table of the URL exists, the latest updating time of the memory cache hash table is extracted, if the latest updating time is greater than the updating time difference of the configuration dependent resource table, the cache is refreshed, and the configuration table is really acquired. At this time, whether the content in the URL is in the form of [ { attribute value 1: display value 1}, { attribute value 2: display value 2} … … { attribute value n: display value n } ] is checked, if so, the content is stored in the memory cache hash table, and is stored in the form of key value pair < attribute value, display value >, and the latest update time of the hash table is recorded. If the time gap is not larger than the update time gap of the configuration dependent resource table, the user-defined function logic is read and executed by using the cache, and a series of interfaces for providing help are provided in the user-defined function logic, so that the downloaded resource table can be read. The interface providing the help provides an input that can directly read the relevant contents in the hash table and serve as one of the outputs in the execution process in the custom function logic. Whether the output is directly output as a final result or output after intermediate processing depends on logic written by the user.
And finally, displaying each field of all the line numbers of the currently reported data according to the method, displaying the data in a form of outputting a display table, and recording the generation time generated by the display table as the final loading time. In addition, the auto-refresh time may also be set for the display module of the monitoring system, for example with a default setting of <30 seconds, 1 minute, 5 minutes, 10 minutes >, which the user can select. And if the time difference of the current time minus the loading time of the last displayed form is larger than the set automatic refreshing time, automatically refreshing the data of the displayed form.
In an embodiment, as shown in fig. 5, the step of S104 further includes:
s1034, setting a data storage format in the time sequence database according to the unified configuration information of the plurality of access services.
At this time, S108 includes the steps of:
s1087, determining a target storage table corresponding to the data storage format contained in the time sequence database according to the identification information.
And S1088, acquiring current time information.
S1089, the current reported data is associated with the current time information and then stored in a target storage table according to a data storage format.
In this embodiment, the monitoring system further stores the current reported data by using a time sequence database. It should be noted that the above relational database only stores the current reporting data of the current access service, and the time-series database is used for storing the historical reporting data of the current access service in addition to the current reporting data of the current access service. Specifically, when the monitoring system stores the current reported data of the current access service, the current time information is acquired, and the current reported data is associated with the current time information and then stored in a target storage table of the time-series database according to a data storage format. For example, a corresponding target storage table measurement is created in the chronological database infiluxdb, and the service key search field 1 … … n is set as the search tag. The monitoring system adopts the time sequence database to store the data of the current reported data of the current access service, and can realize the operations of rapidly storing, grouping, aggregating and the like on a large amount of data according to the dimension of time. In this case, the time-series database stores a large amount of time data, and a quick search using the key search field can be realized. In the process of generating the chart and tracing the past history, the time sequence type database can be used as a data source of data.
Preferably, S1089 is followed by: and determining second historical reported data of the current access service within a preset time period from the current time in the target storage table, and deleting the historical data of the current access service except the second historical reported data in the target storage table.
In this embodiment, the monitoring system clears the historical data in the target storage table in the time-series database, and deletes the historical data other than the second historical report data of the current access service within a preset time period from the current time, thereby ensuring the subsequent query efficiency of the historical data in the target storage table. For example, the corresponding data before 7 days in the target storage table is emptied, so that the data can be automatically expired, the query efficiency is ensured, and the negative influence of the scale of the data on the query performance, the insertion performance and the storage space is avoided.
In an embodiment, after S1089, the method further includes: receiving a second display request of the current access service, wherein the second display request comprises identification information of the current access service, and extracting the identification information of the current access service from the second display request; acquiring second display configuration information of the current access service according to the identification information; acquiring third history report data of the current access service stored in a target storage table according to the identification information; and displaying the data of the third history reported data according to the second display configuration information.
In this embodiment, the monitoring system receives a second display request of the current access service, where the second display request includes identification information of the current access service, acquires second display configuration information of the current access service according to the identification information, and acquires third history report data of the current access service stored in the target storage table according to the identification information. The identification information may be a service internal code corresponding to the current access service set by the monitoring system. And finally, the monitoring system displays the data of the third history reported data according to the second display configuration information. The second display configuration information may include configuration information of a display chart for data display, display data column formatting configuration information of data, and/or type configuration information of a rendering display chart. Specifically, a user may perform detailed display configuration on historical time series data of a current access service in a display module of the monitoring system, the configuration interface may display various service configurations for the user to refer to, and then perform detailed display configuration, where the detailed display configuration may include configuration of multiple graphs, and the configuration of one graph includes, but is not limited to, the following parameters: query SQL (which can correspondingly search all data), display data column formatting configuration (which can configure the data format of display data), rendering chart type, and rendering chart type setting >.
And after the configuration is finished, rendering is carried out on the display module. Specifically, the display module queries data and preprocesses the data, and the data preprocessing process is as follows:
1. reading the query SQL configured by the chart, and performing SQL query on the time sequence database influxdb.
2. Such as SQL query errors, return error reports.
3. And inquiring a corresponding SQL inquiry result, dividing the inquiry result into n rows, wherein each row has m fields, traversing the m fields, and formatting the display data column, wherein the formatting and the rendering process are the same.
4. The graph display processing is performed on the preprocessed data, the types of the rendering graphs are various, the strategy of each rendering graph is different, and the following description is only given by taking a line graph as an example:
4.1 for a line graph, "render chart type settings" includes but not only includes the following configurations: the line graph display width, the line graph display height, the pixel displayed by each point, the value warning line numerical value, the line graph color and the displayed value field.
4.2 dividing the width of the line graph display by the pixel displayed for each dot in the above configuration gives the number of dots displayed a.
4.3, sorting the data according to the time value from small to large, and dividing the number n of the data items by the number a of the points to obtain the sampling distance b of the points needing to be sampled. Dividing the line graph display width by the pixel displayed by each point to obtain the distance of the space of each point. A maximum value max and a minimum value min of a field of a display value of data.
And 4.4, continuously traversing the corresponding data in the number n of the data entries one by one, and rendering.
4.4.1 record the current rendering position abscissa x, ordinate y and the current number of points q.
4.4.2 if the current point q is a boundary, that is, 0 or n, rendering the current point, otherwise, advancing the sampling point sampling distance b, (q + b) as a new q to reach the target data entry q, drawing the displayed value field of the entry, and calculating the new position of drawing as x + the pitch distance of each point on the abscissa and y as the height multiplied by the line graph display height (the field value of the current display value divided by (maximum-minimum)).
4.5 connecting all the points in the step 4.4, wherein the color of the line is the color of the broken line chart in the step 4.1.
4.6 for the value fence value, determine the ordinate y ═ line graph display height multiplied by (value fence value divided by (maximum-minimum)), then draw a straight line on this ordinate, and draw a fence to indicate the fence value.
5. And (4) sequentially displaying all the charts according to the data in the chart setting according to the step of the step 4.
Therefore, the monitoring system can realize data display of the third history reported data of the current access service stored in the target storage table of the time sequence database, thereby being beneficial to data monitoring of the data of the current access service.
In an embodiment, S104 further includes: reading unified configuration information of a plurality of access services, wherein the unified configuration information comprises service identification information, service retrieval information and service business information; and setting a data storage format according to the identification information, the service retrieval information and the service business information of the service.
In this embodiment, the monitoring system sets unified configuration information for the plurality of access services, and the unified configuration information may be manually configured and then input into the monitoring system, and then stored after being identified by the monitoring system. When the unified configuration information contains the identification information, the service retrieval information and the service business information of the service, the data storage format set by the monitoring system contains the identification information, the service retrieval information and the data format setting corresponding to the service business information of the service. For example, the identification information of the service may be a service name, a service internal code, or service description information, etc., the service retrieval information may be a service key retrieval field, and the service business information may be a service business field. At this time, the unified configuration information includes, but is not limited to: service name, service internal code, service description, service key search field (for distinguishing the differences between services, all key search fields must be able to locate a unique service, and also be used for subsequent search optimization and warehousing), and service business field (for collecting the difference indexes between services, which are not usually used for determining unique services, but only for viewing and range search). At this time, the unified configuration information may further include a data reporting mode of the service (the unified configuration information is divided into an active reporting mode and a passive reporting mode, the active reporting mode needs to send an information string to a URL specified by the device receiving module, and the passive reporting mode needs to provide an interface data open to the outside for the device scanning module to perform active acquisition). The monitoring system can set a data storage format in the storage module according to the unified configuration information, and the data storage format is suitable for the reported data of any access service in the plurality of access services, so that the monitoring system can realize the unified management of the reported data of the plurality of access services.
In one embodiment, as shown in fig. 6, S102 includes the steps of:
s1022, the data reporting configuration information of the current access service is read from the unified configuration information of the multiple access services.
And S1024, if the data reporting mode of the current access service is determined to be the active reporting mode according to the data reporting configuration information, receiving the current reporting data of the current access service when the data reporting request of the current access service is determined to be received.
And S1026, if the data reporting mode of the current access service is determined to be the passive reporting mode according to the data reporting configuration information, acquiring interface information of the data providing equipment corresponding to the current access service, and acquiring the current reporting data of the current access service from the data providing equipment according to the interface information so as to receive the current reporting data of the current access service.
In this embodiment, the unified configuration information of the multiple access services set by the monitoring system includes a data reporting mode of the access service. The data reporting mode comprises an active reporting mode and a passive reporting mode. The active reporting mode is that the service providing equipment actively uploads the reported data to the monitoring system, and the monitoring system only needs to receive the reported data of the service providing equipment. The passive reporting mode is the collection of reported data in the service providing equipment by the monitoring system.
If the mode is the active reporting mode, the logic for realizing the active data reporting in the service is as follows:
1. the service providing device refers to the reporting module in the service and configures the reporting module, and the configuration attributes include but are not limited to: the service internal code, the value of the service key retrieval field.
2. And configuring the frequency of the reported data in the reporting module, and writing the logic of acquiring the data at regular time while configuring the frequency of the reported data. The logic of acquiring data regularly is mainly to acquire data in service field, the program needs to realize the logic of acquiring data by itself, and uses the interface provided in the reporting module to write the acquired data into the fields in turn according to the service field name, and then the program adopts the reporting logic to automatically report the reported data to the monitoring system.
3. And (3) starting the service, automatically starting the service providing equipment after the report of the service providing equipment is quoted, and checking whether the parameters are qualified: the method comprises the steps of judging whether the parameters are configured parameters, judging whether the parameters written by the logic for acquiring data are configured parameters, and judging whether the universal parameter values are in a legal configuration range.
4. The reporting module of the service providing equipment automatically executes the logic of acquiring data according to the data reporting frequency, and arranges the logic into data in the forms of < service internal code, service key retrieval field 1, service key retrieval field 2.. service key retrieval field n, service business field 1, service business field 2.. service business field n >.
Therefore, the service providing equipment can report the reported data to the monitoring system. For the monitoring system, when it is monitored that the service providing device reports data, it is enough to receive currently reported data of the currently accessed service sent by the service providing device.
If the mode is the passive reporting mode, the logic for realizing the passive data reporting in the service is as follows:
1. the service providing device refers to the exposed interface module provided by the monitoring system in the service and configures the exposed interface module, and the configuration attributes include but are not limited to: the service internal code, the value of the service key retrieval field.
2. The logic to retrieve the data is written in an exposed interface module of the service providing device. The logic for acquiring data is mainly to acquire data in service field, the program needs to realize the logic for acquiring data by itself, and uses the interface provided in the report module of the service providing device to write the acquired data into the fields in sequence according to the service field names, and then the data is automatically assembled by the exposed interface module.
3. The exposed interface module will automatically arrange the data into the form of < service internal code, service key search field 1, service key search field 2 … … service key search field N, service traffic field 1, service traffic field 2 … … service traffic field N >.
4. The service providing equipment starts the service itself, the exposed interface service will start accordingly, and according to the assembly logic, automatically add the exposed interface with the address of the specific URL, and report the IP, port and URL information of the service to the data receiving module of the monitoring system. Therefore, the data receiving module of the monitoring system can automatically acquire data of the service providing equipment according to the IP, the port and the URL information.
Therefore, the monitoring system can acquire the reported data of the service providing equipment.
It should be noted that, after the data in the active reporting mode is sorted, the service providing device automatically compresses the data through the gzip algorithm and then uploads the data to the data receiving module of the monitoring system. And in the passive scanning mode, the monitoring system automatically scans the corresponding exposed interface address information to acquire reported data, and stores the reported data in the data receiving module. Therefore, the monitoring system can realize various data reporting modes and can meet the requirements of practical application.
For the unified configuration information of the monitoring system for multiple access services, some specific implementation examples are given below to better illustrate the unified configuration information in the present application:
1. service configuration:
service name: and (5) monitoring the process.
Service internal code: PROCESS _ MONITOR.
Service key search field: gmweb.monitor.game gmweb.monitor.platgmweb.monitor.server gmweb.monitor.type.
Service field: not to be exhaustive.
Service data reporting mode: and actively reporting.
When the current access service is process monitoring, the current data display of the current access service is as shown in fig. 7, and the historical data display of the current access service is as shown in fig. 8.
2. Service configuration:
service name: and monitoring an operating system.
Service internal code: OS _ MONITOR.
Service key search field: gmweb.monitor.game gmweb.monitor.platgmweb.monitor.server gmweb.monitor.type.
Service field: not to be exhaustive.
Service data reporting mode: and actively reporting.
According to the processing method of the reported data, the reported data are processed simultaneously by adopting the relational database and the time sequence database, and the common key fields of the services are extracted, so that the services can be monitored, the latest state and the historical state of the services can be quickly checked, and the problem of quick positioning of the data can be solved. In addition, each access service monitoring system can be processed uniformly, the reported data of the access service is subjected to data structure unification, the individuality in the commonality is realized, and a data source and a data display template can be configured in a customized manner, so that a data display interface is generated, and the data can be conveniently and uniformly viewed. In addition, for the monitoring system, all the access services can be monitored uniformly, and a user does not need to monitor and process each service independently or monitor service maintenance, so that later modification and maintenance are more convenient.
It should be understood that, although the steps in the flowchart are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.
As shown in fig. 9, the apparatus includes a receiving module 10, a determining module 20, an extracting module 30, and a storing module 40. A receiving module 10, configured to receive currently reported data of a current access service, where the current access service is any one of multiple access services accessed to a monitoring system, and the currently reported data is configured in a data format according to unified configuration information of the multiple access services and includes identification information of the current access service; a determining module 20, configured to determine a pre-configured data storage format, where the data storage format is set according to unified configuration information of multiple access services; an extracting module 30, configured to extract identification information of a current access service from current reported data; and the storage module 40 is configured to perform data storage on the currently reported data according to the identification information and the data storage format.
In an embodiment, the processing apparatus for reporting data further includes a first setting module (not shown in fig. 9) configured to set a data storage format in the relational database according to the unified configuration information of the multiple access services; at this time, the storage module 40 is specifically configured to obtain service retrieval information of the current access service according to the identification information, store the current reporting data in the relational database in a manner of replacing the first historical reporting data with the current reporting data if it is detected that the first historical reporting data of the current access service, which is stored according to the data storage format, is already included in the relational database according to the service retrieval information, and store the current reporting data in the relational database according to the data storage format if it is detected that the first historical reporting data of the current access service is not included in the relational database according to the service retrieval information.
In an embodiment, the processing apparatus for processing the reported data further includes a first display module (not shown in fig. 9) configured to obtain identification information of the current access service, obtain first display configuration information of the current access service according to the identification information, obtain the current reported data in the relational database according to the identification information, and perform data display on the current reported data according to the first display configuration information.
In an embodiment, the processing apparatus for reporting data further includes a second setting module (not shown in fig. 9) configured to set a data storage format in the time-series database according to the unified configuration information of the multiple access services. At this time, the storage module 40 is specifically configured to determine, according to the identification information, a target storage table corresponding to a data storage format included in the time sequence database, acquire current time information, associate current reported data with the current time information, and store the associated current reported data in the target storage table according to the data storage format.
In an embodiment, the processing apparatus for reporting data further includes a deleting module (not shown in fig. 9) configured to determine second historical reporting data of the current access service within a preset time period from the current time in the target storage table, and delete the historical data of the current access service except the second historical reporting data in the target storage table.
In an embodiment, the processing apparatus for reporting data further includes a second display module (not shown in fig. 9) configured to obtain identification information of the current access service, obtain second display configuration information of the current access service according to the identification information, obtain third history reporting data of the current access service stored in the target storage table according to the identification information, and perform data display on the third history reporting data according to the second display configuration information.
In an embodiment, the processing apparatus for reporting data further includes a third setting module (not shown in fig. 9), configured to read unified configuration information of the multiple access services, where the unified configuration information includes identification information of the services, service retrieval information, and service information, and set a data storage format according to the identification information of the services, the service retrieval information, and the service information.
In an embodiment, the receiving module 10 is specifically configured to read data reporting configuration information of a current access service from unified configuration information of multiple access services, if it is determined that a data reporting mode of the current access service is an active reporting mode according to the data reporting configuration information, receive current reported data of the current access service when it is determined that a data reporting request of the current access service is received, if it is determined that the data reporting mode of the current access service is a passive reporting mode according to the data reporting configuration information, obtain interface information of a data providing device corresponding to the current access service, and collect the current reported data of the current access service from the data providing device according to the interface information, so as to receive the current reported data of the current access service.
For specific limitations of the processing device for reporting data, reference may be made to the above limitations on the processing method for reporting data, which are not described herein again. All or part of each module in the processing device for reporting data can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 10. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for connecting with an external device so as to perform data interaction with the external device. The computer program is executed by a processor to implement a method for processing reported data.
Those skilled in the art will appreciate that the architecture shown in fig. 10 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program: receiving current reported data of a current access service, wherein the current access service is any one of a plurality of access services accessed to a monitoring system, the current reported data is configured in a data format according to unified configuration information of the plurality of access services, and the current reported data contains identification information of the current access service; determining a pre-configured data storage format, wherein the data storage format is set according to the unified configuration information of a plurality of access services; extracting identification information of the current access service from the current reported data; and performing data storage on the current reported data according to the identification information and the data storage format.
In one embodiment, the processor, when executing the computer program, performs the steps of: and setting the data storage format in a relational database according to the unified configuration information of the plurality of access services. When the processor executes the computer program to realize the step of storing the data of the currently reported data according to the identification information and the data storage format, the following steps are specifically realized: acquiring service retrieval information of the current access service according to the identification information; if the relation type database is detected to contain first historical report data of the current access service stored according to the data storage format according to the service retrieval information, storing the current report data into the relation type database in a mode of replacing the first historical report data with the current report data; and if the relational database does not contain the first historical reported data of the current access service according to the service retrieval information, storing the current reported data into the relational database according to the data storage format.
In one embodiment, the processor, when executing the computer program, performs the steps of: acquiring identification information of a current access service; acquiring first display configuration information of the current access service according to the identification information; acquiring current reported data in the relational database according to the identification information; and performing data display on the current reported data according to the first display configuration information.
In one embodiment, the processor, when executing the computer program, performs the steps of: and setting a data storage format in a time sequence database according to the unified configuration information of the plurality of access services. When the processor executes the computer program to realize the step of storing the data of the currently reported data according to the identification information and the data storage format, the following steps are specifically realized: determining a target storage table corresponding to a data storage format contained in the time sequence database according to the identification information; acquiring current time information; and associating the current reported data with the current time information and then storing the current reported data and the current time information into a target storage table according to a data storage format.
In one embodiment, the processor, when executing the computer program, performs the steps of: determining second historical reported data of the current access service in a preset time period from the current time in the target storage table; and deleting the historical data of the current access service except the second historical reported data in the target storage table.
In one embodiment, the processor, when executing the computer program, performs the steps of: acquiring identification information of a current access service; acquiring second display configuration information of the current access service according to the identification information; acquiring third history report data of the current access service stored in a target storage table according to the identification information; and displaying the data of the third history reported data according to the second display configuration information.
In one embodiment, the processor, when executing the computer program, performs the steps of: reading unified configuration information of a plurality of access services, wherein the unified configuration information comprises service identification information, service retrieval information and service business information; and setting a data storage format according to the identification information, the service retrieval information and the service business information of the service.
In one embodiment, when the processor executes the computer program to implement the step of receiving the currently reported data of the current access service, the following steps are specifically implemented: reading data reporting configuration information of the current access service from the unified configuration information of the plurality of access services; if the data reporting mode of the current access service is determined to be the active reporting mode according to the data reporting configuration information, receiving the current reporting data of the current access service when the data reporting request of the current access service is determined to be received; and if the data reporting mode of the current access service is determined to be the passive reporting mode according to the data reporting configuration information, acquiring interface information of the data providing equipment corresponding to the current access service, and acquiring the current reporting data of the current access service from the data providing equipment according to the interface information so as to receive the current reporting data of the current access service.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of: receiving current reported data of a current access service, wherein the current access service is any one of a plurality of access services accessed to a monitoring system, the current reported data is configured in a data format according to unified configuration information of the plurality of access services, and the current reported data contains identification information of the current access service; determining a pre-configured data storage format, wherein the data storage format is set according to the unified configuration information of a plurality of access services; extracting identification information of the current access service from the current reported data; and performing data storage on the current reported data according to the identification information and the data storage format.
In one embodiment, the computer program when executed by the processor performs the steps of: and setting the data storage format in a relational database according to the unified configuration information of the plurality of access services. When the computer program is executed by the processor to realize the step of storing the data of the current reported data according to the identification information and the data storage format, the following steps are specifically realized: acquiring service retrieval information of the current access service according to the identification information; if the relation type database is detected to contain first historical report data of the current access service stored according to the data storage format according to the service retrieval information, storing the current report data into the relation type database in a mode of replacing the first historical report data with the current report data; and if the relational database does not contain the first historical reported data of the current access service according to the service retrieval information, storing the current reported data into the relational database according to the data storage format.
In one embodiment, the computer program when executed by the processor performs the steps of: acquiring identification information of a current access service; acquiring first display configuration information of the current access service according to the identification information; acquiring current reported data in the relational database according to the identification information; and performing data display on the current reported data according to the first display configuration information.
In one embodiment, the computer program when executed by the processor performs the steps of: and setting a data storage format in a time sequence database according to the unified configuration information of the plurality of access services. When the computer program is executed by the processor to realize the step of storing the data of the current reported data according to the identification information and the data storage format, the following steps are specifically realized: determining a target storage table corresponding to a data storage format contained in the time sequence database according to the identification information; acquiring current time information; and associating the current reported data with the current time information and then storing the current reported data and the current time information into a target storage table according to a data storage format.
In one embodiment, the computer program when executed by the processor performs the steps of: determining second historical reported data of the current access service in a preset time period from the current time in the target storage table; and deleting the historical data of the current access service except the second historical reported data in the target storage table.
In one embodiment, the computer program when executed by the processor performs the steps of: acquiring identification information of a current access service; acquiring second display configuration information of the current access service according to the identification information; acquiring third history report data of the current access service stored in a target storage table according to the identification information; and displaying the data of the third history reported data according to the second display configuration information.
In one embodiment, the computer program when executed by the processor performs the steps of: reading unified configuration information of a plurality of access services, wherein the unified configuration information comprises service identification information, service retrieval information and service business information; and setting a data storage format according to the identification information, the service retrieval information and the service business information of the service.
In one embodiment, when the processor executes the step of receiving the current reported data of the current access service, the following steps are specifically implemented: reading data reporting configuration information of the current access service from the unified configuration information of the plurality of access services; if the data reporting mode of the current access service is determined to be the active reporting mode according to the data reporting configuration information, receiving the current reporting data of the current access service when the data reporting request of the current access service is determined to be received; and if the data reporting mode of the current access service is determined to be the passive reporting mode according to the data reporting configuration information, acquiring interface information of the data providing equipment corresponding to the current access service, and acquiring the current reporting data of the current access service from the data providing equipment according to the interface information so as to receive the current reporting data of the current access service.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (10)

1. A processing method of reported data, the method comprising:
receiving current reported data of a current access service, wherein the current access service is any one of a plurality of access services accessed to a monitoring system, the current reported data is configured in a data format according to unified configuration information of the plurality of access services, and the current reported data contains identification information of the current access service;
determining a pre-configured data storage format, wherein the data storage format is set according to the unified configuration information of the plurality of access services;
extracting the identification information of the current access service from the current reported data;
and performing data storage on the current reported data according to the identification information and the data storage format.
2. The method of claim 1, further comprising: setting the data storage format in a relational database according to the unified configuration information of the plurality of access services;
the data storage of the current reported data according to the identification information and the data storage format includes:
acquiring service retrieval information of the current access service according to the identification information;
if it is detected that the relational database already contains first historical report data of the current access service, which is stored according to the data storage format, according to the service retrieval information, storing the current report data into the relational database in a mode of replacing the first historical report data with the current report data;
and if the relational database does not contain the first historical reported data of the current access service according to the service retrieval information, storing the current reported data into the relational database according to the data storage format.
3. The method of claim 2, further comprising:
acquiring the identification information of the current access service;
acquiring first display configuration information of the current access service according to the identification information;
acquiring the current reported data in the relational database according to the identification information;
and performing data display on the current reported data according to the first display configuration information.
4. The method of claim 1, further comprising: setting the data storage format in a time sequence database according to the unified configuration information of the plurality of access services;
the data storage of the current reported data according to the identification information and the data storage format includes:
determining a target storage table corresponding to the data storage format contained in the time sequence database according to the identification information;
acquiring current time information;
associating the current reported data with the current time information and then storing the current reported data and the current time information into the target storage table according to the data storage format;
preferably, the method further comprises:
determining second historical reported data of the current access service in a preset time period away from the current time in the target storage table;
and deleting the historical data of the current access service except the second historical reported data in the target storage table.
5. The method of claim 4, further comprising:
acquiring the identification information of the current access service;
acquiring second display configuration information of the current access service according to the identification information;
acquiring third history report data of the current access service stored in the target storage table according to the identification information;
and performing data display on the third history reported data according to the second display configuration information.
6. The method of claim 1, further comprising:
reading unified configuration information of the plurality of access services, wherein the unified configuration information comprises service identification information, service retrieval information and service business information;
and setting the data storage format according to the identification information, the service retrieval information and the service business information of the service.
7. The method of claim 1, wherein receiving currently reported data for a current access service comprises:
reading the data reporting configuration information of the current access service from the unified configuration information of the plurality of access services;
if the data reporting mode of the current access service is determined to be the active reporting mode according to the data reporting configuration information, receiving the current reporting data of the current access service when the data reporting request of the current access service is determined to be received;
if the data reporting mode of the current access service is determined to be the passive reporting mode according to the data reporting configuration information, acquiring interface information of data providing equipment corresponding to the current access service, and acquiring current reporting data of the current access service from the data providing equipment according to the interface information so as to receive the current reporting data of the current access service.
8. A processing apparatus for reporting data, the apparatus comprising:
a receiving module, configured to receive currently reported data of a current access service, where the current access service is any one of multiple access services accessed to a monitoring system, and the currently reported data performs data format configuration according to unified configuration information of the multiple access services and includes identification information of the current access service;
the determining module is used for determining a pre-configured data storage format, and the data storage format is set according to the unified configuration information of the plurality of access services;
an extraction module, configured to extract the identifier information of the current access service from the current reported data;
and the storage module is used for performing data storage on the current reported data according to the identification information and the data storage format.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of any of claims 1 to 7 are implemented when the computer program is executed by the processor.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
CN202010209894.0A 2020-03-23 2020-03-23 Processing method and device for reported data, computer equipment and storage medium Active CN111385365B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010209894.0A CN111385365B (en) 2020-03-23 2020-03-23 Processing method and device for reported data, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010209894.0A CN111385365B (en) 2020-03-23 2020-03-23 Processing method and device for reported data, computer equipment and storage medium

Publications (2)

Publication Number Publication Date
CN111385365A true CN111385365A (en) 2020-07-07
CN111385365B CN111385365B (en) 2022-09-30

Family

ID=71218778

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010209894.0A Active CN111385365B (en) 2020-03-23 2020-03-23 Processing method and device for reported data, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN111385365B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112600924A (en) * 2020-12-15 2021-04-02 四川长虹电器股份有限公司 IoT terminal data storage system, method, computer device and storage medium
CN113656095A (en) * 2021-08-06 2021-11-16 北京数码大方科技股份有限公司 Method and device for processing configuration data
CN113806307A (en) * 2021-08-09 2021-12-17 阿里巴巴(中国)有限公司 Data processing method and device
CN114531493A (en) * 2020-11-23 2022-05-24 北京达佳互联信息技术有限公司 Request processing method and device, electronic equipment and storage medium
CN117097599A (en) * 2023-10-20 2023-11-21 深圳高灯计算机科技有限公司 Network service detection method, device, computer equipment and storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106101169A (en) * 2016-05-19 2016-11-09 努比亚技术有限公司 A kind of terminal and data reporting method
CN106790380A (en) * 2016-11-21 2017-05-31 广州华多网络科技有限公司 Data reporting method and device
US20180373766A1 (en) * 2015-10-28 2018-12-27 Fractal Industries, Inc. Automated scalable contextual data collection and extraction system
CN109325030A (en) * 2018-09-03 2019-02-12 平安国际融资租赁有限公司 Message processing method, device, computer equipment and storage medium
CN109344183A (en) * 2018-01-30 2019-02-15 深圳壹账通智能科技有限公司 Data interactive method, device, computer equipment and storage medium
CN109618331A (en) * 2019-02-01 2019-04-12 深圳乐信软件技术有限公司 Data reporting method, device, equipment and storage medium
CN110334140A (en) * 2019-05-24 2019-10-15 深圳绿米联创科技有限公司 The method, apparatus and server of processing equipment reported data

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180373766A1 (en) * 2015-10-28 2018-12-27 Fractal Industries, Inc. Automated scalable contextual data collection and extraction system
CN106101169A (en) * 2016-05-19 2016-11-09 努比亚技术有限公司 A kind of terminal and data reporting method
CN106790380A (en) * 2016-11-21 2017-05-31 广州华多网络科技有限公司 Data reporting method and device
CN109344183A (en) * 2018-01-30 2019-02-15 深圳壹账通智能科技有限公司 Data interactive method, device, computer equipment and storage medium
CN109325030A (en) * 2018-09-03 2019-02-12 平安国际融资租赁有限公司 Message processing method, device, computer equipment and storage medium
CN109618331A (en) * 2019-02-01 2019-04-12 深圳乐信软件技术有限公司 Data reporting method, device, equipment and storage medium
CN110334140A (en) * 2019-05-24 2019-10-15 深圳绿米联创科技有限公司 The method, apparatus and server of processing equipment reported data

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
JA HOLLENBACH等: ""Toward a community standard for immunogenetics data reporting and analysis"", 《EUROPEAN IMMUNOGENETICS & HISTOCOMPATIBILITY CONFERENCE》 *
卢中阳: ""基于Android手机数据上报系统的设计与实现"", 《中国硕士学位论文全文数据库信息科技辑》 *
欧阳海波: ""基于ASP.NET MVC B/S架构的任务管理与数据上报系统设计与实现"", 《中国硕士学位论文全文数据库信息科技辑》 *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114531493A (en) * 2020-11-23 2022-05-24 北京达佳互联信息技术有限公司 Request processing method and device, electronic equipment and storage medium
CN114531493B (en) * 2020-11-23 2024-03-26 北京达佳互联信息技术有限公司 Request processing method and device, electronic equipment and storage medium
CN112600924A (en) * 2020-12-15 2021-04-02 四川长虹电器股份有限公司 IoT terminal data storage system, method, computer device and storage medium
CN113656095A (en) * 2021-08-06 2021-11-16 北京数码大方科技股份有限公司 Method and device for processing configuration data
CN113656095B (en) * 2021-08-06 2023-08-04 北京数码大方科技股份有限公司 Processing method and device for configuration data
CN113806307A (en) * 2021-08-09 2021-12-17 阿里巴巴(中国)有限公司 Data processing method and device
CN117097599A (en) * 2023-10-20 2023-11-21 深圳高灯计算机科技有限公司 Network service detection method, device, computer equipment and storage medium
CN117097599B (en) * 2023-10-20 2024-01-16 深圳高灯计算机科技有限公司 Network service detection method, device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN111385365B (en) 2022-09-30

Similar Documents

Publication Publication Date Title
CN111385365B (en) Processing method and device for reported data, computer equipment and storage medium
CN111563368A (en) Report generation method and device, computer equipment and storage medium
WO2017107853A1 (en) Data monitoring management method, and data monitoring method and system
US9842134B2 (en) Data query interface system in an event historian
CN103309928B (en) The method and system of storage and retrieve data
CN112527783A (en) Data quality probing system based on Hadoop
CN111506559A (en) Data storage method and device, electronic equipment and storage medium
CN112286934A (en) Database table importing method, device, equipment and medium
CN112256318B (en) Construction method and equipment for dependent product
CN112613271A (en) Data paging method and device, computer equipment and storage medium
CN111061802B (en) Power data management processing method, device and storage medium
CN112905706A (en) Database synchronization method and device, storage medium and computer equipment
CN111400578B (en) Cargo data query method, cargo data query device, computer equipment and storage medium
CN115357590A (en) Recording method and device for data change, electronic device and storage medium
US20130204913A1 (en) File list generation method, system, and program, and file list generation device
CN107330031B (en) Data storage method and device and electronic equipment
CN111984659B (en) Data updating method, device, computer equipment and storage medium
CN113377789A (en) Processing method and device for database change data, computer equipment and medium
CN112948504A (en) Data acquisition method and device, computer equipment and storage medium
CN115794839B (en) Data collection method based on Php+Mysql system, computer equipment and storage medium
CN115544007A (en) Label preprocessing method and device, computer equipment and storage medium
CN113472881B (en) Statistical method and device for online terminal equipment
CN112163015B (en) Real-time monitoring method, device and system for time sequence data of Internet of things
CN113259154B (en) Method and device for informing middle station data verification, computer equipment and storage medium
CN113778996A (en) Large data stream data processing method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant