CN117312088A - Data monitoring method and device, electronic equipment and storage medium - Google Patents

Data monitoring method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN117312088A
CN117312088A CN202311335785.3A CN202311335785A CN117312088A CN 117312088 A CN117312088 A CN 117312088A CN 202311335785 A CN202311335785 A CN 202311335785A CN 117312088 A CN117312088 A CN 117312088A
Authority
CN
China
Prior art keywords
data
validity period
configuration information
service
service data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311335785.3A
Other languages
Chinese (zh)
Inventor
林培晖
陈智凯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing QIYI Century Science and Technology Co Ltd
Original Assignee
Beijing QIYI Century Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing QIYI Century Science and Technology Co Ltd filed Critical Beijing QIYI Century Science and Technology Co Ltd
Priority to CN202311335785.3A priority Critical patent/CN117312088A/en
Publication of CN117312088A publication Critical patent/CN117312088A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/302—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
    • G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Quality & Reliability (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the invention provides a data monitoring method, a device, electronic equipment and a storage medium, which are applied to the technical field of data processing. The data monitoring method comprises the steps of obtaining a plurality of configuration information; each configuration information comprises data validity period information corresponding to service data in a storage position; different storage locations store business data in different storage modes; judging whether the service data in the storage position accords with the data validity period information or not according to each piece of configuration information; if yes, triggering the data expiration early warning. By applying the data monitoring method, the device, the electronic equipment and the storage medium provided by the embodiment of the invention, the flexibility of data monitoring can be improved.

Description

Data monitoring method and device, electronic equipment and storage medium
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a data monitoring method, a data monitoring device, an electronic device, and a storage medium.
Background
In most transaction scenarios, it is necessary to monitor the timeliness of data, for example, information about goods, prices, payment methods, sales promotion methods, etc. related to transactions. However, the information is typically stored in different storage modes and even distributed among different storage locations, so that the logic code used to monitor the timeliness of the information is also different.
The information to be monitored is different for different service demands, and therefore, the information stored according to different storage modes needs to be monitored according to different service demands, namely, different logic codes for monitoring the timeliness of the information are used, and after the corresponding logic codes are modified for each time of changing the service demands, the corresponding logic codes need to be redeployed in a service program to enable the modification to be effective, so that the flexibility of data monitoring is lacked.
Disclosure of Invention
The embodiment of the invention aims to provide a data monitoring method, a device, electronic equipment and a storage medium, so as to improve the flexibility of data monitoring. The specific technical scheme is as follows:
in a first aspect of the present invention, there is provided a data monitoring method, the method comprising:
acquiring a plurality of configuration information; each configuration information comprises data validity period information corresponding to service data in a storage position; different storage locations store business data in different storage modes;
judging whether the service data in the storage position accords with the data validity period information or not according to each piece of configuration information;
if yes, triggering the data expiration early warning.
Optionally, the configuration information includes a service type, a database address, a table name, and a data validity period field; the different storage modes comprise different database addresses and/or different table names;
The judging whether the service data in the storage position accords with the data validity period information comprises the following steps:
and judging whether the service data stored in the table name of the database address accords with the data validity period information indicated by the data validity period field.
Optionally, the data validity period field includes: the data validity period time and the early warning time corresponding to the service data;
the judging whether the service data in the storage position accords with the data validity period information comprises the following steps:
judging whether the sum of the current time and the early warning time is larger than the data validity period time.
Optionally, the data validity period field includes a data validity period time corresponding to the service data;
the judging whether the service data in the storage position accords with the data validity period information comprises the following steps:
and judging whether the difference value between the data validity period time and the current time is smaller than a preset time difference value or not.
Optionally, after the acquiring the plurality of configuration information, the method further includes:
obtaining a query statement corresponding to the configuration information based on the service type, the database address, the table name and the data validity period field included in the configuration information;
The judging whether the service data stored in the table name of the database address accords with the data validity period information indicated by the data validity period field comprises the following steps:
and acquiring service data stored in the table name of the database address by utilizing the query statement, and judging whether the service data accords with the data validity period information indicated by the data validity period field.
Optionally, if yes, triggering data validity early warning, including:
and sending early warning information to a service terminal under the condition that the service data in the storage position accords with the data validity period information.
In a second aspect of the present invention, there is also provided a data monitoring apparatus, the apparatus comprising:
the acquisition module is used for acquiring a plurality of configuration information; each configuration information comprises data validity period information corresponding to service data in a storage position; different storage locations store business data in different storage modes;
the judging module is used for judging whether the service data in the storage position accords with the data validity period information or not according to each piece of configuration information;
and the early warning model is used for triggering the early warning of the data validity period if yes.
Optionally, the configuration information includes a service type, a database address, a table name, and a data validity period field; the different storage modes comprise different database addresses and/or different table names;
the judging module is specifically configured to judge whether the service data stored in the table name of the database address accords with the data validity period information indicated by the data validity period field.
Optionally, the data validity period field includes a data validity period time corresponding to the service data and an early warning time;
the judging module is specifically configured to judge whether the sum of the current time and the early warning time is greater than the data validity period time.
Optionally, the data validity period field includes a data validity period time corresponding to the service data;
the judging module is specifically configured to judge whether a difference between the data validity period time and the current time is smaller than a preset time difference.
Optionally, the apparatus further comprises,
the query statement module is used for obtaining a query statement corresponding to the configuration information based on the service type, the database address, the table name and the data validity period field included in the configuration information after the plurality of configuration information are obtained;
The judging module is specifically configured to obtain, by using the query statement, service data stored in the table name of the database address, and judge whether the service data accords with the data validity period information indicated by the data validity period field.
Optionally, the early warning module is specifically configured to send early warning information to a service end when the service data in the storage location accords with the data validity period information.
In a third aspect of the present invention, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;
the memory is used for storing a computer program;
the processor is configured to implement the data monitoring method according to the first aspect when executing the program stored in the memory.
In a fourth aspect of the present invention, there is also provided a computer readable storage medium having a computer program stored therein, which when executed by a processor implements the data monitoring method according to the first aspect.
In a further aspect of the invention there is also provided a computer program product comprising instructions which, when run on a computer, cause the computer to perform the data monitoring method of the first aspect described above.
The data monitoring method provided by the embodiment of the invention acquires a plurality of configuration information; each configuration information comprises data validity period information corresponding to service data in a storage position; different storage locations store business data in different storage modes; judging whether the service data in the storage position accords with the data validity period information or not according to each piece of configuration information; and triggering data validity early warning under the condition that the service data in the storage position accords with the data validity period information.
And managing the data validity period information of the service data stored in different storage modes by using the configuration information, and when the data is required to be monitored, uniformly monitoring the service data stored in different storage modes by acquiring a plurality of configuration information at one time. The service data stored in different storage modes can be monitored only by acquiring different configuration information, and meanwhile, the service program is not required to be redeployed by acquiring different configuration information, so that the flexibility of data monitoring can be improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
FIG. 1 is a flowchart of a data monitoring method according to an embodiment of the present invention;
FIG. 2 is a diagram of an example of expiration data provided by an embodiment of the present invention;
FIG. 3 is a flowchart of an application data monitoring method according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a data monitoring device according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described below with reference to the accompanying drawings in the embodiments of the present invention.
The service data generated in most transaction scenes, such as configured commodity, price, payment mode, promotion and the like, are time-efficient, so that the time-efficient of the service data needs to be monitored, and the data about to expire is found and processed in time.
However, these service data are generally scattered and distributed in different storage locations in different storage modes, for example, different databases, different tables and different data files, so that when different service data are monitored, corresponding logic codes need to be deployed respectively to realize the monitoring, and the flexibility of data monitoring is lacking.
In order to solve the above problem, an embodiment of the present invention provides a data monitoring method, as shown in fig. 1, which may include:
s101, acquiring a plurality of configuration information;
wherein each configuration information includes data validity period information corresponding to service data in a storage location; different storage locations store traffic data in different storage modes.
S102, judging whether the service data in the storage position accords with the data validity period information according to each piece of configuration information.
And S103, if yes, triggering data expiration early warning.
The data monitoring method provided by the embodiment of the invention manages the data validity period information of the business data stored in different storage modes by using the configuration information, and when the data monitoring is needed, the unified monitoring of the business data stored in different storage modes is realized by acquiring a plurality of configuration information at one time. The service data stored in different storage modes can be monitored only by acquiring different configuration information, and meanwhile, the service program is not required to be redeployed by acquiring different configuration information, so that the flexibility of data monitoring can be improved.
Referring to fig. 1, a data monitoring method provided by an embodiment of the present invention will be described in detail.
S101, acquiring a plurality of configuration information;
wherein each configuration information includes data validity period information corresponding to service data in a storage location; different storage locations store traffic data in different storage modes.
The configuration information may also be referred to as data routing information and may be understood as a constraint on the timeliness of the traffic data. In particular, the timeliness of the service data may be constrained by the service type, the storage location, and the data expiration information, i.e., the configuration information may include the service type, the storage location, and the data expiration information, etc.
The service type, that is, the service module (module), is used to identify the service for which the configuration information is specific, so that the service type for which the technician is currently configured with the configuration information is prompted, and the service type for which the technician is currently monitored is also understood as being prompted. Such as payment means, vouchers, etc.
Different storage locations store traffic data in different storage modes, where the storage modes may include different database addresses (databases), different data files, and/or different table names (tablenames), etc.
The service data may be stored by using a database, in particular, the service data may be stored by using a table included in the database, and further different storage modes may be defined by the database and the table names, in particular, the different storage modes may include a storage mode indicated by different database addresses, a storage mode indicated by different table names, a storage mode indicated by different database addresses and different table names, and so on. The database address may be understood as a name of a database where the service data is located, and the table name is a name of a table in the database specifically used for storing the service data. Further, in the embodiment of the present invention, the service data in different storage locations may include different database addresses and/or different table names to store the service data.
It will be appreciated that since different service data is stored in different storage locations in different storage modes, service data of one service type may be considered to be stored in a particular storage location (table name of a particular database address) in a particular storage mode, with different service data being stored in different storage locations. The storage mode of the service data of different service types can be set by a developer according to actual conditions, and the embodiment of the invention is not limited.
The data validity period information may include a data validity period field. The service data is constrained by using the data validity period field as a time constraint condition, which can be understood that the data validity period information indicates a time condition that the service data with timeliness needs to meet.
In one implementation, the data validity period field may include only a data validity period time (or a data expiration time) corresponding to the service data.
In another implementation manner, the data validity period field may further include an early warning time (in_advance_day), that is, the data validity period field may correspond to the data validity period time and the early warning time of the service data.
The term "data validity period" corresponding to the service data may be understood as a field indicating the validity period of the service data stored in a storage location in a storage mode, such as the service data stored in a table name of a database address, and may also be referred to as a specified expiration field or a table field name recording the validity period.
The field names of the data validity periods may be different for different storage modes, i.e. in different databases/tables, e.g. readline, valid end time etc., subject to the naming convention of the developer. Therefore, the embodiment of the invention can accurately acquire the data validity period time aiming at different databases/tables by setting the data validity period time in the configuration information. Moreover, a technician can set the specific time indicated by the service validity period according to the service requirement, and the embodiment of the invention is not limited.
The early warning time can be understood as the number of days of early warning, namely the number of days of early warning sent to technicians in advance when the service data has an effective period, namely before the service data expires. The early warning time is related to the service, and a technician can set the early warning time based on different service requirements, for example, the early warning time can be 7 days, 5 days, 3 days, etc., which is not limited herein.
A configuration information may indicate a storage location, representing constraints on timeliness of the business data at the storage location; different configuration information may contain the same or different data validity period information according to different service requirements.
For example, the configuration information 1 may include data validity period information a corresponding to the service data of the storage location a; the configuration information 2 may include data validity period information a corresponding to the service data of the storage location B; the configuration information 3 may include data validity period information C corresponding to the service data of the storage location C. Wherein the configuration information 1, 2 and 3 are service data respectively indicating different storage locations, the configuration information 1 and 2 comprise the same data validity period information a, and the configuration information 3 comprises different data validity period information c.
In one implementation, the configuration information may be configured at a configuration center. Configuration information can be obtained from the configuration center at fixed time in a timing polling access mode, so that after technicians modify the configuration information, the changed configuration information is obtained from the configuration center in time, and the new configuration information is used for monitoring, so that dynamic configuration and dynamic monitoring of service data are realized.
The preset timing polling time, namely the polling frequency, the lower the polling frequency is, the longer the delay of the sensing configuration change is; the higher the polling frequency, the shorter the time to perceive the configuration change, but the higher the performance consumption, and the specific timing polling time can be determined according to the actual requirement, without limitation.
In one implementation, the time of the timed poll may be one day, i.e., each day, configuration information is obtained from a configuration center.
S102, judging whether the service data in the storage position accords with the data validity period information according to each piece of configuration information.
Since the configuration information may include, in particular, a service type, a database address, a table name, and a data validity period field. Accordingly, determining whether the service data in the storage location corresponds to the data validity period information may include: and judging whether the service data stored under the table name of the database address accords with the data validity period time indicated by the data validity period field.
In one implementation, the data validity period field may include a data validity period time corresponding to the business data, and an early warning time. For example, the data validity period field may include: the data validity period time and the early warning time corresponding to the service data stored in the table name of the database address can be based on the data validity period time and the early warning time corresponding to the service data stored in the table name of the database address, and whether the service data has timeliness or not can be judged.
The estimated termination time of the service data validity period, that is, the estimated failure time of the service data, can be estimated based on the current time and the early warning time. The sum of the current time and the early warning time can be used as the expected expiration time of the validity period of the service data, and the expected expiration time of the validity period is compared with the data validity period time to judge whether the service data has timeliness.
It can be understood that the date later than the date of the data validity period time indicates the date of the data without timeliness, so that when the validity period ending time is longer than the data validity period time, that is, the sum of the current time and the early warning time is longer than the data validity period time, the current service data is considered to be in accordance with the data validity period information indicated by the data validity period field, that is, the validity period of the current service data is about to be terminated, that is, the service data about to be invalidated, and the timeliness of the service data needs to be early warned.
For example, the data validity period of one configuration information is 2023, 9 and 1, and the early warning time is 3 days; when the current time is 2023, 8 and 30 days, and the timeliness of the service data is verified by acquiring the configuration information, the estimated expiration time of the validity period is 2023, 9 and 2 days, and the estimated expiration time of the validity period is compared with the data validity period time, so that the estimated expiration time of the validity period is larger than the data validity period time, namely the data does not have timeliness in the estimated expiration time of the validity period, namely the current service data is the service data about to be invalidated, and the timeliness of the current service data needs to be pre-warned.
When the expected expiration time of the validity period is not more than, namely not more than, the data validity period time, namely, the sum of the current time and the early warning time is not more than, the current service data is considered to be not in accordance with the data validity period information indicated by the data validity period field, and is the service data with timeliness, and the timeliness of the service data does not need to be early warned.
Still taking the configuration information as an example, if the current time is 2023, 8, 28 days, the estimated expected expiration time is 2023, 8, 31 days, and the expected expiration time is smaller than the data expiration time, that is, the data has timeliness in the expected expiration time, so that early warning of the timeliness is not needed.
In another implementation, the data validity period field may include a data validity period time corresponding to the traffic data. The validity period time of the data can be utilized to judge whether the service data has timeliness or not. For example, the data validity period field may include: data validity period time corresponding to the business data stored in the table name of the database address; judging whether the difference value between the validity period time and the current time is smaller than a preset time difference value or not; if yes, determining the service data stored in the table name of the database address, and conforming to the data validity period information indicated by the data validity period field.
The remaining validity time of the service data may be estimated based on the current time and the data validity time. Specifically, a difference between the data validity period time and the current time may be used as the remaining validity time of the service data. It will be appreciated that the traffic data fails when the remaining active time is zero. Taking the configuration information as an example, when the current time is 2023, 8, 30 days, the remaining valid time of the service data is the difference between the data valid period time 2023, 9, 1 day and the current time 2023, 8, 30 days, i.e. 2 days.
And comparing the remaining effective time with a preset time difference value, and judging whether the remaining effective time is smaller than the preset time difference value. The preset time difference may also be understood as an early warning time, for example, 1 day, 2 days, etc., and the specific time may be determined according to the actual requirement, which is not limited in the embodiment of the present invention.
When the remaining effective time is smaller than a preset time difference value, namely the difference value between the data effective period time and the current time is smaller than the preset time difference value, the service data is considered to be close to the effective period, and the service data is about to lose effectiveness and needs to be subjected to timeliness early warning; when the remaining effective time is not less than the preset time difference, namely the difference between the data effective period time and the current time is not less than the preset time difference, the service data can be considered to have a certain time from the failure, and early warning is not needed.
According to the embodiment of the invention, the service data about to lose timeliness is pre-warned in advance by setting the pre-warning time or the preset time difference value, so that technicians can find out the service data about to lose timeliness and possible problems more flexibly and timely. In addition, through the early warning time or the preset time difference value, more time adjustment data can be provided for related services and operators, and a certain fault tolerance rate is provided.
According to the embodiment of the invention, after the plurality of configuration information is acquired, the service data can be acquired from the storage position based on the configuration information, and then whether the service data of the storage position accords with the data validity period time indicated by the data validity period field is judged.
Specifically, based on the service type, the database address, the table name and the data validity period field included in the configuration information, the service data stored in the table name of the database address is obtained first by utilizing the query statement, and then whether the service data accords with the data validity period information indicated by the database validity period field is judged.
In one implementation, the business logic may be pre-deployed with general query logic. The general query logic includes query parameters that do not have a specific meaning, and therefore, the general query logic may also be understood as a query logic template. Specific query logic, or query statements, may be constructed by assigning specific parameter values to query parameters included in the general query logic. The embodiment of the invention can take the configuration information as a specific query parameter value, so that a query statement corresponding to the configuration information can be obtained by combining the configuration information with general query logic.
In one implementation manner, after a plurality of configuration information is acquired, a query statement corresponding to the configuration information may be obtained based on a service type, a database address, a table name, and a data validity period field included in the configuration information. Specifically, a specific query statement corresponding to each configuration information can be obtained by splicing the service type, the database address, the table name and the data validity period field included in the configuration information with general query logic.
The embodiment of the invention acquires a plurality of configuration information at one time and generates a plurality of specific query logics through one-time business logic, so that the configuration information can have the same format in order to match the configuration information with a preset query logic template.
In one implementation, one configuration information may include configurations regarding module, database, tableName, in_advance_day, and validity_field in order.
For example, configuring "cash register items" for the business module, i.e., business data that may represent "cash register items" for the business; configuring a 'store' for a database, and configuring a 'product' for a table name, namely, a table 'product' for indicating that service data is positioned in the 'store' of the database; and configuring 'readline' for the data validity period time, namely indicating that the data is valid before 'readline', and configuring 7 for the early warning time, namely indicating that early warning is performed 7 days in advance.
The specific configuration may be determined according to the actual situation, for example, the "price rule for ordering," "activity of payment result page," etc. are configured for the service module, and the corresponding database, table name, early warning time and data validity period time are not described herein. The embodiment of the invention does not limit the format of the configuration information.
The embodiment of the invention can uniformly manage the configuration information by utilizing the configuration center, namely, decoupling the service type, the storage position and the data validity period information included in the configuration information from specific query logic. The general query logic and the configuration information are spliced into a specific query statement, so that a technician can obtain different query logics only by editing the configuration information, and the logic implementation process is simple and is not easy to make mistakes.
Different service requirements can correspond to different configuration information, so that technicians can modify the configuration information in real time according to the current service requirements.
Referring to the data monitoring method shown in fig. 1, the acquired plurality of configuration information may include a plurality of configuration information after modification. The configuration information can be modified by adding, deleting and the like, and the modification of the configuration information can be realized by modifying the service type, the storage position and the data validity period information included in the configuration information. The specific modification mode and modification result can be determined according to the actual service requirement, and the embodiment of the invention is not limited.
After a technician modifies the configuration information according to service requirements, the modified configuration information can be obtained, so that service types, storage positions and data validity period information included in the modified configuration information are obtained, and therefore, the timeliness of service data can be monitored by utilizing the modified configuration information by executing the steps of the subsequent other data monitoring methods with reference to fig. 1.
The configuration information is modified without changing service logic, so that the modification can be effected without redeploying service, and thus, when the service requirement is adjusted and the timeliness of new service data is required to be monitored, the configuration information is modified aiming at the new service data to be monitored, the timeliness of the new service data can be monitored without suspending service, and the access to the timeliness alarm of the new service data is dynamically supported. And only configuration information is modified, so that the modification efficiency can be improved, and the influence of redeployment service on the service is reduced.
After the plurality of configuration information is acquired, the configuration information may exist in the form of a list. Specifically, one configuration information may correspond to one table, where each element included in the table corresponds to each item of content included in the configuration information, that is, a service type, a database address, a table name, and a data validity period field.
The configuration information may include a string in JSON (JavaScript Object Notation, JS object numbered musical notation) format; after the configuration information is acquired, the JSON format string may be converted into an array, so as to obtain configuration information in a list format. The embodiment of the invention does not limit the method for converting the JSON format character strings into the array to obtain the configuration information in the list format.
In one implementation, configuration information in JSON string format may be obtained from a configuration center, and then the JSON string configured by the configuration center may be converted into an array using an open source tool, for example, setting an open source toolkit Fastjson (JSON parser and generator). For example, the configuration information of the character string format may be stored in a list () type using a method json.paramisarray () that parses the character string into an array. Thereby obtaining configuration information in list format.
After obtaining the plurality of configuration information lists, the plurality of lists may be traversed in turn, so as to obtain, for each list, each element included in the list, and further, the obtained element may be combined with a preset query logic template to obtain a query statement corresponding to each list, that is, each configuration information.
The traffic data may include valid traffic data and invalid traffic data. The valid service data can be understood as service data with timeliness; correspondingly, the invalid service data is the service data which has no timeliness, namely has expired.
In one implementation, a status identifier may also be added to the service data to distinguish between valid service data and invalid service data. For example, with status as a status identifier, valid traffic data can be identified with status=1. Further, after the service data expires, the status identifier thereof may be modified, i.e. the status identifier indicating validity is modified to be invalid. The embodiment of the invention does not limit the form of the state identification for marking whether the service data is valid or not.
In one implementation manner, the service data in the table name of the database address to be queried is valid state status=1, so as to reduce the range of service data needing to judge timeliness. For example, the service data in a valid state may be acquired from a storage location indicated by the configuration information using the following logic.
When further acquiring service data satisfying the timeliness time condition, whether the validity end time of the service data is empty can be judged first, and then the timeliness condition of the service data, namely whether the service data has timeliness, is judged. For example, it may be determined whether the service data satisfies the validity time condition by comparing the relationship between the data validity time and the current time.
In one implementation manner, if valid_end_time is used to represent the data validity period time and now () is used to represent the current time, valid_end_time > now () may be used to determine whether the service data satisfies the validity period condition, that is, whether the data validity period time is greater than the current time. It can be appreciated that if the current service data is larger than the current service data, the current service data is not expired, and timeliness is achieved.
According to the embodiment of the invention, general query logic can be preset in the business logic, so that specific data query logic does not need to be written in the business logic, a corresponding configuration information list is directly obtained from the configuration file, then the list is traversed, the traversing result and the general query logic are spliced to obtain specific execution SQL (structured query language), namely query sentences, and the corresponding data to be expired is obtained by utilizing the query sentences. Therefore, only one set of general business logic is required to be written, and the data of a plurality of storage positions can be queried. When the service data to be monitored is required to be adjusted, the service code is not required to be modified, the service is not required to be redeployed, and whether the service data is about to expire or not can be inquired only by correspondingly modifying the configuration information, so that a great amount of time for writing and checking the logic code is saved for technicians, the labor cost is saved, and the working efficiency of the technicians is improved.
And S103, if yes, triggering data expiration early warning.
If so, the service data in the storage location is expired when the service data accords with the data validity period information. At this point, a data expiration pre-alarm may be triggered.
The data expiration early warning can also be called as data validity early warning to remind that the remaining time of the data validity is insufficient. Triggering the data expiration pre-warning may include sending a pre-warning prompt message to the service end.
In one implementation, the early warning prompt information may include an early warning prompt statement, which may be understood as a prompt that informs a technician at the service end that service data is about to expire. For example, the alert cue statement may include "insufficient service data validity period", "service data about to expire", and the like.
And generating corresponding early warning prompt sentences based on the configuration information. I.e., an early warning alert statement may be generated based on one or more of the business module, database address, table name, and data validity period field included in the configuration information. For example, the early warning prompt information may include prompt sentences such as "the service data in the database a table a is about to expire", "the service data of the service module M is about to expire", "the service data validity period of the service module M is also N days", and the like. The embodiment of the invention does not limit the form of the early warning prompt statement.
In another implementation, the early warning cue information may also include upcoming expiration service data. The early warning prompt statement and the service data to be expired can be sent to the service end, so that specific service data to be expired can be provided for technicians.
Under the condition that the service data in the storage position accords with the data validity period information, the service data which accords with the data validity period information is obtained and stored in an Excel (electronic table) format, and the service data and the early warning prompt statement are used as early warning prompt information together to be sent to a service end.
An example diagram of expiration data is shown in fig. 2. The service data to be expired may include service data of service 1 and service 2 indicated by sequence number 1 and sequence number 2, respectively, where id (identity) may include a number of the service data in a database where the service data is located, and the start time and the end time may represent a start time and an end time of an validity period of the service data.
The mode of carrying out data expiration early warning on the service end is not limited by the embodiment of the invention. For example, early warning prompt information can be sent to a service end through communication software, mails and other forms to perform data expiration early warning.
Meanwhile, in order to more intuitively prompt a developer that the service data is about to expire, when the data expiration early warning is sent in the forms of communication software, mails and the like, an early warning prompt statement can be sent at the same time when the service data is sent, so that the sent data is the data about to expire. The early warning prompt statement may include "[ important prompt ] about to expire transaction configuration data" [ important prompt ] about to expire, ask the product and the operating classmates to confirm whether the policy adjustment is needed-! The following is carried out The following is carried out "etc., embodiments of the present invention are not limited.
In one implementation, the configuration information may include, in a service module, a service responsible person contact, for example, a mailbox address of a related product and developer. Therefore, when the data expiration early warning is carried out, early warning prompt information can be sent to appointed personnel, so that the data about to expire can be found and processed more quickly.
In the embodiment of the invention, the storage device for storing the service data may include one or more storage devices, so in order to improve the searching efficiency of the service data, in an achievable manner, the configuration information may further include configuration for the storage device.
Furthermore, the different storage modes may include different storage devices and/or different database addresses and/or different table names, and whether to trigger expiration pre-warning of the service data may be determined by determining whether the service data stored in a storage device under a table name of a database conforms to the data validity period information indicated by the data validity period field. By means of configuration for the storage device, when the service data to be monitored are stored in different storage devices in a scattered mode, the storage device for storing the service data can be rapidly located, so that the service data to be subjected to timeliness monitoring can be rapidly obtained, and the efficiency of data monitoring is improved.
According to the data monitoring method provided by the embodiment of the invention, the configuration information is obtained once, and is combined with a set of general preset query logic in sequence, so that a plurality of specific query sentences are obtained, and the timeliness of the service data stored in different positions in different storage modes is judged, so that the time for constructing the specific logic by technicians can be saved, the labor cost is saved, and the development efficiency is improved.
And the configuration information is uniformly managed in the configuration center and decoupled from the service logic, so that the service can be inquired without redeployment when the configuration information is modified, the data monitoring process is more flexible, and the service data to be monitored can be conveniently adjusted at any time by technicians according to service requirements.
In addition, the data monitoring method provided by the embodiment of the invention is not only limited to monitoring the timeliness of the data, but also can be suitable for dynamic configuration of other requirements by adjusting the structure of the configuration information, so that unified monitoring of other requirements is realized.
In one implementation manner, as shown in fig. 3, the data monitoring method provided by the embodiment of the present invention may include:
s301, starting a timing task.
The frequency of acquiring the configuration information can be preset, and the configuration information can be acquired at regular time. For example, the frequency of acquiring the configuration information can be preset to be acquired once a day, that is, the configuration information is acquired once a day, and whether the service data aimed by the configuration information is about to expire is judged, so that the service data about to expire is found in time, and a technician can conveniently adjust the service in time.
S302, acquiring a configuration information list from a configuration center.
The configuration center may be configured with a plurality of configuration information, and the plurality of configuration information may have the same format. Each configuration information includes data validity period information corresponding to service data in a storage location.
In particular, the configuration information may include a service type, a storage location, and data validity period information. Wherein the storage location may include a database address indicating a specific storage location for the service data and a table name; the data validity period information may include a data validity period field indicating a time condition that the service data having timeliness needs to satisfy.
The configuration information may be preconfigured in the configuration center by a technician for the current business needs. The configuration information obtained from the configuration center can comprise configuration information in a JSON character string format, and the configuration information in the character string format can be saved through an array by using an open source tool, so that a list-format distribution to an information list is obtained.
Each configuration information list corresponds to one configuration information, and thus the number of configuration information lists acquired from the configuration center may be plural.
S303, traversing the elements in the list, and constructing a corresponding SQL sentence for each element.
And traversing the configuration information lists respectively to obtain a plurality of elements included in each configuration information list. The plurality of elements included in each configuration information list may correspond to the plurality of information included in the configuration information, respectively, i.e., the configuration information list elements may correspond to the service type, the storage location, and the data validity period information.
The business logic can be pre-deployed with general query logic or query logic templates, and a plurality of elements obtained by traversing the configuration information list can be regarded as specific parameters needed by constructing specific query logic, namely, the general query logic is spliced with the plurality of elements in the configuration information list, or the general parameters included in the general query logic are replaced by the plurality of elements in the configuration information list, so that specific query logic aiming at each configuration information can be obtained, and further, whether the business data in different storage positions have timeliness can be queried by using the specific query logic.
Thus, the query codes for the business data stored in different storage positions in different storage modes can be obtained by combining the preset query logic, so that unified monitoring of the business data stored in different storage positions in different storage modes is realized.
S304, inquiring the data about to expire from the database.
The data validity period information may include an early warning time. And further, the SQL statement is utilized, and the queried service data meeting the data validity period information is the service data about to be invalidated in the storage position indicated by the configuration information, so that a certain time is provided for technicians to process the service data about to be expired.
It will be appreciated that none of the service data in the current storage location may be data that is about to expire, i.e. after searching using the SQL statement, there is no data in the database that satisfies the data expiration information, and step S305 may not be performed any further.
S305, interfacing with a content communication tool or mail, and sending out early warning information.
After the data to be expired are queried from the database, the data can be stored in a form of a table and sent to related personnel for early warning through an internal communication tool or mail, so that the related personnel can process the data to be expired according to the received information. For example, adjust the validity time of the relevant service data, etc.
Expiration data pre-warning logic can be preset, and is invoked when the database is queried for the data to be expired, and the data to be expired is sent to related personnel.
The data monitoring method provided by the embodiment of the invention can acquire the configuration information at regular time, sense the change of the configuration information, and timely discover and acquire the service data indicated by the modified configuration information when the configuration information changes, thereby realizing the dynamic monitoring of the data. By acquiring a plurality of configuration information at a time, unified monitoring of service data stored in different storage positions in different storage modes is realized. The monitoring of different service data can be realized only by acquiring different configuration information, and meanwhile, the service program is not required to be redeployed by acquiring different configuration information, so that the flexibility of data monitoring can be improved.
As shown in fig. 4, an embodiment of the present invention provides a data monitoring apparatus, which may include:
an obtaining module 401, configured to obtain a plurality of configuration information; each configuration information comprises data validity period information corresponding to service data in a storage position; different storage locations store business data in different storage modes;
A judging module 402, configured to judge, for each configuration information, whether the service data in the storage location accords with the data validity period information;
and the early warning model 403 is used for triggering early warning of the data validity period if yes.
Optionally, the configuration information includes a service type, a database address, a table name, and a data validity period field; different storage modes include different database addresses and/or different table names;
the judging module 402 is specifically configured to judge whether the service data stored in the table name of the database address accords with the data validity period information indicated by the data validity period field.
Optionally, the data validity period field comprises data validity period time corresponding to the service data and early warning time;
the judging module 402 is specifically configured to judge whether the sum of the current time and the early warning time is greater than the data validity period time.
Optionally, the data validity period field includes a data validity period time corresponding to the service data;
the determining module 402 is specifically configured to determine whether a difference between the data validity time and the current time is smaller than a preset time difference.
Optionally, the apparatus further comprises,
the query statement module is used for obtaining a query statement corresponding to the configuration information based on the service type, the database address, the table name and the data validity period field included in the configuration information after the plurality of configuration information are obtained;
The judging module 402 is specifically configured to obtain, using the query statement, service data stored in a table name of the database address, and judge whether the service data accords with the data validity period information indicated by the data validity period field.
Optionally, the early warning module 403 is specifically configured to send early warning information to the service end when the service data in the storage location accords with the data validity period information.
The data monitoring device provided by the embodiment of the invention manages the data validity period information of the service data stored in different storage modes by using the configuration information, and when the data monitoring is needed, the unified monitoring of the service data stored in different storage modes is realized by acquiring a plurality of configuration information at one time. The service data stored in different storage modes can be monitored only by acquiring different configuration information, and meanwhile, the service program is not required to be redeployed by acquiring different configuration information, so that the flexibility of data monitoring can be improved.
The embodiment of the invention also provides an electronic device, as shown in fig. 5, which comprises a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502 and the memory 503 complete communication with each other through the communication bus 504,
A memory 503 for storing a computer program;
the processor 501 is configured to implement the data monitoring method when executing the program stored in the memory 503.
The communication bus mentioned by the above terminal may be a peripheral component interconnect standard (Peripheral Component Interconnect, abbreviated as PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated as EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface is used for communication between the terminal and other devices.
The memory may include random access memory (Random Access Memory, RAM) or non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but also digital signal processors (Digital Signal Processor, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field-programmable gate arrays (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
In yet another embodiment of the present invention, a computer readable storage medium is provided, in which a computer program is stored, which when executed by a processor, implements the data monitoring method according to any of the above embodiments.
In yet another embodiment of the present invention, a computer program product containing instructions that, when run on a computer, cause the computer to perform the data monitoring method of any of the above embodiments is also provided.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for an apparatus, an electronic device, a computer readable storage medium, a computer program product embodiment containing instructions, the description is relatively simple as it is substantially similar to the method embodiment, as relevant see the partial description of the method embodiment.
The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims (10)

1. A method of data monitoring, the method comprising:
acquiring a plurality of configuration information; each configuration information comprises data validity period information corresponding to service data in a storage position; different storage locations store business data in different storage modes;
judging whether the service data in the storage position accords with the data validity period information or not according to each piece of configuration information;
if yes, triggering the data expiration early warning.
2. The method of claim 1, wherein the configuration information includes a service type, a database address, a table name, and a data validity period field; the different storage modes comprise different database addresses and/or different table names;
the judging whether the service data in the storage position accords with the data validity period information comprises the following steps:
and judging whether the service data stored in the table name of the database address accords with the data validity period information indicated by the data validity period field.
3. The method of claim 1, wherein the data validity period information includes a data validity period time corresponding to the service data, and an early warning time;
the judging whether the service data in the storage position accords with the data validity period information comprises the following steps:
judging whether the sum of the current time and the early warning time is larger than the data validity period time.
4. The method of claim 1, wherein the data validity period information comprises: the data validity period time corresponding to the service data;
the judging whether the service data in the storage position accords with the data validity period information comprises the following steps:
and judging whether the difference value between the data validity period time and the current time is smaller than a preset time difference value or not.
5. The method of claim 2, wherein after the obtaining the plurality of configuration information, the method further comprises:
obtaining a query statement corresponding to the configuration information based on the service type, the database address, the table name and the data validity period field included in the configuration information;
the judging whether the service data stored in the table name of the database address accords with the data validity period information indicated by the data validity period field comprises the following steps:
And acquiring service data stored in the table name of the database address by utilizing the query statement, and judging whether the service data accords with the data validity period information indicated by the data validity period field.
6. The method of claim 1, wherein if so, triggering a data expiration pre-alarm comprises:
and sending early warning information to a service terminal under the condition that the service data in the storage position accords with the data validity period information.
7. A data monitoring device, the device comprising:
the acquisition module is used for acquiring a plurality of configuration information; each configuration information comprises data validity period information corresponding to service data in a storage position; different storage locations store business data in different storage modes;
the judging module is used for judging whether the service data in the storage position accords with the data validity period information or not according to each piece of configuration information;
and the early warning model is used for triggering the data expiration early warning if yes.
8. The apparatus of claim 7, wherein the configuration information comprises a service type, a database address, a table name, and a data validity period field; the different storage modes comprise different database addresses and/or different table names;
The judging module is specifically configured to judge whether the service data stored in the table name of the database address accords with the data validity period information indicated by the data validity period field.
9. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
a memory for storing a computer program;
a processor for implementing the steps of the method of any one of claims 1-6 when executing a program stored on a memory.
10. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program which, when executed by a processor, implements the steps of the method of any of claims 1-6.
CN202311335785.3A 2023-10-16 2023-10-16 Data monitoring method and device, electronic equipment and storage medium Pending CN117312088A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311335785.3A CN117312088A (en) 2023-10-16 2023-10-16 Data monitoring method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311335785.3A CN117312088A (en) 2023-10-16 2023-10-16 Data monitoring method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN117312088A true CN117312088A (en) 2023-12-29

Family

ID=89280916

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311335785.3A Pending CN117312088A (en) 2023-10-16 2023-10-16 Data monitoring method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN117312088A (en)

Similar Documents

Publication Publication Date Title
CN108156022B (en) Service calling method and device and electronic equipment
CN108874558B (en) Message subscription method of distributed transaction, electronic device and readable storage medium
CN109617646B (en) Message conversion method and device, computer equipment and computer readable storage medium
CN109213519B (en) Heterogeneous API adaptation processing method and system
CN107276970B (en) Unbinding and binding method and device
US8862613B2 (en) Extensibility of business process and application logic
CN110688828A (en) File processing method and device, file processing system and computer equipment
CN109617647A (en) Message forwarding method, device, computer equipment and computer readable storage medium
CN110895472A (en) Method and device for identifying service change
CN114266539A (en) File flow processing method, system, device and computer readable storage medium
CN113298331A (en) Work order processing method, device and system and electronic equipment
CN116151631A (en) Service decision processing system, service decision processing method and device
CN117312088A (en) Data monitoring method and device, electronic equipment and storage medium
CN112491943A (en) Data request method, device, storage medium and electronic equipment
CN112579682B (en) Method and device for notifying change of data model, electronic equipment and storage medium
CN111291127B (en) Data synchronization method, device, server and storage medium
US8661454B2 (en) System and method for receiving and transmitting event information
CN111177155B (en) Message filtering method, system and computer equipment
CN114036132A (en) Object information processing method and device, storage medium and electronic equipment
CN112860292A (en) Configuration management method and device based on application
CN109254855B (en) Registration method and device of parameter transmission protocol and electronic equipment
CN113064905A (en) Business process processing method and device, electronic equipment and computer readable medium
CN110782380A (en) Aeroderivative information management method, aeroderivative information management device and storage medium
CN111163088B (en) Message processing method, system and device and electronic equipment
CN113778423B (en) Interface document generation method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination