CN117149568A - Index collection method, device, electronic equipment and computer readable medium - Google Patents
Index collection method, device, electronic equipment and computer readable medium Download PDFInfo
- Publication number
- CN117149568A CN117149568A CN202311118461.4A CN202311118461A CN117149568A CN 117149568 A CN117149568 A CN 117149568A CN 202311118461 A CN202311118461 A CN 202311118461A CN 117149568 A CN117149568 A CN 117149568A
- Authority
- CN
- China
- Prior art keywords
- index
- acquisition
- collection
- threshold
- script
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 200
- 238000013515 script Methods 0.000 claims abstract description 85
- 238000009434 installation Methods 0.000 claims abstract description 34
- 238000012423 maintenance Methods 0.000 claims abstract description 25
- 230000008569 process Effects 0.000 claims description 22
- 238000004590 computer program Methods 0.000 claims description 15
- 238000003860 storage Methods 0.000 claims description 13
- 238000007689 inspection Methods 0.000 abstract description 14
- 230000008859 change Effects 0.000 abstract description 6
- 238000012986 modification Methods 0.000 description 12
- 230000004048 modification Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 10
- 238000004519 manufacturing process Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000012544 monitoring process Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000010276 construction Methods 0.000 description 3
- 238000005538 encapsulation Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 241001137251 Corvidae Species 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 239000000523 sample Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3089—Monitoring arrangements determined by the means or processing involved in sensing the monitored data, e.g. interfaces, connectors, sensors, probes, agents
- G06F11/3093—Configuration details thereof, e.g. installation, enabling, spatial arrangement of the probes
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses an index acquisition method, an index acquisition device, electronic equipment and a computer readable medium, and relates to the technical field of cloud computing. One embodiment of the method comprises the following steps: receiving an index collection script pushed by an operation and maintenance platform, and storing the index collection script under a temporary path; the index collection script comprises at least one collection method; running an index collection script to perform the steps of: acquiring an index configuration table from a database, wherein each index rule comprises an index name, a threshold judgment symbol, a threshold, an acquisition method identifier and an acquisition main key, and the acquisition main key comprises an installation path and a keyword of a service on a physical machine; traversing each index rule, and obtaining an index acquisition result according to the acquisition main key, the acquisition method corresponding to the acquisition method identifier, the threshold judgment symbol and the threshold for each index rule. The implementation mode can solve the technical problems that the inspection script needs to be continuously modified and the version is released due to the change of cluster new establishment, cluster upgrading and cluster operation and maintenance.
Description
Technical Field
The present invention relates to the field of cloud computing technologies, and in particular, to an index collection method, an index collection device, an electronic device, and a computer readable medium.
Background
With the development and further deepening of business, a large number of TDSQL clusters are deployed in a production environment, and the production operation and maintenance face higher challenges. The inspection device is used as a complementary scheme of TDSQL original inspection, can personalize the newly-increased required acquisition index, and is used as a disaster recovery scheme of TDSQL inspection, and has the capability of cluster basic inspection when TDSQL self-contained monitoring faults are not available.
In the process of implementing the present invention, the inventor finds that at least the following problems exist in the prior art:
index items and acquisition logic in the inspection device are completely solidified in the script, and when cluster upgrading, operation and maintenance change and other factors introduce cluster change, the current inspection script can not meet the inspection requirement and needs frequent modification and script release for adaptation. For example, when a new process or timing task is introduced, the current script needs to be modified for version release; when a cluster introduces a new variable (deploying non-cluster service) to cause inaccurate current acquisition results, the current script needs to be modified for version release; when some indexes are abandoned or risks are introduced in some index collection extreme cases, the indexes cannot be closed or the risks cannot be avoided in time; when some indexes are collected and the collection threshold value is required to be adjusted according to production requirements, the current script is required to be modified for version release.
In addition, because the index item is not standardized, different inspection scripts have different configuration item standards, and the unified management of operation and maintenance is inconvenient. Moreover, because the production environment is complex, the scripts are lengthy and do not encapsulate the acquisition method aiming at the index type, and risks are easy to be introduced when the scripts are modified and released.
Disclosure of Invention
In view of this, the embodiments of the present invention provide an index collection method, apparatus, electronic device, and computer readable medium, so as to solve the technical problems of continuous modification and version release of inspection scripts caused by changes in cluster creation, cluster upgrade, and cluster operation and maintenance.
In order to achieve the above object, according to an aspect of the embodiments of the present invention, there is provided an index collection method applied to a target physical machine, including:
receiving an index collection script pushed by an operation and maintenance platform, and storing the index collection script under a temporary path; wherein the index collection script comprises at least one collection method;
and running the index collection script to execute the following steps:
acquiring an index configuration table from a database, wherein the index configuration table comprises at least one index rule, each index rule comprises an index name, a threshold judgment symbol, a threshold, an acquisition method identifier and an acquisition main key, and the acquisition main key comprises an installation path and a keyword of a service on a physical machine;
Traversing each index rule, and for each index rule, obtaining an acquisition result of the index according to the acquisition main key, the acquisition method corresponding to the acquisition method identifier, the threshold judgment symbol and the threshold;
and writing the acquisition result of each index into the database.
Optionally, for each index rule, according to the collection main key, the collection method corresponding to the collection method identifier, the threshold judgment symbol and the threshold, obtaining a collection result of the index includes:
for each index rule, extracting an installation path of the service on the physical machine from the acquisition main key, and judging whether the installation path of the service on the physical machine exists on the target physical machine or not;
if yes, extracting a keyword from the acquisition main key, and acquiring an acquisition result of the index according to the keyword, an acquisition method corresponding to the acquisition method identifier, the threshold judgment symbol and the threshold;
if not, ending the collection of the index.
Optionally, according to the keyword, the acquisition method corresponding to the acquisition method identifier, the threshold judgment symbol and the threshold, acquiring an acquisition result of the index includes:
Calling the acquisition method corresponding to the acquisition method identifier and taking the keyword as a reference, thereby obtaining a return value;
and obtaining the acquisition result of the index according to the return value, the threshold judgment symbol and the threshold.
Optionally, calling the collection method to identify a corresponding collection method and taking the keyword as a reference, thereby obtaining a return value, including:
judging whether an acquisition method corresponding to the acquisition method identifier exists in the index acquisition script or not;
if yes, calling a collection method corresponding to the collection method identifier and taking the keyword as a reference, so as to obtain a return value;
if not, ending the collection of the index, and writing a prompt which does not support the collection of the index into the database.
Optionally, the keywords include a process keyword or a database retrieval keyword; each index rule further comprises an index switch, an index grade and an index classification.
Optionally, after writing the acquired result of each index into the database, the method further includes:
and deleting the index collection script and the index configuration table.
In addition, according to another aspect of the embodiment of the present invention, there is provided an index collection device, provided to a target physical machine, including:
The receiving module is used for receiving the index acquisition script pushed by the operation and maintenance platform and storing the index acquisition script under the temporary path; wherein the index collection script comprises at least one collection method;
the acquisition module is used for running the index acquisition script to execute the following steps:
acquiring an index configuration table from a database, wherein the index configuration table comprises at least one index rule, each index rule comprises an index name, a threshold judgment symbol, a threshold, an acquisition method identifier and an acquisition main key, and the acquisition main key comprises an installation path and a keyword of a service on a physical machine;
traversing each index rule, and for each index rule, obtaining an acquisition result of the index according to the acquisition main key, the acquisition method corresponding to the acquisition method identifier, the threshold judgment symbol and the threshold;
and writing the acquisition result of each index into the database.
Optionally, the acquisition module is further configured to:
for each index rule, extracting an installation path of the service on the physical machine from the acquisition main key, and judging whether the installation path of the service on the physical machine exists on the target physical machine or not;
If yes, extracting a keyword from the acquisition main key, and acquiring an acquisition result of the index according to the keyword, an acquisition method corresponding to the acquisition method identifier, the threshold judgment symbol and the threshold;
if not, ending the collection of the index.
Optionally, the acquisition module is further configured to:
calling the acquisition method corresponding to the acquisition method identifier and taking the keyword as a reference, thereby obtaining a return value;
and obtaining the acquisition result of the index according to the return value, the threshold judgment symbol and the threshold.
Optionally, the acquisition module is further configured to:
judging whether an acquisition method corresponding to the acquisition method identifier exists in the index acquisition script or not;
if yes, calling a collection method corresponding to the collection method identifier and taking the keyword as a reference, so as to obtain a return value;
if not, ending the collection of the index, and writing a prompt which does not support the collection of the index into the database.
Optionally, the keywords include a process keyword or a database retrieval keyword; each index rule further comprises an index switch, an index grade and an index classification.
Optionally, the acquisition module is further configured to:
and after the acquisition results of the indexes are written into the database, deleting the index acquisition script and the index configuration table.
According to another aspect of an embodiment of the present invention, there is also provided an electronic device including:
one or more processors;
storage means for storing one or more programs,
the one or more processors implement the method of any of the embodiments described above when the one or more programs are executed by the one or more processors.
According to another aspect of an embodiment of the present invention, there is also provided a computer readable medium having stored thereon a computer program which, when executed by a processor, implements the method according to any of the embodiments described above.
According to another aspect of embodiments of the present invention, there is also provided a computer program product comprising a computer program which, when executed by a processor, implements the method according to any of the embodiments described above.
One embodiment of the above invention has the following advantages or benefits: because the index configuration table is obtained from the database, the index configuration table comprises at least one index rule, each index rule comprises an index name, a threshold value judgment symbol, a threshold value, an acquisition method identifier and an acquisition main key, the acquisition main key comprises an installation path and a keyword which serve on a physical machine, each index rule is traversed, and the technical means of acquiring an index acquisition result is obtained according to the acquisition main key, the acquisition method corresponding to the acquisition method identifier, the threshold value judgment symbol and the threshold value for each index rule, so that the technical problems that in the prior art, the inspection script needs to be continuously modified and the version is released due to the changes introduced by cluster new establishment, cluster upgrading and cluster operation and maintenance are solved. According to the embodiment of the invention, the standardized index item configuration and encapsulation index collection method is abstracted, so that an index collection script is concise and easy to maintain, and the collection logic is more robust; and the modification operation of the index configuration table is adopted to flexibly cope with the change of the production cluster, the modification frequency of the index acquisition script is reduced, the risk introduction is reduced, and the lengthy version release process is avoided.
Further effects of the above-described non-conventional alternatives are described below in connection with the embodiments.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. Wherein:
FIG. 1 is a flow chart of an index collection method according to an embodiment of the present invention;
FIG. 2 is a system architecture diagram of a TDSQL cluster according to an embodiment of the invention;
FIG. 3 is a flow chart of a method of index acquisition according to one referenceable embodiment of the invention;
FIG. 4 is a flow chart of an index collection method according to another exemplary embodiment of the present invention;
FIG. 5 is a schematic diagram of an index collection device according to an embodiment of the present invention;
FIG. 6 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;
fig. 7 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
In the technical scheme of the invention, the aspects of acquisition, analysis, use, transmission, storage and the like of the related user personal information all meet the requirements of related laws and regulations, are used for legal and reasonable purposes, are not shared, leaked or sold outside the aspects of legal use and the like, and are subjected to supervision and management of a supervision department. Necessary measures should be taken for the personal information of the user to prevent illegal access to such personal information data, ensure that personnel having access to the personal information data comply with the regulations of the relevant laws and regulations, and ensure the personal information of the user. Once these user personal information data are no longer needed, the risk should be minimized by limiting or even prohibiting the data collection and/or deletion.
User privacy is protected, when applicable, by de-identifying the data, including in some related applications, such as by removing specific identifiers (e.g., name, age, gender, date of birth, etc.), controlling the amount or specificity of data stored, controlling how the data is stored, and/or other methods.
Fig. 1 is a flowchart of an index collection method according to an embodiment of the present invention. As an embodiment of the present invention, as shown in fig. 1, the method for collecting the index is applied to the target physical machine, and may include:
and step 101, receiving an index acquisition script pushed by the operation and maintenance platform, and storing the index acquisition script under a temporary path.
The operation and maintenance personnel can issue an index acquisition script on the operation and maintenance platform, wherein the index acquisition script comprises at least one acquisition method, and then the IP of the index acquisition script and a target physical machine needing to be patrolled and examined is bound by creating a timing task. When the execution time of the timing task is reached, the operation and maintenance platform automatically pushes the index collection script to the temporary path (such as/tmp path) of the target physical machine. And after the target physical machine receives the index collection script pushed by the operation and maintenance platform, storing the index collection script under the temporary path.
Optionally, the index collection script may be Shell, which is a scripting language, and has rich commands, and completes collection of indexes such as whether the TDSQL cluster process survives, whether the timing task is configured, whether the NTPD service is healthy, whether the physical machine CPU, the memory, the load, whether to restart, and the like through the commands.
Alternatively, the target physical machine may be any physical machine in a TDSQL cluster. As shown in FIG. 2, the TDSQL cluster mainly comprises ES, proxy, zookeeper, meta cluster, db module, scheduler/manager, collector, monitor db, oss, chitu, analysis, clouddba and other components, wherein the components except proxy and db module are management and control components. It should be noted that, one component may be deployed on each physical machine, or multiple components may be deployed, which is not limited in this embodiment of the present invention.
TDSQL is a domestic database supporting distributed, non-distributed MySQL on enterprise-level kernel optimization, which is a collection of management and control components and DB components; the chip is an operation management platform and provides a management platform for cluster monitoring, instance adding, deleting, checking and the like; the zookeeper provides functions of configuration maintenance, election decision, route synchronization and the like for the decision TDSQL cluster, can support the works of creating, deleting, replacing and the like of a database node group, and uniformly issues and schedules all DDL (database schema definition language) operations; the Scheduler is used for scheduling the TDSQL cluster, helping the DBA or database users to automatically schedule and run various types of jobs, such as database backup, collection monitoring, generation of various reports or execution of business processes; the manager is responsible for monitoring the scheduling management of the cluster, and mainly managing operations such as capacity expansion/contraction, route switching and the like of the instance; OSS is a service encapsulated at the upper layers of manager component and Scheduler component, and is used for third party application to call OSS interface to complete operation of instance; ES is ELASTIC DB, which is responsible for collecting and storing proxy logs; the collector mainly collects index values from the zookeeper and writes the index values into a cluster monitoring library (monitor db); analysis is an alarm process; clouddba is a magpie component that helps the DBA perform performance analysis and fault diagnosis on example operating states; PROXY is a cluster computing node mainly responsible for forwarding service connections to DB; DB mobile contains MySQL and MYSQLAGENT components, MYSQLAGENT is the probe component of DB and is responsible for DB log cleaning and instance creation, switching, etc.
Step 102, running the index collection script to execute steps 103-105.
And after the target physical machine stores the index collection script pushed by the operation and maintenance platform under the temporary path, the target physical machine automatically runs the index collection script, so that steps 103-105 are executed.
Step 103, obtaining an index configuration table from a database, wherein the index configuration table comprises at least one index rule, each index rule comprises an index name, a threshold judgment symbol, a threshold, an acquisition method identifier and an acquisition main key, and the acquisition main key comprises an installation path and a keyword of a service on a physical machine.
And acquiring the configured index configuration table from the database by running the index acquisition script. The index configuration table comprises at least one index rule, wherein each index rule comprises an index name, a threshold judgment symbol (such as greater than, less than, greater than or equal to or less than), a threshold, an acquisition method identifier and an acquisition main key (key), and the acquisition main key comprises an installation path and a keyword of a service on a physical machine. Optionally, each index rule may further include an index identifier, an index primary classification, an index secondary classification, and an index tertiary classification.
Optionally, the keywords include a process keyword or a database retrieval keyword; each index rule further comprises an index switch, an index grade and an index classification. For example, the acquisition primary key may be/data/application/clouddba; the/conf/diagnostis.conf may be/data/application/clouddba; check_clo_active.sh, wherein,/data/application/clouddba is the installation path of the service on the physical machine,/conf/diagnostis.conf and check_clo_active.sh are process keys.
Step 104, traversing each index rule, and for each index rule, obtaining an acquisition result of the index according to the acquisition main key, the acquisition method corresponding to the acquisition method identifier, the threshold judgment symbol and the threshold.
After an index configuration table is obtained from a database, traversing each index rule in the index configuration table, and carrying out index acquisition on each index rule according to an acquisition main key of the index rule, an acquisition method corresponding to an acquisition method identifier, a threshold judgment symbol and a threshold, thereby obtaining an acquisition result of the index.
Optionally, for each index rule, according to the collection main key, the collection method corresponding to the collection method identifier, the threshold judgment symbol and the threshold, obtaining a collection result of the index includes: for each index rule, extracting an installation path of the service on the physical machine from the acquisition main key, and judging whether the installation path of the service on the physical machine exists on the target physical machine or not; if yes, extracting a keyword from the acquisition main key, and acquiring an acquisition result of the index according to the keyword, an acquisition method corresponding to the acquisition method identifier, the threshold judgment symbol and the threshold; if not, ending the collection of the index. For each index rule, extracting an installation path of a service on a physical machine from an acquisition main key, such as/data/application/cloddba, judging whether the installation path of the service on the target physical machine exists on the target physical machine, if so, extracting a keyword, such as/conf/diagnostis.conf or check_clo_active.sh, from the acquisition main key, and obtaining an acquisition result of the index according to the extracted keyword, an acquisition method corresponding to an acquisition method identifier, a threshold judgment symbol and a threshold. If the installation path of the service on the target physical machine does not exist, the acquisition of the index is ended, the next index rule is traversed, and the next index is continuously acquired.
Optionally, according to the keyword, the acquisition method corresponding to the acquisition method identifier, the threshold judgment symbol and the threshold, acquiring an acquisition result of the index includes: calling the acquisition method corresponding to the acquisition method identifier and taking the keyword as a reference, thereby obtaining a return value; and obtaining the acquisition result of the index according to the return value, the threshold judgment symbol and the threshold. Because the index collection script comprises at least one collection method, the collection method corresponding to the collection method identifier in the index collection script can be called according to the collection method identifier, meanwhile, the key word extracted from the collection main key is taken as a reference, so that a return value is obtained, and then a collection result of the index is obtained according to the return value, the threshold value judgment symbol and the threshold value. If the return value meets the judgment condition of the threshold judgment coincidence and the threshold composition, the index is judged to be normal, otherwise, the index is judged to be abnormal.
Optionally, calling the collection method to identify a corresponding collection method and taking the keyword as a reference, thereby obtaining a return value, including: judging whether an acquisition method corresponding to the acquisition method identifier exists in the index acquisition script or not; if yes, calling a collection method corresponding to the collection method identifier and taking the keyword as a reference, so as to obtain a return value; if not, ending the collection of the index, and writing a prompt which does not support the collection of the index into the database. Judging whether an acquisition method corresponding to the acquisition method identifier exists in the index acquisition script or not in the process of calling the acquisition method, if so, calling the acquisition method corresponding to the acquisition method identifier, and taking keywords extracted from the acquisition main key as input parameters of the acquisition method, thereby obtaining a return value; if the index is not acquired, the acquisition of the index is ended, and a prompt which does not support the acquisition of the index is written into a database so as to be convenient for operation and maintenance personnel to inquire, and an acquisition method corresponding to the index is written into the index acquisition script in the follow-up process, so that a universal index acquisition script is provided for the acquisition of the index of the multi-cluster basic inspection.
And traversing each index rule in the index configuration table to complete the result acquisition of all indexes.
And 105, writing the acquisition result of each index into the database.
After the acquisition result of the index is obtained, the acquisition result of the index is written into a database. In the embodiment of the invention, the acquisition result of one index can be written into the database every time the acquisition of the result of the index is completed, or the acquisition results of all indexes can be written into the database together after the acquisition of the result of all indexes is completed.
Optionally, after the result collection of all the indexes is completed, the index collection script is automatically deleted.
In the embodiment of the invention, the acquisition of the newly added index is completed by modifying the index configuration table, and by modifying any index rule in the index configuration table, for example, a threshold judgment symbol, a threshold, an acquisition method identifier, an acquisition main key and the like in the index rule can be adjusted without modifying an index acquisition script, so that the risk introduced by frequently modifying the index acquisition script is avoided. Moreover, when some indexes are abandoned or the system risk is acquired and introduced under extreme conditions, the indexes can be closed in time by modifying the index configuration table, so that the production risk is avoided.
According to the various embodiments described above, it can be seen that, in the embodiment of the present invention, by obtaining an index configuration table from a database, where the index configuration table includes at least one index rule, each index rule includes an index name, a threshold value judgment symbol, a threshold value, an acquisition method identifier, and an acquisition primary key, the acquisition primary key includes an installation path and a keyword that are served on a physical machine, each index rule is traversed, and for each index rule, a technical means of obtaining an acquisition result of an index according to the acquisition primary key, the acquisition method identifier, the threshold value judgment symbol, and the threshold value is provided, so that technical problems of continuous modification and version release of a routing inspection script caused by changes introduced in cluster new construction, cluster upgrade, and cluster operation and maintenance in the prior art are solved. According to the embodiment of the invention, the standardized index item configuration and encapsulation index collection method is abstracted, so that an index collection script is concise and easy to maintain, and the collection logic is more robust; and the modification operation of the index configuration table is adopted to flexibly cope with the change of the production cluster, the modification frequency of the index acquisition script is reduced, the risk introduction is reduced, and the lengthy version release process is avoided.
Fig. 3 is a flowchart of an index collection method according to a reference embodiment of the present invention. As yet another embodiment of the present invention, as shown in fig. 3, steps 103 to 105 may include the steps of:
Step 301, an index configuration table is obtained from a database.
The index configuration table comprises at least one index rule, each index rule comprises an index name, a threshold judgment symbol, a threshold, an acquisition method identifier and an acquisition main key, and the acquisition main key comprises an installation path and keywords of a service on a physical machine. Optionally, each index rule may further include an index identifier, an index primary classification, an index secondary classification, and an index tertiary classification. Optionally, the keywords include a process keyword or a database retrieval keyword; each index rule further comprises an index switch, an index grade and an index classification.
Step 302, judging whether an index rule which is not traversed exists; if not, go to step 307; if yes, go to step 303.
After obtaining the index configuration table from the database, traversing each index rule in the index configuration table, and executing steps 303-306 for each index rule, thereby obtaining the acquisition result of each index.
Step 303, reading the index rule.
Step 304, extracting the installation path of the service on the physical machine from the acquisition main key, and judging whether the installation path of the service on the physical machine exists on the target physical machine; if yes, go to step 305; if not, go to step 302.
And 305, extracting keywords from the acquisition main key, calling an acquisition method identifier corresponding to the acquisition method, and taking the keywords as input parameters to obtain a return value.
And 306, obtaining the acquisition result of the index according to the return value, the threshold judgment symbol and the threshold.
And step 307, writing the acquisition result of each index into the database.
According to the invention, index collection is carried out in a mode of traversing the index configuration table, so that the current state maintenance of index collection is facilitated, and when index collection is not precisely matched due to the fact that other product processes are introduced into the cluster, the index collection can be directly adjusted by modifying the configuration table; when the cluster needs to newly increase the acquisition index item, if the related acquisition method exists in the script, the index acquisition can be newly increased in a mode of newly increasing the configuration row through the configuration table; when some collected indexes are abandoned or the system risk is introduced by collection under extreme conditions, indexes can be closed in time by modifying an index configuration table, so that the production risk is avoided.
In addition, the specific implementation of the index collection method according to one embodiment of the present invention is described in detail in the above-mentioned index collection method, and thus the description thereof will not be repeated here.
Fig. 4 is a flowchart of an index collection method according to another exemplary embodiment of the present invention. As another embodiment of the present invention, as shown in fig. 4, steps 103 to 105 may include the steps of:
step 401, obtaining an index configuration table from a database.
The index configuration table comprises at least one index rule, each index rule comprises an index name, a threshold judgment symbol, a threshold, an acquisition method identifier and an acquisition main key, and the acquisition main key comprises an installation path and keywords of a service on a physical machine.
Step 402, judging whether an index rule which is not traversed exists; if not, ending; if yes, go to step 403.
In step 403, the index rule is read.
Step 404, extracting the installation path of the service on the physical machine from the acquisition main key, and judging whether the installation path of the service on the physical machine exists on the target physical machine; if yes, go to step 405; if not, step 402 is performed.
Step 405, extracting a keyword from the acquisition primary key, and judging whether an acquisition method corresponding to the acquisition method identifier exists in the index acquisition script; if yes, go to step 406; if not, go to step 409.
Step 406, calling the acquisition method to identify the corresponding acquisition method and taking the keywords as the input parameters, thereby obtaining a return value.
Step 407, obtaining the acquisition result of the index according to the return value, the threshold judgment symbol and the threshold.
And step 408, writing the acquisition result of the index into the database.
And 409, ending the collection of the index, and writing a prompt which does not support the collection of the index into the database.
According to the invention, index collection is carried out in a mode of traversing the index configuration table, so that the current state maintenance of index collection is facilitated, and when index collection is not precisely matched due to the fact that other product processes are introduced into the cluster, the index collection can be directly adjusted by modifying the configuration table; when the cluster needs to newly increase the collection index item, if the related collection method is already in the script, the index collection can be newly increased by a mode of newly increasing the configuration row in the configuration table, and if the related collection method is not in the script, the collection method can be newly increased by a mode of modifying the index collection script; when some collected indexes are abandoned or the system risk is introduced by collection under extreme conditions, indexes can be closed in time by modifying an index configuration table, so that the production risk is avoided.
In addition, in another embodiment of the present invention, the specific implementation of the index collection method is already described in detail in the above-mentioned index collection method, so that the description is not repeated here.
Fig. 5 is a schematic diagram of an index collection device according to an embodiment of the present invention. As shown in fig. 5, the index collection device 500 is disposed on a target physical machine, and includes a receiving module 501 and a collection module 502; the receiving module 501 is configured to receive an index collection script pushed by the operation and maintenance platform, and store the index collection script under a temporary path; wherein the index collection script comprises at least one collection method; the collection module 502 is configured to run the index collection script to perform the following steps: acquiring an index configuration table from a database, wherein the index configuration table comprises at least one index rule, each index rule comprises an index name, a threshold judgment symbol, a threshold, an acquisition method identifier and an acquisition main key, and the acquisition main key comprises an installation path and a keyword of a service on a physical machine; traversing each index rule, and for each index rule, obtaining an acquisition result of the index according to the acquisition main key, the acquisition method corresponding to the acquisition method identifier, the threshold judgment symbol and the threshold; and writing the acquisition result of each index into the database.
Optionally, the acquisition module 502 is further configured to:
for each index rule, extracting an installation path of the service on the physical machine from the acquisition main key, and judging whether the installation path of the service on the physical machine exists on the target physical machine or not;
if yes, extracting a keyword from the acquisition main key, and acquiring an acquisition result of the index according to the keyword, an acquisition method corresponding to the acquisition method identifier, the threshold judgment symbol and the threshold;
if not, ending the collection of the index.
Optionally, the acquisition module 502 is further configured to:
calling the acquisition method corresponding to the acquisition method identifier and taking the keyword as a reference, thereby obtaining a return value;
and obtaining the acquisition result of the index according to the return value, the threshold judgment symbol and the threshold.
Optionally, the acquisition module 502 is further configured to:
judging whether an acquisition method corresponding to the acquisition method identifier exists in the index acquisition script or not;
if yes, calling a collection method corresponding to the collection method identifier and taking the keyword as a reference, so as to obtain a return value;
if not, ending the collection of the index, and writing a prompt which does not support the collection of the index into the database.
Optionally, the keywords include a process keyword or a database retrieval keyword; each index rule further comprises an index switch, an index grade and an index classification.
Optionally, the acquisition module 502 is further configured to:
and after the acquisition results of the indexes are written into the database, deleting the index acquisition script and the index configuration table.
The specific implementation of the index collection device of the present invention is described in detail in the above-mentioned index collection method, and thus the description thereof will not be repeated here.
Fig. 6 illustrates an exemplary system architecture 600 in which an index collection method or index collection device of an embodiment of the present invention may be applied.
As shown in fig. 6, the system architecture 600 may include terminal devices 601, 602, 603, a network 604, and a server 605. The network 604 is used as a medium to provide communication links between the terminal devices 601, 602, 603 and the server 605. The network 604 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
A user may interact with the server 605 via the network 604 using the terminal devices 601, 602, 603 to receive or send messages, etc. Various communication client applications such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only) may be installed on the terminal devices 601, 602, 603.
The terminal devices 601, 602, 603 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 605 may be a server providing various services, such as a background management server (by way of example only) providing support for shopping-type websites browsed by users using terminal devices 601, 602, 603. The background management server can analyze and other data such as the received article information inquiry request and feed back the processing result to the terminal equipment.
It should be noted that, the index collection method provided by the embodiment of the present invention is generally executed by the server 605, and accordingly, the index collection device is generally disposed in the server 605. The index collection method provided by the embodiment of the invention can also be executed by the terminal equipment 601, 602 and 603, and correspondingly, the index collection device can be arranged in the terminal equipment 601, 602 and 603.
It should be understood that the number of terminal devices, networks and servers in fig. 6 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 7, there is illustrated a schematic diagram of a computer system 700 suitable for use in implementing an embodiment of the present invention. The terminal device shown in fig. 7 is only an example, and should not impose any limitation on the functions and the scope of use of the embodiment of the present invention.
As shown in fig. 7, the computer system 700 includes a Central Processing Unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. In the RAM703, various programs and data required for the operation of the system 700 are also stored. The CPU 701, ROM 702, and RAM703 are connected to each other through a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
The following components are connected to the I/O interface 705: an input section 706 including a keyboard, a mouse, and the like; an output portion 707 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 708 including a hard disk or the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. The drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read therefrom is mounted into the storage section 708 as necessary.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 709, and/or installed from the removable medium 711. The above-described functions defined in the system of the present invention are performed when the computer program is executed by a Central Processing Unit (CPU) 701.
The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer programs according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules involved in the embodiments of the present invention may be implemented in software or in hardware. The described modules may also be provided in a processor, for example, as: a processor comprises a receiving module and an acquisition module, wherein the names of these modules do not constitute a limitation of the module itself in some cases.
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by a device, implement the method of: receiving an index collection script pushed by an operation and maintenance platform, and storing the index collection script under a temporary path; wherein the index collection script comprises at least one collection method; and running the index collection script to execute the following steps: acquiring an index configuration table from a database, wherein the index configuration table comprises at least one index rule, each index rule comprises an index name, a threshold judgment symbol, a threshold, an acquisition method identifier and an acquisition main key, and the acquisition main key comprises an installation path and a keyword of a service on a physical machine; traversing each index rule, and for each index rule, obtaining an acquisition result of the index according to the acquisition main key, the acquisition method corresponding to the acquisition method identifier, the threshold judgment symbol and the threshold; and writing the acquisition result of each index into the database.
As a further aspect, embodiments of the present invention also provide a computer program product comprising a computer program which, when executed by a processor, implements the method according to any of the above embodiments.
According to the technical scheme of the embodiment of the invention, the index configuration table is obtained from the database, and comprises at least one index rule, wherein each index rule comprises an index name, a threshold judgment symbol, a threshold, an acquisition method identifier and an acquisition main key, the acquisition main key comprises an installation path and a keyword which serve on a physical machine, each index rule is traversed, and the technical means of acquiring an acquisition result of an index according to the acquisition main key, the acquisition method identifier, the threshold judgment symbol and the threshold is achieved for each index rule, so that the technical problems that in the prior art, the inspection script needs to be continuously modified and released due to changes caused by cluster new construction, cluster upgrading and cluster operation and maintenance are solved. According to the embodiment of the invention, the standardized index item configuration and encapsulation index collection method is abstracted, so that an index collection script is concise and easy to maintain, and the collection logic is more robust; and the modification operation of the index configuration table is adopted to flexibly cope with the change of the production cluster, the modification frequency of the index acquisition script is reduced, the risk introduction is reduced, and the lengthy version release process is avoided.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (15)
1. The index collection method is characterized by being applied to a target physical machine and comprising the following steps of:
receiving an index collection script pushed by an operation and maintenance platform, and storing the index collection script under a temporary path; wherein the index collection script comprises at least one collection method;
and running the index collection script to execute the following steps:
acquiring an index configuration table from a database, wherein the index configuration table comprises at least one index rule, each index rule comprises an index name, a threshold judgment symbol, a threshold, an acquisition method identifier and an acquisition main key, and the acquisition main key comprises an installation path and a keyword of a service on a physical machine;
traversing each index rule, and for each index rule, obtaining an acquisition result of the index according to the acquisition main key, the acquisition method corresponding to the acquisition method identifier, the threshold judgment symbol and the threshold;
And writing the acquisition result of each index into the database.
2. The method according to claim 1, wherein for each index rule, obtaining the acquisition result of the index according to the acquisition primary key, the acquisition method corresponding to the acquisition method identifier, the threshold judgment symbol, and the threshold comprises:
for each index rule, extracting an installation path of the service on the physical machine from the acquisition main key, and judging whether the installation path of the service on the physical machine exists on the target physical machine or not;
if yes, extracting a keyword from the acquisition main key, and acquiring an acquisition result of the index according to the keyword, an acquisition method corresponding to the acquisition method identifier, the threshold judgment symbol and the threshold;
if not, ending the collection of the index.
3. The method of claim 2, wherein obtaining the collection result of the indicator according to the keyword, the collection method corresponding to the collection method identifier, the threshold judgment symbol, and the threshold comprises:
calling the acquisition method corresponding to the acquisition method identifier and taking the keyword as a reference, thereby obtaining a return value;
And obtaining the acquisition result of the index according to the return value, the threshold judgment symbol and the threshold.
4. A method according to claim 3, wherein invoking the acquisition method to identify the corresponding acquisition method and taking the key as an entry to obtain a return value comprises:
judging whether an acquisition method corresponding to the acquisition method identifier exists in the index acquisition script or not;
if yes, calling a collection method corresponding to the collection method identifier and taking the keyword as a reference, so as to obtain a return value;
if not, ending the collection of the index, and writing a prompt which does not support the collection of the index into the database.
5. A method according to claim 3, wherein the key comprises a process key or a database retrieval key; each index rule further comprises an index switch, an index grade and an index classification.
6. The method according to any one of claims 1-5, further comprising, after writing the acquisition result of each of the metrics to the database:
and deleting the index collection script and the index configuration table.
7. An index acquisition device, characterized in that, set up in target physical machine, include:
The receiving module is used for receiving the index acquisition script pushed by the operation and maintenance platform and storing the index acquisition script under the temporary path; wherein the index collection script comprises at least one collection method;
the acquisition module is used for running the index acquisition script to execute the following steps:
acquiring an index configuration table from a database, wherein the index configuration table comprises at least one index rule, each index rule comprises an index name, a threshold judgment symbol, a threshold, an acquisition method identifier and an acquisition main key, and the acquisition main key comprises an installation path and a keyword of a service on a physical machine;
traversing each index rule, and for each index rule, obtaining an acquisition result of the index according to the acquisition main key, the acquisition method corresponding to the acquisition method identifier, the threshold judgment symbol and the threshold;
and writing the acquisition result of each index into the database.
8. The apparatus of claim 7, wherein the acquisition module is further configured to:
for each index rule, extracting an installation path of the service on the physical machine from the acquisition main key, and judging whether the installation path of the service on the physical machine exists on the target physical machine or not;
If yes, extracting a keyword from the acquisition main key, and acquiring an acquisition result of the index according to the keyword, an acquisition method corresponding to the acquisition method identifier, the threshold judgment symbol and the threshold;
if not, ending the collection of the index.
9. The apparatus of claim 8, wherein the acquisition module is further configured to:
calling the acquisition method corresponding to the acquisition method identifier and taking the keyword as a reference, thereby obtaining a return value;
and obtaining the acquisition result of the index according to the return value, the threshold judgment symbol and the threshold.
10. The apparatus of claim 9, wherein the acquisition module is further configured to:
judging whether an acquisition method corresponding to the acquisition method identifier exists in the index acquisition script or not;
if yes, calling a collection method corresponding to the collection method identifier and taking the keyword as a reference, so as to obtain a return value;
if not, ending the collection of the index, and writing a prompt which does not support the collection of the index into the database.
11. The apparatus of claim 9, wherein the key comprises a process key or a database retrieval key; each index rule further comprises an index switch, an index grade and an index classification.
12. The apparatus of any one of claims 7-11, wherein the acquisition module is further configured to:
and after the acquisition results of the indexes are written into the database, deleting the index acquisition script and the index configuration table.
13. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs,
the one or more processors implement the method of any of claims 1-6 when the one or more programs are executed by the one or more processors.
14. A computer readable medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-6.
15. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311118461.4A CN117149568A (en) | 2023-08-31 | 2023-08-31 | Index collection method, device, electronic equipment and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311118461.4A CN117149568A (en) | 2023-08-31 | 2023-08-31 | Index collection method, device, electronic equipment and computer readable medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117149568A true CN117149568A (en) | 2023-12-01 |
Family
ID=88898269
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311118461.4A Pending CN117149568A (en) | 2023-08-31 | 2023-08-31 | Index collection method, device, electronic equipment and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117149568A (en) |
-
2023
- 2023-08-31 CN CN202311118461.4A patent/CN117149568A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106156186B (en) | Data model management device, server and data processing method | |
| CN109257200B (en) | Method and device for monitoring big data platform | |
| US10523580B2 (en) | Automatic cloud provisioning based on related internet news and social network trends | |
| CN111190888A (en) | Method and device for managing graph database cluster | |
| CN109901918B (en) | Method and device for processing overtime task | |
| CN110874307A (en) | Event buried point collecting and reporting method and device | |
| CN111190892A (en) | Method and device for processing abnormal data in data backfilling | |
| CN113076253A (en) | Test method and test device | |
| CN113760677A (en) | Abnormal link analysis method, device, equipment and storage medium | |
| US10331484B2 (en) | Distributed data platform resource allocator | |
| CN113377626A (en) | Visual unified alarm method, device, equipment and medium based on service tree | |
| CN110795331A (en) | Software testing method and device | |
| CN112445860B (en) | Method and device for processing distributed transaction | |
| CN106777265B (en) | Service data processing method and device | |
| CN112825165A (en) | Project quality management method and device | |
| CN112445691B (en) | Non-invasive intelligent contract performance detection method and device | |
| CN112860538B (en) | Method and device for carrying out interface regression test based on online log | |
| CN116450622B (en) | Method, apparatus, device and computer readable medium for data warehouse entry | |
| CN110866031B (en) | Database access path optimization method and device, computing equipment and medium | |
| JP7305641B2 (en) | Methods and systems for tracking application activity data from remote devices and generating corrective behavior data structures for remote devices | |
| CN117149568A (en) | Index collection method, device, electronic equipment and computer readable medium | |
| CN115202973A (en) | Application running state determining method and device, electronic equipment and medium | |
| CN112182080A (en) | Data integration system and data processing method based on data integration system | |
| CN112463616A (en) | Chaos testing method and device for Kubernetes container platform | |
| CN113254325A (en) | Test case processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |