CN111371647A - Data center monitoring data preprocessing method and device - Google Patents
Data center monitoring data preprocessing method and device Download PDFInfo
- Publication number
- CN111371647A CN111371647A CN202010136517.9A CN202010136517A CN111371647A CN 111371647 A CN111371647 A CN 111371647A CN 202010136517 A CN202010136517 A CN 202010136517A CN 111371647 A CN111371647 A CN 111371647A
- Authority
- CN
- China
- Prior art keywords
- value
- data center
- monitoring
- data
- monitoring data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/04—Processing captured monitoring data, e.g. for logfile generation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/16—Threshold monitoring
Abstract
The embodiment of the invention provides a method and a device for preprocessing monitoring data of a data center, wherein the method comprises the following steps: detecting an illegal value acquisition error of a data center infrastructure management platform; correcting illegal value acquisition errors of a data center infrastructure management platform; and detecting and pushing the dead value acquisition error of the data center infrastructure management platform. According to the method and the device for preprocessing the monitoring data of the data center, provided by the embodiment of the invention, the illegal value acquisition error and the dead value acquisition error of the platform are processed, so that the interference of the monitoring data acquisition error to the abnormal operation state of the platform detection equipment is avoided, the generation of error alarm information is reduced, the reliability of the monitoring data of the data center is improved, and the operation and maintenance efficiency of the equipment is improved.
Description
Technical Field
The invention relates to the technical field of data mining, in particular to a method and a device for preprocessing monitoring data of a data center.
Background
With the rapid development of the information technology industry, the data center serving as an industrial infrastructure is also rapidly expanding in size.
Data Center Infrastructure Management platform (DCIM): in order to ensure high reliability of the data center, all-around monitoring of the internal devices of the data center is required. In the early stage, a manual inspection mode is adopted for monitoring a data center, and a large amount of manpower is used for performing inspection monitoring on key equipment of the data center, but the mode is not only low in efficiency, but also error detection, omission and the like are easily caused due to the difference of professional qualities of inspection personnel. With the expansion of the scale of the data center, the traditional manual inspection mode cannot adapt to the requirements of data center monitoring and efficient management, and along with the continuous improvement of the intelligence degree of the data center, the concept of a data center infrastructure management platform is provided. The method aims to monitor the states of power environment equipment and IT equipment in a data center by means of a whole set of hardware facilities, sensors, software tools and the like, and realize the collection and aggregation of all monitoring data through a unified platform.
Although the data center infrastructure management platform can monitor the real-time state of the data center equipment and acquire the monitoring index data of all the equipment in the system, the acquired value of the monitoring index is inconsistent with the actual value thereof due to sensor faults, network transmission errors, abnormal data storage and the like, which is called as the acquisition error of the monitoring index. These acquisition errors may cause interference to the abnormal operation state of the platform detection device, easily cause the platform to generate a large amount of error alarm information, and reduce the operation and maintenance efficiency of the device, and the data center infrastructure management platform lacks a mechanism for identifying and filtering the acquisition errors of the device monitoring indexes, so that the reliability verification of the acquired data is lacked.
Disclosure of Invention
The embodiment of the invention provides a data center monitoring data preprocessing method and device, which are used for solving the technical problems in the prior art.
In order to solve the foregoing technical problem, in one aspect, an embodiment of the present invention provides a data center monitoring data preprocessing method, including:
detecting an illegal value acquisition error of a data center infrastructure management platform;
correcting illegal value acquisition errors of a data center infrastructure management platform;
and detecting and pushing the dead value acquisition error of the data center infrastructure management platform.
Further, the detecting the illegal value acquisition error of the data center infrastructure management platform specifically includes:
acquiring a monitoring data value of a target monitoring index of a data center in a current time window;
calculating the fluctuation value of the monitoring data value at the current moment;
if the fluctuation value exceeds the fluctuation threshold value of the target monitoring index, comparing the monitoring data value at the current moment with a preset upper limit value and a preset lower limit value respectively;
and if the monitoring data value at the current moment is larger than the upper limit value or smaller than the lower limit value, marking the monitoring data value at the current moment as an illegal value acquisition error.
Further, the fluctuation threshold is determined according to the following formula:
wherein the content of the first and second substances,
to monitor the fluctuation threshold of the index j at the current time τ, β is a coefficient, median represents the median statistic,
to monitor the monitored data value of the indicator j at the current time tau,
t is the length of the time window for monitoring the monitoring data value of the index j at the moment tau-1.
Further, the correcting the illegal value acquisition error of the data center infrastructure management platform specifically includes:
acquiring a monitoring data value of a time window in which an illegal value acquisition error is positioned;
correcting error values by using median values of all monitoring data values in the time window;
and storing the corrected data value into a database.
Further, after storing the modified data value into the database, the method further includes:
reporting the illegal value acquisition error record for confirming by operation and maintenance personnel;
and adjusting the upper and lower limit values of the illegal value acquisition error judgment according to the confirmation result of the operation and maintenance personnel.
Further, the detecting and pushing the data center infrastructure management platform dead value acquisition error specifically includes:
acquiring a monitoring data value of a target monitoring index of a data center in a current time window;
if the monitoring data values of the current time window are all the same, calculating a dead value sensitivity value of the target monitoring index at the current moment, wherein the dead value sensitivity value is used for judging whether the target monitoring index has dead value collection errors or not;
and if the sensitivity value of the dead value is greater than or equal to the preset threshold value, determining that the target monitoring index in the current time window has a dead value acquisition error.
Further, the dead-value sensitivity value is calculated by the following formula:
wherein the content of the first and second substances,
indicating a dead value sensitive value of the monitoring index j after i acquisition time intervals;
a monitoring data value representing a monitoring index j at the current moment tau;
presentation monitoringThe monitored data value of the index j at the time of tau-1, num (delta) represents the change times of the monitored index j in i acquisition time intervals, αjIndicating the accuracy of the monitoring index j.
On the other hand, an embodiment of the present invention provides a data center monitoring data preprocessing apparatus, including:
the first detection module is used for detecting illegal value acquisition errors of the data center infrastructure management platform;
the correction module is used for correcting the illegal value acquisition errors of the data center infrastructure management platform;
and the second detection module is used for detecting and pushing the dead value acquisition error of the data center infrastructure management platform.
In another aspect, an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method provided by the first aspect when executing the computer program.
In yet another aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the method provided in the first aspect.
According to the method and the device for preprocessing the monitoring data of the data center, provided by the embodiment of the invention, the illegal value detection error and the dead value detection error of the platform are processed, so that the interference of the monitoring data acquisition error to the abnormal operation state of the platform detection equipment is avoided, the generation of error alarm information is reduced, the reliability of the monitoring data of the data center is improved, and the operation and maintenance efficiency of the equipment is improved.
Drawings
Fig. 1 is a flowchart of a data center monitoring data preprocessing method according to an embodiment of the present invention;
FIG. 2 is a flowchart of a method for detecting an illegal value acquisition error according to an embodiment of the present invention;
FIG. 3 is a flowchart of a method for detecting a dead value collection error according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a data center monitoring data preprocessing apparatus according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a flowchart of a data center monitoring data preprocessing method according to an embodiment of the present invention, and as shown in fig. 1, an execution subject of the data center monitoring data preprocessing method according to the embodiment of the present invention is a data center monitoring data preprocessing device. The method comprises the following steps:
and S101, detecting illegal value acquisition errors of the data center infrastructure management platform.
Specifically, the improper value collection for the data center infrastructure management platform is detected incorrectly: and carrying out illegal value acquisition and error detection on real-time monitoring data of the power environment equipment and the IT equipment acquired by the data center infrastructure management platform to obtain a monitoring index set of the platform with the wrong illegal value acquisition at the current moment. The illegal value is a value beyond the normal range, for example, the normal range of the temperature value of the data center monitored by the temperature sensor is-10 ℃ and 40 ℃, and if the infrastructure management platform collects a temperature value of 1000 ℃, the illegal value is obtained.
And S102, correcting illegal value acquisition errors of the data center infrastructure management platform.
Specifically, the data center infrastructure management platform illegal value collection is incorrectly corrected: and correcting the illegal values of the monitoring indexes according to the obtained monitoring index set with wrong illegal value acquisition to obtain the corrected values of the monitoring indexes, and storing the corrected values into a database.
And S103, detecting and pushing dead value acquisition errors of the data center infrastructure management platform.
Specifically, dead-value collection is erroneously detected and pushed for a data center infrastructure management platform: and carrying out platform dead value acquisition error detection on real-time monitoring data acquired by the data center infrastructure management platform, acquiring a monitoring index set of the platform with dead value acquisition errors at the current moment, and automatically pushing dead value acquisition error records to data center operation and maintenance personnel. The dead value refers to the condition that a plurality of times of continuously acquired data are abnormally kept unchanged and is distinguished from the condition that the monitoring index acquisition value is kept unchanged in a normal state. The dead value collection error may be caused by sensor failure, network abnormality, data storage error and the like.
According to the data center monitoring data preprocessing method provided by the embodiment of the invention, by processing the illegal value acquisition error and the dead value acquisition error of the platform, the interference of the monitoring data acquisition error to the abnormal operation state of the platform detection equipment is avoided, the generation of error alarm information is reduced, the reliability of the data center monitoring data is improved, and the operation and maintenance efficiency of the equipment is improved.
Based on any one of the above embodiments, further, the detecting an illegal value collection error of a data center infrastructure management platform specifically includes:
acquiring a monitoring data value of a target monitoring index of a data center in a current time window;
calculating the fluctuation value of the monitoring data value at the current moment;
if the fluctuation value exceeds the fluctuation threshold value of the target monitoring index, comparing the monitoring data value at the current moment with a preset upper limit value and a preset lower limit value respectively;
and if the monitoring data value at the current moment is larger than the upper limit value or smaller than the lower limit value, marking the monitoring data value at the current moment as an illegal value acquisition error.
Specifically, fig. 2 is a flowchart of a method for detecting an illegal value acquisition error according to an embodiment of the present invention, and as shown in fig. 2, in the embodiment of the present invention, the method for detecting an illegal value acquisition error of a data center infrastructure management platform specifically includes the following steps:
firstly, platform acquisition data of a time window T of each monitoring index at the current time tau is acquired from a database for storing real-time acquisition data of a data center infrastructure management platform, such as
Wherein the content of the first and second substances,
a monitoring data value representing the monitoring index j at the current time τ,
the monitoring data value of the monitoring index j at the time tau-i is obtained.
For example: the database for collecting data in real time by the platform is a non-relational database SSDB, the database is connected by an open source tool kit SSDB of Python, data of each monitoring index in a current time window is obtained, and the length of the time window needs to be set by operation and maintenance personnel.
Then, the fluctuation amount of the collected value at the current moment of each monitoring index is calculated
For example: the acquisition value of the platform at the current moment about a certain monitoring index is 40.8, the acquisition value at the last moment is 50.3, and the fluctuation amount of the acquisition value at the current moment is 0.5 through calculation.
Then, a fluctuation threshold value of each monitoring index at the current time is calculated
For example, a fluctuating threshold
The calculation formula of (a) is as follows:
wherein, the
A fluctuation threshold value of a monitoring index j acquisition value at the current moment tau, β is a coefficient which needs to be set by operation and maintenance personnel and reflects the tolerance degree of the operation and maintenance personnel to the fluctuation of the monitoring index acquisition value, median represents median statistic,
for the monitoring index collection value of the monitoring index j at the current moment tau,
t is the length of the time window for monitoring the monitoring data value of the index j at the moment tau-1.
For example, in this embodiment, the set time window length is T equal to 5, the fluctuation threshold calculation coefficient β is equal to 3, and if the collected value of the platform with respect to a certain monitoring index in the current time window is [50.2,50.1,49.7,50.3,49.8], the fluctuation threshold at the current time is calculated to be 1.35.
Then, the fluctuation amount of each monitoring index at the current time is compared
Fluctuation threshold from the current time
If it is not
The upper and lower limit values U stored in the platform and used for judging the illegal value of the monitoring indexj,Lj。
If the value is collected
Or
And marking the collection value of the monitoring index at the moment as an illegal value collection error by default, and storing the illegal value collection error into an illegal value collection error database in the platform. Preferably, the database is selected as a MySQL database, the illegal value collection error records are stored according to month tables, and the field information includes: the current timestamp, the number of the monitoring index, the number of the moving ring equipment corresponding to the monitoring index and the current time acquisition value.
Finally, collecting error records of the illegal values obtained in the last step, and automatically reporting the error records to operation and maintenance personnel through a socket technology; if the operation and maintenance personnel confirm that the index is legal, the system automatically updates the upper and lower limit values U judged by the illegal value of the monitoring index j stored in the platformj,Lj. Wherein, if
The upper limit value of the platform about the monitoring index is updated
On the contrary, if
The lower limit value of the platform about the monitoring index is updated
Considering that the operation and maintenance personnel cannot confirm the illegal acquisition error record reported by the platform in time, the whole method is realized in a non-blocking mode, and the front end calls the record from the illegal acquisition error database and presents the record to the operation and maintenance personnel while being isolated from the real-time detection program of the rear end.
According to the data center monitoring data preprocessing method provided by the embodiment of the invention, the illegal value detection error and the dead value detection error of the platform are processed, so that the interference of the monitoring data acquisition error to the abnormal operation state of the platform detection equipment is avoided, the generation of error alarm information is reduced, the reliability of the data center monitoring data is improved, and the operation and maintenance efficiency of the equipment is improved.
Based on any one of the above embodiments, further, the fluctuation threshold is determined according to the following formula:
wherein the content of the first and second substances,
to monitor the fluctuation threshold of the index j at the current time τ, β is a coefficient, median represents the median statistic,
to monitor the monitored data value of the indicator j at the current time tau,
t is the length of the time window for monitoring the monitoring data value of the index j at the moment tau-1.
Specifically, in the embodiment of the present invention, the fluctuation threshold is determined according to the following formula:
wherein the content of the first and second substances,
to monitor the fluctuation threshold of the index j at the current time τ, β is a coefficient, median represents the median statistic,
to monitor the monitored data value of the indicator j at the current time tau,
t is the length of the time window for monitoring the monitoring data value of the index j at the moment tau-1.
According to the data center monitoring data preprocessing method provided by the embodiment of the invention, the illegal value detection error and the dead value detection error of the platform are processed, so that the interference of the monitoring data acquisition error to the abnormal operation state of the platform detection equipment is avoided, the generation of error alarm information is reduced, the reliability of the data center monitoring data is improved, and the operation and maintenance efficiency of the equipment is improved.
Based on any one of the above embodiments, further, the correcting the error of collecting the illegal value of the data center infrastructure management platform specifically includes:
acquiring a monitoring data value of a time window in which an illegal value acquisition error is positioned;
correcting error values by using median values of all monitoring data values in the time window;
and storing the corrected data value into a database.
Specifically, in the embodiment of the present invention, if an illegal value acquisition error is detected, the illegal value acquisition error of the data center infrastructure management platform needs to be corrected, and the specific steps are as follows:
firstly, according to the obtained monitoring index set J with corresponding illegal value acquisition errors of the platform at the current time, obtaining platform acquisition data of a time window T of each monitoring index J (J ∈ J) at the current time tau
Then, calculating the median of the collected values of the platform in the time window T as the illegal values collected by the platform
And (7) correcting.
And finally, storing the corrected data into a database. For example, the database for collecting data in real time by the storage platform is a non-relational database SSDB, and the modified data is inserted into the SSDB database using the open source toolkit SSDB of Python.
According to the data center monitoring data preprocessing method provided by the embodiment of the invention, the illegal value detection error and the dead value detection error of the platform are processed, so that the interference of the monitoring data acquisition error to the abnormal operation state of the platform detection equipment is avoided, the generation of error alarm information is reduced, the reliability of the data center monitoring data is improved, and the operation and maintenance efficiency of the equipment is improved.
Based on any one of the above embodiments, further, after storing the modified data value into the database, the method further includes:
reporting the illegal value acquisition error record for confirming by operation and maintenance personnel;
and adjusting the upper and lower limit values of the illegal value acquisition error judgment according to the confirmation result of the operation and maintenance personnel.
Specifically, as shown in fig. 2, in the embodiment of the present invention, after storing the modified data value into the database, the method further includes:
reporting the illegal value acquisition error record for confirming by operation and maintenance personnel;
and adjusting the upper and lower limit values of the illegal value acquisition error judgment according to the confirmation result of the operation and maintenance personnel.
According to the embodiment of the invention, the median of the monitoring index acquisition value in the time window is used as the threshold value for the illegal value detection method, so that the statistical attribute is more stable, and the abnormal value or other illegal values in the same window can be accurately judged without being influenced. In addition, the illegal value detection method adopts a feedback mechanism, the knowledge of operation and maintenance personnel is added to guide the judgment of the illegal value, the upper limit value and the lower limit value of the optimal monitoring index can be learned along with the continuous feedback process, and the dependence on the operation and maintenance personnel is gradually eliminated.
According to the data center monitoring data preprocessing method provided by the embodiment of the invention, the illegal value detection error and the dead value detection error of the platform are processed, so that the interference of the monitoring data acquisition error to the abnormal operation state of the platform detection equipment is avoided, the generation of error alarm information is reduced, the reliability of the data center monitoring data is improved, and the operation and maintenance efficiency of the equipment is improved.
Based on any one of the above embodiments, further, the detecting and pushing the data center infrastructure management platform dead value acquisition error specifically includes:
acquiring a monitoring data value of a target monitoring index of a data center in a current time window;
if the monitoring data values of the current time window are all the same, calculating a dead value sensitivity value of the target monitoring index at the current moment, wherein the dead value sensitivity value is used for judging whether the target monitoring index has dead value collection errors or not;
and if the sensitivity value of the dead value is greater than or equal to the preset threshold value, determining that the target monitoring index in the current time window has a dead value acquisition error.
Specifically, fig. 3 is a flowchart of a method for detecting a dead value collection error according to an embodiment of the present invention, and as shown in fig. 3, in the embodiment of the present invention, specific steps of detecting and pushing the dead value collection error of a data center infrastructure management platform include:
firstly, platform acquisition data of each monitoring index in a current time window is acquired from a database for storing data acquired by a data center infrastructure management platform in real time.
Then, a dead value sensitive value record stored in a database in the last time interval of the monitoring index is obtained, for example, the used database is a MySQL database, and for each monitoring index in the platform, the stored field information comprises the change times num (delta) of the monitoring index, the collection time interval number i, and the precision α of the monitoring indexjMonitoring the collected value of the index
Obtaining a dead value sensitive value record of the previous moment by using a Python open source kit pymysql, and calculating a dead value sensitive value of the monitoring index at the current moment based on the acquisition value of the current moment
And inserting the dead value sensitive value record of the current time moment into the MySQL database through the open source toolkit pymysql.
Then, detecting whether the acquisition value is constant or not in the current time window T aiming at each monitoring index; if so, the time window continues to slide into the future; otherwise, calculating the sensitivity value of the dead value of the monitoring index
And will be
With a set threshold value rhothresAnd (6) comparing. The dead-value sensitivity is used for judging whether a dead-value collection error occurs in the target monitoring index or not, and automatically distinguishing the dead-value collection error from a system set value, a state attribute monitoring value, a collection value in a normal state and the like.
If it is not
And the time window continues to slide towards the future, otherwise, the dead value acquisition error record is stored in a dead value acquisition error database in the platform and is pushed to operation and maintenance personnel through a socket technology, so that the operation and maintenance personnel can conveniently troubleshoot the problems of sensor failure, network transmission, data storage and the like, and the data acquisition function of the platform can be repaired in time.
Preferably, the database is selected as a MySQL database, the dead value collection error records are stored according to month sub-tables, and the field information includes: the current timestamp, the number of the monitoring index, the number of the moving ring equipment corresponding to the monitoring index and the current time acquisition value.
In the embodiment of the invention, the dead value detection method automatically distinguishes dead value acquisition errors from system set values, state attribute monitoring values, acquisition values in a normal state and the like by defining the dead value sensitive indexes, improves the universality of the method, and is suitable for different monitoring indexes acquired by a platform.
According to the data center monitoring data preprocessing method provided by the embodiment of the invention, the illegal value detection error and the dead value detection error of the platform are processed, so that the interference of the monitoring data acquisition error to the abnormal operation state of the platform detection equipment is avoided, the generation of error alarm information is reduced, the reliability of the data center monitoring data is improved, and the operation and maintenance efficiency of the equipment is improved.
Based on any one of the above embodiments, further, the death value sensitivity value is calculated by the following formula:
wherein the content of the first and second substances,
indicating a dead value sensitive value of the monitoring index j after i acquisition time intervals;
a monitoring data value representing a monitoring index j at the current moment tau;
representing the monitoring data value of the monitoring index j at the moment tau-1, num (delta) representing the change times of the monitoring index j in i acquisition time intervals, αjIndicating the accuracy of the monitoring index j.
Specifically, in the embodiment of the present invention, the defined death value sensitivity value is calculated by the following formula:
wherein the content of the first and second substances,
indicating a dead value sensitive value of the monitoring index j after i acquisition time intervals;
a monitoring data value representing a monitoring index j at the current moment tau;
representing the monitoring data value of the monitoring index j at the moment tau-1, num (delta) representing the change times of the monitoring index j in i acquisition time intervals, αjIndicating the accuracy of the monitoring index j. .
According to the data center monitoring data preprocessing method provided by the embodiment of the invention, the illegal value detection error and the dead value detection error of the platform are processed, so that the interference of the monitoring data acquisition error to the abnormal operation state of the platform detection equipment is avoided, the generation of error alarm information is reduced, the reliability of the data center monitoring data is improved, and the operation and maintenance efficiency of the equipment is improved.
Based on any of the above embodiments, fig. 4 is a schematic diagram of a data center monitoring data preprocessing device provided in an embodiment of the present invention, and as shown in fig. 4, an embodiment of the present invention provides a data center monitoring data preprocessing device, which includes a first detection module 401, a block modification module 402, and a second detection module 403, where:
the first detection module 401 is used for detecting an illegal value acquisition error of a data center infrastructure management platform; the correcting module 402 is used for correcting the illegal value acquisition errors of the data center infrastructure management platform; the second detection module 403 is configured to detect and push a dead-value collection error of the data center infrastructure management platform.
Embodiments of the present invention provide a data center monitoring data preprocessing apparatus, configured to execute the method described in any of the above embodiments, where specific steps of executing the method described in one of the above embodiments by using the apparatus provided in this embodiment are the same as those in the corresponding embodiment, and are not described herein again.
According to the data center monitoring data preprocessing device provided by the embodiment of the invention, the illegal value detection error and the dead value detection error of the platform are processed, so that the interference of the monitoring data acquisition error to the abnormal operation state of the platform detection equipment is avoided, the generation of error alarm information is reduced, the reliability of the data center monitoring data is improved, and the operation and maintenance efficiency of the equipment is improved.
Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, and as shown in fig. 5, the electronic device includes: a processor (processor)501, a communication Interface (Communications Interface)502, a memory (memory)503, and a communication bus 504, wherein the processor 501, the communication Interface 502, and the memory 503 are configured to communicate with each other via the communication bus 504. The processor 501 and the memory 502 communicate with each other via a bus 503. The processor 501 may call logic instructions in the memory 503 to perform the following method:
detecting an illegal value acquisition error of a data center infrastructure management platform;
correcting illegal value acquisition errors of a data center infrastructure management platform;
and detecting and pushing the dead value acquisition error of the data center infrastructure management platform.
In addition, the logic instructions in the memory may be implemented in the form of software functional units and may be stored in a computer readable storage medium when sold or used as a stand-alone product. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Further, embodiments of the present invention provide a computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the steps of the above-described method embodiments, for example, including:
detecting an illegal value acquisition error of a data center infrastructure management platform;
correcting illegal value acquisition errors of a data center infrastructure management platform;
and detecting and pushing the dead value acquisition error of the data center infrastructure management platform.
Further, an embodiment of the present invention provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the steps in the above method embodiments, for example, including:
detecting an illegal value acquisition error of a data center infrastructure management platform;
correcting illegal value acquisition errors of a data center infrastructure management platform;
and detecting and pushing the dead value acquisition error of the data center infrastructure management platform.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A data center monitoring data preprocessing method is characterized by comprising the following steps:
detecting an illegal value acquisition error of a data center infrastructure management platform;
correcting illegal value acquisition errors of a data center infrastructure management platform;
and detecting and pushing the dead value acquisition error of the data center infrastructure management platform.
2. The data center monitoring data preprocessing method according to claim 1, wherein the detecting of the data center infrastructure management platform illegal value acquisition error specifically comprises:
acquiring a monitoring data value of a target monitoring index of a data center in a current time window;
calculating the fluctuation value of the monitoring data value at the current moment;
if the fluctuation value exceeds the fluctuation threshold value of the target monitoring index, comparing the monitoring data value at the current moment with a preset upper limit value and a preset lower limit value respectively;
and if the monitoring data value at the current moment is larger than the upper limit value or smaller than the lower limit value, marking the monitoring data value at the current moment as an illegal value acquisition error.
3. The data center monitoring data preprocessing method of claim 2, wherein the fluctuation threshold is determined according to the following formula:
wherein the content of the first and second substances,
to monitor the fluctuation threshold of the index j at the current time τ, β is a coefficient, median represents the median statistic,
to monitor the monitored data value of the indicator j at the current time tau,
t is the length of the time window for monitoring the monitoring data value of the index j at the moment tau-1.
4. The data center monitoring data preprocessing method according to claim 1, wherein the correcting the data center infrastructure management platform illegal value acquisition error specifically comprises:
acquiring a monitoring data value of a time window in which an illegal value acquisition error is positioned;
correcting error values by using median values of all monitoring data values in the time window;
and storing the corrected data value into a database.
5. The method for preprocessing the monitoring data of the data center according to claim 4, wherein after storing the corrected data value into the database, the method further comprises:
reporting the illegal value acquisition error record for confirming by operation and maintenance personnel;
and adjusting the upper and lower limit values of the illegal value acquisition error judgment according to the confirmation result of the operation and maintenance personnel.
6. The data center monitoring data preprocessing method according to claim 1, wherein the detecting and pushing of the data center infrastructure management platform dead-value collection error specifically comprises:
acquiring a monitoring data value of a target monitoring index of a data center in a current time window;
if the monitoring data values of the current time window are all the same, calculating a dead value sensitivity value of the target monitoring index at the current moment, wherein the dead value sensitivity value is used for judging whether the target monitoring index has dead value collection errors or not;
and if the sensitivity value of the dead value is greater than or equal to the preset threshold value, determining that the target monitoring index in the current time window has a dead value acquisition error.
7. The data center monitoring data preprocessing method of claim 6, wherein the dead-value sensitivity value is calculated by the following formula:
wherein the content of the first and second substances,
indicating a dead value sensitive value of the monitoring index j after i acquisition time intervals;
a monitoring data value representing a monitoring index j at the current moment tau;
representing the monitoring data value of the monitoring index j at the time of tau-1; num (. DELTA.) tableIndicating the number of changes of the monitoring indicator j in the i acquisition time intervals αjIndicating the accuracy of the monitoring index j.
8. A data center monitoring data preprocessing device is characterized by comprising:
the first detection module is used for detecting illegal value acquisition errors of the data center infrastructure management platform;
the correction module is used for correcting the illegal value acquisition errors of the data center infrastructure management platform;
and the second detection module is used for detecting and pushing the dead value acquisition error of the data center infrastructure management platform.
9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the data center monitoring data preprocessing method according to any one of claims 1 to 7 when executing the computer program.
10. A non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the data center monitoring data preprocessing method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010136517.9A CN111371647A (en) | 2020-03-02 | 2020-03-02 | Data center monitoring data preprocessing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010136517.9A CN111371647A (en) | 2020-03-02 | 2020-03-02 | Data center monitoring data preprocessing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111371647A true CN111371647A (en) | 2020-07-03 |
Family
ID=71208498
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010136517.9A Pending CN111371647A (en) | 2020-03-02 | 2020-03-02 | Data center monitoring data preprocessing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111371647A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112380065A (en) * | 2020-11-18 | 2021-02-19 | 北京百度网讯科技有限公司 | Data restoration method and device, electronic equipment and storage medium |
| CN113127363A (en) * | 2021-04-23 | 2021-07-16 | 中国工商银行股份有限公司 | Parameter adjusting method, parameter adjusting device, electronic device and readable storage medium |
| CN116663747A (en) * | 2023-07-19 | 2023-08-29 | 广东云下汇金科技有限公司 | Intelligent early warning method and system based on data center infrastructure |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103955202A (en) * | 2014-04-11 | 2014-07-30 | 国家电网公司 | Automatic data diagnosis and identification method based on desulfurization system of coal-fired power plant |
| CN104504237A (en) * | 2014-11-21 | 2015-04-08 | 国家电网公司 | Automatic diagnosis and discrimination model of data for denitrification system of coal-fired power plant |
| CN104765354A (en) * | 2014-01-10 | 2015-07-08 | 北京博锐尚格节能技术股份有限公司 | Fault diagnosis method, device and system for sensors and execution elements |
| CN105425775A (en) * | 2015-12-04 | 2016-03-23 | 河南中烟工业有限责任公司许昌卷烟厂 | Sensor fault automatic judgment method and system |
| CN109491289A (en) * | 2018-11-15 | 2019-03-19 | 国家计算机网络与信息安全管理中心 | A kind of dynamic early-warning method and device for data center's dynamic environment monitoring |
| US20200028728A1 (en) * | 2018-07-23 | 2020-01-23 | International Business Machines Corporation | Cognitive Thermal Cable Holder |
-
2020
- 2020-03-02 CN CN202010136517.9A patent/CN111371647A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104765354A (en) * | 2014-01-10 | 2015-07-08 | 北京博锐尚格节能技术股份有限公司 | Fault diagnosis method, device and system for sensors and execution elements |
| CN103955202A (en) * | 2014-04-11 | 2014-07-30 | 国家电网公司 | Automatic data diagnosis and identification method based on desulfurization system of coal-fired power plant |
| CN104504237A (en) * | 2014-11-21 | 2015-04-08 | 国家电网公司 | Automatic diagnosis and discrimination model of data for denitrification system of coal-fired power plant |
| CN105425775A (en) * | 2015-12-04 | 2016-03-23 | 河南中烟工业有限责任公司许昌卷烟厂 | Sensor fault automatic judgment method and system |
| US20200028728A1 (en) * | 2018-07-23 | 2020-01-23 | International Business Machines Corporation | Cognitive Thermal Cable Holder |
| CN109491289A (en) * | 2018-11-15 | 2019-03-19 | 国家计算机网络与信息安全管理中心 | A kind of dynamic early-warning method and device for data center's dynamic environment monitoring |
Non-Patent Citations (1)
| Title |
|---|
| 周永章 等: "《地球科学大数据挖掘与机器学习》", 31 December 2019 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112380065A (en) * | 2020-11-18 | 2021-02-19 | 北京百度网讯科技有限公司 | Data restoration method and device, electronic equipment and storage medium |
| CN112380065B (en) * | 2020-11-18 | 2023-08-29 | 北京百度网讯科技有限公司 | Data restoration method and device, electronic equipment and storage medium |
| CN113127363A (en) * | 2021-04-23 | 2021-07-16 | 中国工商银行股份有限公司 | Parameter adjusting method, parameter adjusting device, electronic device and readable storage medium |
| CN113127363B (en) * | 2021-04-23 | 2024-02-23 | 中国工商银行股份有限公司 | Parameter adjustment method, parameter adjustment device, electronic device, and readable storage medium |
| CN116663747A (en) * | 2023-07-19 | 2023-08-29 | 广东云下汇金科技有限公司 | Intelligent early warning method and system based on data center infrastructure |
| CN116663747B (en) * | 2023-07-19 | 2024-04-12 | 广东云下汇金科技有限公司 | Intelligent early warning method and system based on data center infrastructure |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111126824B (en) | Multi-index correlation model training method and multi-index anomaly analysis method | |
| CN111371647A (en) | Data center monitoring data preprocessing method and device | |
| CN111461533B (en) | Fault monitoring method and system for industrial production line based on big data | |
| CN110286656B (en) | False alarm filtering method and device for tolerance of error data | |
| CN111637924B (en) | Detection method and detection device for abnormality of excavator and readable storage medium | |
| CN110825798A (en) | Electric power application data maintenance method and device | |
| KR101953558B1 (en) | Apparatus and Method for Fault Management of Smart Devices | |
| CN116502925B (en) | Digital factory equipment inspection evaluation method, system and medium based on big data | |
| CN113934720A (en) | Data cleaning method and equipment and computer storage medium | |
| CN115372816A (en) | Power distribution switchgear operation fault prediction system and method based on data analysis | |
| CN116028887A (en) | Analysis method of continuous industrial production data | |
| CN111814557A (en) | Action flow detection method, device, equipment and storage medium | |
| CN114531618A (en) | Data acquisition method and system for water meter collector, storage medium and intelligent terminal | |
| CN117314020A (en) | Wetland carbon sink data monitoring system of plankton | |
| CN116522096A (en) | Three-dimensional digital twin content intelligent manufacturing method based on motion capture | |
| CN108536777B (en) | Data processing method, server cluster and data processing device | |
| CN111382172A (en) | Power distribution terminal backup power supply multivariate health degree self-checking method and system | |
| CN113036917B (en) | Power distribution network monitoring information monitoring system and method based on machine learning | |
| CN115774159A (en) | Fault detection system for power unit of high-voltage frequency converter | |
| CN115270982A (en) | Switch cabinet fault prediction method based on multi-data neural network | |
| CN115511374A (en) | Method, device and equipment for calculating correlation of process indexes and storage medium | |
| CN115184808A (en) | Battery thermal runaway risk detection method, device, equipment and computer storage medium | |
| CN115346164A (en) | Automatic model reconstruction method and system for component recognition model | |
| US20210397992A1 (en) | Inference apparatus, information processing apparatus, inference method, program and recording medium | |
| CN112416896A (en) | Data abnormity warning method and device, storage medium and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200703 |