CN114881540B - Method and device for determining water source treatment scheme, electronic equipment and storage medium - Google Patents

Method and device for determining water source treatment scheme, electronic equipment and storage medium Download PDF

Info

Publication number
CN114881540B
CN114881540B CN202210777656.9A CN202210777656A CN114881540B CN 114881540 B CN114881540 B CN 114881540B CN 202210777656 A CN202210777656 A CN 202210777656A CN 114881540 B CN114881540 B CN 114881540B
Authority
CN
China
Prior art keywords
target
monitoring data
machine learning
water source
learning algorithm
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210777656.9A
Other languages
Chinese (zh)
Other versions
CN114881540A (en
Inventor
戈燕红
黄辉勤
舒少君
王俊杰
马东晓
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Yingfeng Technology Co ltd
Original Assignee
Guangdong Yingfeng Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Yingfeng Technology Co ltd filed Critical Guangdong Yingfeng Technology Co ltd
Priority to CN202210777656.9A priority Critical patent/CN114881540B/en
Publication of CN114881540A publication Critical patent/CN114881540A/en
Application granted granted Critical
Publication of CN114881540B publication Critical patent/CN114881540B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063—Operations research, analysis or management
    • G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
    • G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/10—Pre-processing; Data cleansing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/20—Ensemble learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/06—Energy or water supply
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A20/00—Water conservation; Efficient water supply; Efficient water use
    • Y02A20/152—Water filtration

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • Human Resources & Organizations (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Economics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Strategic Management (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Software Systems (AREA)
  • Educational Administration (AREA)
  • Evolutionary Biology (AREA)
  • Marketing (AREA)
  • Development Economics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Tourism & Hospitality (AREA)
  • General Business, Economics & Management (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Primary Health Care (AREA)
  • Water Supply & Treatment (AREA)
  • Public Health (AREA)
  • Game Theory and Decision Science (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Testing And Monitoring For Control Systems (AREA)

Abstract

The application relates to the technical field of water source treatment, and particularly provides a method and a device for determining a water source treatment scheme, electronic equipment and a storage medium. The method comprises the following steps: acquiring original monitoring data acquired after water quality monitoring is carried out on a target water source by target water quality monitoring equipment; preprocessing original monitoring data to obtain target monitoring data; calibrating the target monitoring data through each of at least two machine learning algorithms to obtain calibrated data after the target monitoring data are calibrated; obtaining accurate monitoring data used for indicating an accurate water quality monitoring result of the target water source based on all the calibrated data; determining target pollution information of a target water source according to the accurate monitoring data; and determining a target water source treatment scheme based on the pollution indexes indicated in the target pollution information. The method and the device can solve the problem that the applicability of the water source treatment scheme determined in the prior art is poor.

Description

Method and device for determining water source treatment scheme, electronic equipment and storage medium
Technical Field
The application relates to the technical field of water source treatment, in particular to a method and a device for determining a water source treatment scheme, electronic equipment and a storage medium.
Background
With the increasing attention of people on environmental protection, water sources have great influence on the production and life of people, so that the requirements on water quality monitoring and water quality treatment are more and more increased.
In the related art, the precision of the miniature water quality environment monitoring equipment cannot be guaranteed, the high-precision monitor cannot be distributed widely, and the equipment may break down after long-term use, so that the monitoring data drift. In order to overcome the problems, a calibration system is needed to calibrate the monitoring data, but the calibration system in the related art may have errors due to incomplete feature extraction, and because most of the existing calibration methods are based on a machine learning mode, and for the same feature data set, different machine learning algorithms may also generate calibration results with different accuracies, thereby causing low applicability of the determined water source treatment scheme.
Therefore, the method for determining the water source treatment scheme in the related technology has the problem that the determined water source treatment scheme is low in applicability because different machine learning algorithms can generate calibration results with different accuracies for the same characteristic data set.
Disclosure of Invention
The application provides a method and a device for determining a water source treatment scheme, electronic equipment and a storage medium, which are used for solving the problem that the determined water source treatment scheme is low in applicability due to the fact that different machine learning algorithms can generate calibration results with different accuracies for the same characteristic data set in the related technology.
According to an aspect of an embodiment of the present application, there is provided a method for determining a water source treatment plan, including:
acquiring original monitoring data acquired after water quality monitoring is carried out on a target water source by target water quality monitoring equipment, wherein the accuracy of the target water quality monitoring equipment in water quality monitoring is lower than a preset lower limit of an accurate value;
preprocessing the original monitoring data to obtain target monitoring data;
calibrating the target monitoring data through each of at least two machine learning algorithms to obtain calibrated data after the target monitoring data are calibrated, wherein the accuracy of the target monitoring data calibrated by any two machine learning algorithms in the at least two machine learning algorithms is different from each other;
obtaining accurate monitoring data used for indicating an accurate water quality monitoring result of the target water source based on all the calibrated data;
determining target pollution information of the target water source according to the accurate monitoring data;
and determining a target water source treatment scheme based on pollution indexes indicated in the target pollution information, wherein each target pollution information comprises at least one pollution index, and the target water source treatment scheme comprises target treatment equipment and/or target treatment agents for treating pollutants corresponding to each pollution index.
Optionally, as in the foregoing method, before the calibrating the target monitoring data by each of at least two machine learning algorithms to obtain calibrated data after calibrating the target monitoring data, the method further includes:
step S1, according to the initial weight corresponding to each machine learning algorithm, weighting the deviation between the initial calibration data corresponding to each machine learning algorithm with respect to the initial accurate monitoring data to obtain an initial weighted deviation corresponding to each machine learning algorithm, and accumulating all the initial weighted deviations to obtain an initial verification result, wherein the initial calibration data corresponding to each machine learning algorithm is obtained by calibrating the target monitoring data by each machine learning algorithm, and the initial accurate monitoring data is determined according to the initial calibration data;
step S2, for each machine learning algorithm, adjusting the initial weight corresponding to each machine learning algorithm by a preset weight adjusting method to obtain a candidate weight, weighting all the preliminary calibration data according to the candidate weight corresponding to each machine learning algorithm to obtain a first weighting result corresponding to each machine learning algorithm, and accumulating all the first weighting results to obtain adjusted accurate monitoring data;
step S3, according to the candidate weight corresponding to each machine learning algorithm, weighting the deviation of the preliminary calibration data corresponding to each machine learning algorithm relative to the adjusted accurate monitoring data to obtain a candidate weighted deviation corresponding to each machine learning algorithm, and accumulating all the candidate weighted deviations to obtain a current verification result;
step S4, under the condition that the preset convergence requirement is not met between the current verification result and the historical verification result, for each machine learning algorithm, the candidate weight corresponding to each machine learning algorithm is adjusted through a preset weight adjusting method to obtain an adjusted weight, all the preliminary calibration data are weighted according to the adjusted weight corresponding to each machine learning algorithm to obtain a second weighting result corresponding to each machine learning algorithm, all the second weighting results are accumulated to obtain current accurate monitoring data, the candidate weight in the step S3 is updated through the adjusted weight, the adjusted accurate monitoring data in the step S3 are updated through the current accurate monitoring data, the step S3 is skipped to, and the process is repeated, until the current verification result and the historical verification result meet the preset convergence requirement, wherein the historical verification result at least comprises a verification result obtained at the previous time of obtaining the current verification result;
step S5, when the preset convergence requirement is satisfied between the current verification result and the historical verification result, determining that all the candidate weights satisfy a preset accuracy requirement, and determining the candidate weight corresponding to each machine learning algorithm as the target weight corresponding to each machine learning algorithm.
Optionally, as in the foregoing method, the calibrating the target monitoring data by each of at least two machine learning algorithms to obtain calibrated data after calibrating the target monitoring data includes:
for any one machine learning algorithm, inputting the target monitoring data into the machine learning algorithm to obtain primary calibrated subdata obtained by calibrating the target monitoring data by the machine learning algorithm;
and weighting all the primary calibrated subdata according to the target weight corresponding to each machine learning algorithm to obtain a target weighting result corresponding to each machine learning algorithm, and adding all the target weighting results to obtain the calibrated data.
Optionally, as in the foregoing method, the preprocessing the raw monitoring data to obtain target monitoring data includes:
obtaining the preprocessed monitoring data by performing the following operations on the original monitoring data: performing first anomaly check on the original monitoring data, and removing outliers under the condition that the outliers exist in the original monitoring data; performing second anomaly check on the original monitoring data to determine that missing parameters exist in the original monitoring data, and predicting to obtain a predicted value of the missing parameters of a first specified characteristic type under the condition that all the missing parameters include the missing parameters of the first specified characteristic type, wherein the missing parameters are characteristic parameters without parameter values, the first specified characteristic type is one of all the characteristic types of the target monitoring data, and in a preset time period, the number of the missing parameters corresponding to the first specified characteristic type is less than or equal to a preset number; performing second anomaly check on the original monitoring data to determine that missing parameters exist in the original monitoring data, and removing all parameters corresponding to a second feature type under the condition that all the missing parameters include the missing parameters of the second specified feature type, wherein the second specified feature type is one of all the feature types of the target monitoring data, and the number of the missing parameters corresponding to the second specified feature type is larger than the preset number within a preset time period;
and performing feature extraction on the preprocessed monitoring data to obtain the target monitoring data.
Optionally, as in the foregoing method, before the performing feature extraction on the preprocessed monitoring data to obtain the target monitoring data, the method further includes:
determining the characteristic type in monitoring data acquired by the target water quality monitoring equipment after the target water quality monitoring equipment monitors the water quality of the candidate water source;
determining a target characteristic type in all the characteristic types by performing characteristic correlation analysis between the characteristic types and the real water quality of the candidate water source, wherein the correlation between the parameter value of the target characteristic type and the real water quality of the candidate water source meets the preset correlation requirement;
and determining a new characteristic type in all the characteristic types by performing characteristic correlation analysis between the characteristic types and the real water quality of the candidate water source, wherein the new characteristic type is obtained according to a characteristic parameter corresponding to at least one characteristic type, and the correlation between the parameter value of the new characteristic type and the real water quality of the candidate water source meets the preset correlation requirement.
Optionally, as in the foregoing method, the performing feature extraction on the preprocessed monitoring data to obtain the target monitoring data includes:
extracting a first target parameter corresponding to the target feature type from the preprocessed monitoring data, and taking the first target parameter corresponding to the target feature type as first sub-target monitoring data;
extracting the existing feature type corresponding to the new feature type from the preprocessed monitoring data; constructing the characteristic parameters corresponding to the existing characteristic types to obtain the characteristic parameters corresponding to the new characteristic types, and obtaining second sub-target monitoring data;
and obtaining the target monitoring data based on the first sub-target monitoring data and the second sub-target monitoring data.
Optionally, as in the foregoing method, before the obtaining of the raw monitoring data collected after the target water source is monitored by the target water quality monitoring device for water quality, the method further includes:
determining target water source attribute information corresponding to the target water source, wherein the target water source attribute information is used for indicating the target geographical attribute and the water source use of the target water source;
obtaining appointed water source attribute information with the highest matching degree with the target water source attribute information through matching in all candidate water source attribute information, and taking candidate water quality monitoring equipment corresponding to the appointed water source attribute information as the target water quality monitoring equipment, wherein each candidate water source attribute information is preset with corresponding candidate water quality monitoring equipment, and for each candidate water source attribute information, the characteristic correlation between the characteristic type of the parameter acquired by the candidate water quality monitoring equipment corresponding to the candidate water source attribute information and the water quality of a water source indicated by the candidate water source attribute information meets the preset correlation requirement.
According to another aspect of the embodiments of the present application, there is also provided an apparatus for determining a water source treatment plan, including:
the system comprises an acquisition module, a data acquisition module and a data processing module, wherein the acquisition module is used for acquiring original monitoring data acquired after water quality monitoring is carried out on a target water source by target water quality monitoring equipment, and the accuracy of the water quality monitoring carried out by the target water quality monitoring equipment is lower than the lower limit of a preset accurate value;
the preprocessing module is used for preprocessing the original monitoring data to obtain target monitoring data;
the calibration module is used for calibrating the target monitoring data through each of at least two machine learning algorithms to obtain calibrated data after the target monitoring data are calibrated, wherein the accuracy of the calibration of any two machine learning algorithms to the target monitoring data is different from each other;
the obtaining module is used for obtaining accurate monitoring data used for indicating an accurate water quality monitoring result of the target water source based on all the calibrated data;
the target pollution information determining module is used for determining the target pollution information of the target water source according to the accurate monitoring data;
and the scheme determining module is used for determining a target water source treatment scheme based on the pollution indexes indicated in the target pollution information, wherein each target pollution information comprises at least one pollution index, and the target water source treatment scheme comprises target treatment equipment and/or target treatment agents for treating pollutants corresponding to each pollution index.
According to another aspect of the embodiments of the present application, there is also provided an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory communicate with each other through the communication bus; wherein the memory is used for storing the computer program; a processor for performing the method steps in any of the above embodiments by running the computer program stored on the memory.
According to a further aspect of the embodiments of the present application, there is also provided a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to perform the method steps of any of the above embodiments when the computer program is executed.
In the embodiment of the application, the target monitoring data is calibrated through each of at least two machine learning algorithms to obtain the calibrated data after the target monitoring data is calibrated, and the accurate monitoring data is obtained based on the calibrated data, so that the purpose of determining the accurate monitoring data by synthesizing the results obtained by each machine learning algorithm is achieved, the accuracy of the accurate monitoring data is improved, and the problem that the applicability of the determined water source management scheme is low due to the fact that different machine learning algorithms can generate calibration results with different accuracies for the same characteristic data set in the method for determining the water source management scheme in the related technology can be solved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a schematic flow diagram of an alternative method for determining a water source remediation program according to an embodiment of the present application;
FIG. 2 is a flowchart illustrating an alternative implementation of step S103 shown in FIG. 1;
FIG. 3 is a schematic flow diagram of an alternative method of determining a water source remediation program according to an application example of the present application;
FIG. 4 is a block diagram of an alternative apparatus for determining a water source remediation scheme according to an embodiment of the present application;
fig. 5 is a block diagram of an alternative electronic device according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Moreover, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
According to one aspect of an embodiment of the present application, a method of determining a water source remediation program is provided. Alternatively, in this embodiment, the method for determining the water source treatment plan may be applied to a hardware environment formed by a terminal and a server. The server is connected with the terminal through a network, and can be used for providing services (such as data analysis push service, data storage service and the like) for the terminal or a client installed on the terminal, and a database can be arranged on the server or independent of the server and is used for providing data storage service for the server.
The network may include, but is not limited to, at least one of: wired networks, wireless networks. The wired network may include, but is not limited to, at least one of: wide area networks, metropolitan area networks, local area networks, which may include, but are not limited to, at least one of the following: WIFI (Wireless Fidelity), bluetooth. The terminal may not be limited to a PC, a mobile phone, a tablet computer, and the like.
The method for determining the water source treatment scheme in the embodiment of the application can be executed by the server, the terminal or both. The method for determining the water source treatment scheme by the terminal according to the embodiment of the application can also be executed by a client installed on the terminal.
Taking the method for determining a water source treatment plan in this embodiment executed by a server as an example, fig. 1 is a method for determining a water source treatment plan provided in this embodiment of the present application, including the following steps:
step S101, acquiring original monitoring data acquired after water quality monitoring is carried out on a target water source by target water quality monitoring equipment, wherein the accuracy of the target water quality monitoring equipment in water quality monitoring is lower than a preset lower limit of an accurate value;
the method for determining a water source treatment scheme in this embodiment can be applied to a scene that needs to calibrate monitored raw monitoring data, for example: the method comprises the following steps of calibrating original monitoring data of a drinking water source, calibrating original monitoring data of a water source of an aquaculture area and the like, and can also be used for calibrating original monitoring data of other types of water sources; for other types of water sources, the above-described method of determining a water source remediation scheme is equally applicable without contradiction.
Optionally, a water quality monitoring micro station may be set as the target water quality monitoring device, so as to capture the water quality data of the target water source in real time through the water quality monitoring micro station, and transmit the monitored original monitoring data to a server (e.g., a background data processing system), so as to perform subsequent operations such as exception handling and feature extraction. Further, the target water quality monitoring device may be used to monitor one or more water quality parameters, including but not limited to: PH, conductivity, dissolved oxygen, turbidity, temperature. Thus, the target water quality monitoring devices may include, but are not limited to: PH monitoring sensor, conductivity sensor, water oxygen monitoring sensor, turbidity monitoring facilities and temperature sensor.
After the target water quality monitoring equipment monitors the water quality components of the target water source in real time and obtains original monitoring data, the original monitoring data can be sent to the server for realizing the method of the embodiment in a wired or wireless communication mode.
The accuracy of the target water quality monitoring equipment for water quality monitoring is lower than the lower limit of the preset accurate value, and can be as follows: the target water quality monitoring equipment cannot ensure the precision (for example, the monitoring data drift is caused because the precision of the equipment is low or the equipment possibly fails in long-term use); due to the fact that high-precision monitoring equipment is expensive and cannot be widely distributed, monitored data cannot represent the overall situation of a target water source; the number of target water quality monitoring devices arranged in the target water source is too small, and the like.
The lower limit of the preset accurate value may be selected according to the accuracy requirement of water quality monitoring, for example, an error between a monitoring value of the target water quality monitoring device for performing water quality monitoring on any water source and an accurate monitoring value of the water source is less than 5% or 10% (or other values).
Further, the target water quality monitoring device itself may have a certain data analysis capability, for example, whether the original monitoring data is abnormal or not may be obtained based on the analysis of the original monitoring data, when the difference between the original monitoring data obtained by the analysis and Q pieces of historical original monitoring data whose collection time is closest is greater than or equal to the minimum difference, the transmission of the original monitoring data to the server is stopped, and when the difference between the original monitoring data obtained by the analysis and Q pieces of historical original monitoring data whose collection time is closest is less than the minimum difference, the original monitoring data is transmitted to the server.
And S102, preprocessing the original monitoring data to obtain target monitoring data.
After receiving the original monitoring data, the server may perform preprocessing on the original monitoring data, so that the target monitoring data obtained based on the original monitoring data is calibrated to obtain calibrated data only when the original monitoring data meets the requirement of the next calibration.
Alternatively, the pretreatment may include, but is not limited to: data cleansing, data supplementation, etc. The data cleansing may be the deletion of data that is significantly erroneous and the data supplementation may be the supplementation of missing data.
Step S103, calibrating the target monitoring data through each of at least two machine learning algorithms to obtain calibrated data after the target monitoring data are calibrated, wherein the accuracy of the target monitoring data calibration performed by any two machine learning algorithms in the at least two machine learning algorithms is different from each other.
After the target monitoring data is obtained, the accuracy of the target water quality monitoring equipment for water quality monitoring is lower than the lower limit of the preset accurate value; therefore, the original monitoring data itself has a certain error, and the target monitoring data is only preprocessed by the method related to step S102, so that the error of the original monitoring data itself cannot be calibrated.
In the step, at least two machine learning algorithms are set, so that each machine learning algorithm can calibrate the target monitoring data; wherein the at least two machine learning algorithms include, but are not limited to: linear regression, decision tree, random forest, gradient lifting tree, XGBoost. And before each machine learning algorithm is used, the original machine learning algorithm can be trained so as to obtain the machine learning algorithm after the original machine learning algorithm meets the preset precision requirement.
Due to incomplete feature extraction and the fault problem of the data acquisition equipment, the calibration result of the machine learning algorithm has certain deviation, and different machine learning algorithms have different implementation principles and different emphasis (for example, random forests are focused on reducing the variance of the prediction result, and gradient lifting trees are focused on reducing the residual error of the prediction result). Therefore, the accuracy of the calibration result obtained by calibrating the target monitoring data by each machine learning algorithm is also inconsistent.
For example, when at least two machine learning algorithms exist: the method comprises the steps of performing linear regression on target monitoring data to obtain a first calibration result, performing decision tree processing on the target monitoring data to obtain a second calibration result, performing random forest processing on the target monitoring data to obtain a third calibration result, performing gradient lifting tree processing on the target monitoring data to obtain a fourth calibration result, performing XGboost processing on the target monitoring data to obtain a fifth calibration result, and obtaining calibrated data based on the first calibration result, the second calibration result, the third calibration result, the fourth calibration result and the fifth calibration result.
And step S104, obtaining accurate monitoring data for indicating the accurate water quality monitoring result of the target water source based on all the calibrated data.
After the calibrated data are obtained, the calibrated data can be used as accurate monitoring data, and then the accurate monitoring data can be used for indicating an accurate monitoring result of the water quality of the target water source.
And step S105, determining target pollution information of the target water source according to the accurate monitoring data.
After the accurate monitoring data is determined, the unqualified index in the target water source can be determined based on the accurate monitoring data, and the unqualified index is used as the target pollution information.
For example: the indexes which do not meet the requirements comprise overhigh turbidity and over-low PH value; the determined target pollution information of the target water source can be as follows: (turbidity is too high, pH is too low).
Step S106, determining a target water source treatment scheme based on pollution indexes indicated in target pollution information, wherein each target pollution information comprises at least one pollution index, and the target water source treatment scheme comprises target treatment equipment and/or target treatment agents for treating pollutants corresponding to each pollution index.
After the target pollution information is determined, the target water source needs to be treated according to the target pollution information.
In this embodiment, the corresponding relationship between different candidate pollution information and candidate water source treatment schemes may be predetermined, so after the target pollution information is determined, the specified pollution information that is the same as the target pollution information may be determined from all candidate pollution information, and then the candidate water source treatment scheme corresponding to the specified pollution information is used as the target water source treatment scheme.
In addition, it is also possible to set in advance a target abatement device and/or a target abatement agent corresponding to an unsatisfactory index corresponding to each candidate pollution index. And then after the pollution indexes are obtained, a target water source treatment scheme can be obtained by determining target treatment equipment and/or target treatment agents for treating the pollutants corresponding to each pollution index.
According to the method in the embodiment, the target monitoring data is calibrated through each of at least two machine learning algorithms, the calibrated data after the target monitoring data is calibrated is obtained, the accurate monitoring data is obtained based on the calibrated data, the purpose that the accurate monitoring data is determined by synthesizing results obtained by the machine learning algorithms is achieved, the accuracy of the accurate monitoring data is improved, and the problem that the applicability of the determined water source management scheme is low due to the fact that different machine learning algorithms can generate calibration results with different accuracies for the same characteristic data set in the method for determining the water source management scheme in the related technology can be solved.
As an alternative embodiment, as in the foregoing method, before the step S103 calibrates the target monitoring data through each of at least two machine learning algorithms, and obtains calibrated data calibrated by the target monitoring data, the method further includes:
and step S1, according to the initial weight corresponding to each machine learning algorithm, weighting the deviation of the initial calibration data corresponding to each machine learning algorithm relative to the initial accurate monitoring data to obtain the initial weighted deviation corresponding to each machine learning algorithm, and accumulating all the initial weighted deviations to obtain an initial verification result, wherein the initial calibration data corresponding to each machine learning algorithm is obtained after each machine learning algorithm calibrates the target monitoring data, and the initial accurate monitoring data is determined according to the initial calibration data.
After determining each machine learning algorithm, the weight values of each machine learning algorithm may be initialized to obtain an initial weight corresponding to each machine learning algorithm, for example, when at least two machine learning algorithms exist: in linear regression, decision tree, random forest, gradient lifting tree, XGBoost, one of the initial weight setting methods may be: the initial weight of the linear regression is 0.2, the initial weight of the decision tree is 0.2, the initial weight of the random forest is 0.2, the initial weight of the gradient lifting tree is 0.2, and the initial weight of the XGboost is 0.2.
Optionally, the machine learning algorithm can be obtained after the machine learning algorithm to be trained is trained in advance; after all the machine learning algorithms are obtained, the target monitoring data can be preliminarily calibrated through each machine learning algorithm, and preliminary calibration data corresponding to each machine learning algorithm can be obtained; after all the preliminary calibration data are obtained, the initial accurate monitoring data can be obtained by processing (e.g., calculating an average value, weighting according to a candidate weight, weighting according to any preset weight, etc.) all the preliminary calibration data.
After the initial calibration data corresponding to each machine learning algorithm is obtained and the accurate monitoring data is initialized, the deviation between the initial calibration data corresponding to each machine learning algorithm and the accurate monitoring data can be calculated; thus, a bias corresponding to each machine learning algorithm can be obtained.
Further, the deviation corresponding to each machine learning algorithm may be weighted based on the initial weight corresponding to each machine learning algorithm to obtain an initial weighted deviation corresponding to each machine learning algorithm.
And finally, accumulating all initial weighted deviations to obtain an initial verification result.
For example: the machine learning algorithm comprises: linear regression, decision tree, random forest, gradient lifting tree and XGboost, wherein the initial weight of the linear regression is 0.2, the initial weight of the decision tree is 0.1, the initial weight of the random forest is 0.2, the initial weight of the gradient lifting tree is 0.3, the initial weight of the XGboost is 0.2, and the initial calibration data corresponding to the linear regression is a 1 The preliminary calibration data corresponding to the decision tree is a 2 The preliminary calibration data corresponding to the random forest is a 3 The preliminary calibration data corresponding to the gradient lifting tree is a 4 The preliminary calibration data corresponding to XGboost is a 5 Initializing accurate monitoring data as A; the weighted deviation candidates corresponding to the linear regression are d (A, a) 1 ) And 5, the candidate weighted deviation corresponding to the decision tree is d (A, a) 2 ) And 10, the candidate weighted deviation corresponding to the random forest is d (A, a) 3 ) And 5, the candidate weighted deviation corresponding to the gradient lifting tree is 3d (A, a) 4 ) The candidate weighting deviation corresponding to the XGboost is d (A, a) 5 )/5. Wherein d (-) is a function for calculating the deviation, d (A, a) t ) Preliminary calibration data a for indicating that A corresponds to a machine learning algorithm t t A deviation between A and a t The difference between A and a t The absolute value of the difference between. The initial verification result is: d (A, a) 1 )/5+d(A，a 2 )/10+d(A，a 3 )/5+3d(A，a 4 )/10+d(A，a 5 )/5。
And step S2, for each machine learning algorithm, adjusting the initial weight corresponding to each machine learning algorithm by a preset weight adjusting method to obtain a candidate weight, weighting all the preliminary calibration data according to the candidate weight corresponding to each machine learning algorithm to obtain a first weighting result corresponding to each machine learning algorithm, and accumulating all the first weighting results to obtain adjusted accurate monitoring data.
For each machine learning algorithm, the initial weight corresponding to each machine learning algorithm can be adjusted through a preset weight adjusting method to obtain a candidate weight, and then the candidate weight corresponding to each machine learning algorithm can be obtained. Alternatively, the preset weight adjustment method may be a lagrange multiplier method.
And after the new candidate weight is obtained, all the preliminary calibration data can be weighted according to the candidate weight corresponding to each machine learning algorithm to obtain a first weighting result corresponding to each machine learning algorithm, and further, the adjusted accurate monitoring data can be obtained by accumulating all the first weighting results. The specific implementation manner may refer to the method for determining the initialized accurate monitoring data in the foregoing example, and details are not described here.
And step S3, according to the candidate weight corresponding to each machine learning algorithm, weighting the deviation of the initial calibration data corresponding to each machine learning algorithm relative to the adjusted accurate monitoring data to obtain the candidate weighted deviation corresponding to each machine learning algorithm, and accumulating all the candidate weighted deviations to obtain the current verification result.
After steps S1 and S2 have been performed, candidate weights corresponding to each machine learning algorithm and adjusted accurate monitoring data have been generated. Therefore, a verification result can be calculated based on the adjusted accurate monitoring data and the candidate weight corresponding to each machine learning algorithm, and the verification result can be used as a current verification result. Specifically, the manner of calculating the current verification result may refer to the implementation manner of calculating the initial verification result in the foregoing example, and is not described herein again.
Step S4, under the condition that the preset convergence requirement is not satisfied between the current check result and the historical check result, for each machine learning algorithm, adjusting the candidate weight corresponding to each machine learning algorithm by a preset weight adjusting method to obtain an adjusted weight, weighting all the preliminary calibration data according to the adjusted weight corresponding to each machine learning algorithm to obtain a second weighting result corresponding to each machine learning algorithm, accumulating all the second weighting results to obtain current accurate monitoring data, updating the candidate weight in step S3 by the adjusted weight, updating the adjusted accurate monitoring data in step S3 by the current accurate monitoring data, and skipping to step S3, and repeating the steps until the preset convergence requirement is satisfied between the current check result and the historical check result, the historical verification result at least comprises a verification result obtained at the previous time when the current verification result is obtained.
After the current check result is determined, the current check result can be compared with the historical check result to judge whether the preset convergence requirement is met.
The historical verification result at least comprises the verification result obtained at the previous time of obtaining the current verification result, so that when the current verification result is the verification result obtained by the second calculation, the historical verification result is the initial verification result. In addition, the number of the historical verification results may be F, and the F historical verification results are verification results which have an obtaining time earlier than the obtaining time of the current verification result and are closest to the obtaining time of the current verification result.
The preset convergence requirement may be that the deviation between the current verification result and each historical verification result and the specified value is less than or equal to a preset value, for example: when F is 3 and the current verification result is 5.1, all the historical verification results comprise the following steps from near to far according to the acquisition time: when the historical verification result J1 is 4.9, the historical verification result J2 is 4.8, the historical verification result J3 is 5.2, the specified value is 5 and the preset value is 0.2, based on that the deviation between each historical verification result and the specified value and the deviation between the current verification result and the specified value are both less than or equal to 0.2, the preset convergence requirement between the current verification result and the historical verification result is judged to be met; otherwise, when the deviation between one historical verification result and the designated value is greater than 0.2 or the deviation between the current verification result and the designated value is greater than 0.2 in all the historical verification results, it is determined that the preset convergence requirement is not met between the current verification result and the historical verification result.
The preset convergence requirement may also be that an error between the current verification result and the historical verification result is smaller than a preset error value.
Under the condition that the preset convergence requirement is not met between the current verification result and the historical verification result; the candidate weight corresponding to each machine learning algorithm is adjusted by a preset weight adjustment method to obtain an adjusted weight, and then the adjusted weight corresponding to each machine learning algorithm can be obtained. Alternatively, the preset weight adjustment method may be a lagrange multiplier method.
After the adjusted weight corresponding to each machine learning algorithm is obtained, weighting all the preliminary calibration data according to the adjusted weight corresponding to each machine learning algorithm to obtain a second weighting result corresponding to each machine learning algorithm; and after all the second weighting results are determined, the current accurate monitoring data can be obtained by accumulating all the second weighting results. For example, in machine learning algorithms include: linear regression, decision tree, random forest, gradient boostingWhen the XGboost is carried out on the trees and the XGboost, the adjusted weight of the linear regression is 0.1, the adjusted weight of the decision tree is 0.2, the adjusted weight of the random forest is 0.1, the adjusted weight of the gradient lifting tree is 0.2, the adjusted weight of the XGboost is 0.4, and the initial calibration data corresponding to the linear regression is a 1 The preliminary calibration data corresponding to the decision tree is a 2 The preliminary calibration data corresponding to the random forest is a 3 The preliminary calibration data corresponding to the gradient lifting tree is a 4 The preliminary calibration data corresponding to XGboost is a 5 Then a second weighted result corresponding to the linear regression can be obtained as a 1 10, the second weighted result of the decision tree is a 2 And/5, the second weighted result of the random forest is a 3 10, the second weighting result of the gradient lifting tree is a 4 The second weighting result of XGboost is 2a 1 (iii)/5; the current accurate monitoring data is: a is 1 /10+a 2 /5+a 3 /10+a 4 /5+2a 1 /5 。
And then updating the candidate weight in the step S3 through the adjusted weight, updating the adjusted accurate monitoring data in the step S3 through the current accurate monitoring data, and jumping to the step S3, and repeating the steps until the preset convergence requirement is met between the current verification result and the historical verification result.
And step S5, under the condition that the preset convergence requirement is met between the current verification result and the historical verification result, determining that all candidate weights meet the preset accuracy requirement, and determining the candidate weight corresponding to each machine learning algorithm as the target weight corresponding to each machine learning algorithm.
And if the preset convergence requirement is met between the current verification result and the historical verification result, not executing the step S4, directly executing the step S5, determining that all the candidate weights meet the preset accuracy requirement, and determining the candidate weight corresponding to each machine learning algorithm as the target weight corresponding to each machine learning algorithm.
And determining that all the candidate weights meet the preset accurate requirement, namely weighting the preliminary calibration data corresponding to each machine learning algorithm according to the candidate weights corresponding to each machine learning algorithm to obtain a weighting result corresponding to each machine learning algorithm, and accumulating the obtained monitoring data based on each weighting result to accurately reflect the water quality of the target water source.
Furthermore, candidate weights corresponding to each machine learning algorithm can be adjusted through a truth finding technology, final accurate monitoring data is obtained, and optimal parameter values are obtained by minimizing the objective function (1) when the candidate weights are in multiple (for example, multiple candidate weights are obtained through a truth finding technologyNIndividual) monitoring location for water quality monitoring, and the types of the monitored water quality include various types (for exampleMSeed), a specific mathematical expression for determining the verification result is as follows:
Figure 771252DEST_PATH_IMAGE001
（1）
wherein the above expressionf（χ （*） ，W）Is a minimum objective function for calculating the current verification result, whereinWRepresenting the candidate weight for each data source (i.e. the machine learning algorithm),v (*) representing the true value (i.e. the adjusted accurate monitored data),v (k) is indicated from the firstkPreliminary calibration data, χ, for each data source (*) In the form of a set of real values,Wfor the set of all the candidate weights,Kthe number of the data sources is,w k is thatWIs of neutral degreekThe candidate weights corresponding to the data sources,Nis the total number of monitoring positions (namely, the water quality monitoring can be carried out at a plurality of positions of any water source, and corresponding original monitoring data is obtained),ii.e. for indicating that the data source corresponds to the secondiThe number of the monitoring positions is,Mis the total number of water quality monitoring species (i.e., PH, conductivity, dissolved oxygen, turbidity, temperature, etc.),mis the firstmThe species of the seeds to be monitored for water quality,v im (*) for indicating the firstiTo a monitoring locationmTrue value of species water quality monitoring species (i.e. the firstiTo a monitoring locationmSpecies of water quality monitoringAccurate monitoring data) of the initialization of the mobile terminal,v im (k) is the firstiTo a monitoring locationmUnder the species of water quality monitoringkPreliminary calibration data corresponding to a machine learning algorithm,d(·)for the purpose of the function used to calculate the deviation,d m (v im (*) ，v im (k) )for indication at the firstiTo a monitoring locationmPreliminary calibration data corresponding to the kth machine learning algorithm under the species of water quality monitoringv im (k) Relative to the adjusted accurate monitoring datav im (*) The deviation therebetween;s.t.for representingδ（W）AndW∈Sfor the constraint, exp (-) is an exponential function with the natural constant e as the base,W∈Squantity and data source indicating weight valueSThe number of (2) is the same. Accurate monitoring data after adjustment of the real water quality closest to the target water source can be obtained by minimizing the target function.
Under the condition that the preset convergence requirement is not met between the verification result and the historical verification result, it is indicated that the candidate weight corresponding to the machine learning algorithm needs to be continuously adjusted, and the weight of the data sourceWEach of (1)w k This can be expressed by using the following equation based on the lagrange multiplier method:
Figure 688392DEST_PATH_IMAGE002
（2）
based on the error obtained by the error function, and comparing w by Lagrange multiplier method k Multiple adjustments are made until minimizing the objective function (1) yields a minimum. Wherein the content of the first and second substances,v im (k’) for indicating the firstiTo a monitoring locationmUnder the species of water quality monitoringk’And (5) preliminary calibration data corresponding to the machine learning algorithm. By combining the machine learning algorithm with the truth finding technology, the problems of incomplete feature extraction and acquisition equipment can be effectively solvedAnd the problems of faults and the like cause the problem that the determined result of the water source treatment scheme is inaccurate.
By the method in the embodiment, the candidate weight corresponding to each machine learning algorithm can be obtained through training, and further, under the condition that the preset convergence requirement is met between the current verification result and the historical verification result, the candidate weight corresponding to each machine learning algorithm can be determined as the target weight corresponding to each machine learning algorithm.
As shown in fig. 2, as an alternative embodiment, as the foregoing method, in step S103, calibrating the target monitoring data by using each of at least two machine learning algorithms, to obtain calibrated data after calibrating the target monitoring data, the method includes:
step S301, inputting target monitoring data into a machine learning algorithm for any machine learning algorithm to obtain primary calibrated subdata obtained by calibrating the target monitoring data by the machine learning algorithm;
step S302, weighting all the primary calibrated subdata according to the target weight corresponding to each machine learning algorithm to obtain a target weighting result corresponding to each machine learning algorithm, and adding all the target weighting results to obtain calibrated data.
After determining the target weight corresponding to each machine learning algorithm, the target monitoring data can be input into the machine learning algorithms to obtain primary calibrated subdata obtained by calibrating the target monitoring data by the machine learning algorithms, that is, each machine learning algorithm independently processes the target monitoring data and can obtain the primary calibrated subdata corresponding to each machine learning algorithm.
After determining the primary calibrated sub-data corresponding to each machine learning algorithm, weighting all the primary calibrated sub-data according to the target weight corresponding to each machine learning algorithm to obtain a target weighting result corresponding to each machine learning algorithm, and finally adding all the target weighting results to obtain calibrated data.
For example: for the target monitoring data M1, primary calibrated sub-data obtained by performing linear regression calibration on the target monitoring data M1 is M3, primary calibrated sub-data obtained by performing decision tree calibration on the target monitoring data M1 is M4, primary calibrated sub-data obtained by performing random forest calibration on the target monitoring data M1 is M5, primary calibrated sub-data obtained by performing gradient lifting tree calibration on the target monitoring data M1 is M6, and primary calibrated sub-data obtained by performing XGBoost calibration on the target monitoring data M1 is M7; and the candidate weight of linear regression is 0.1, the candidate weight of decision tree is 0.1, the candidate weight of random forest is 0.3, the candidate weight of gradient lifting tree is 0.3, and the candidate weight of XGBoost is 0.2, then the calibrated data is (0.1 × M3+0.1 × M4+0.3 × M5+0.3 × M6+0.2 × M7).
Optionally, a weight set corresponding to each of the at least two candidate water source attribute information may be obtained by pre-training in advance through the method of the foregoing steps S1 to S3 and by using training data corresponding to each of the at least two candidate water source attribute information, where the candidate water source attribute information is used to indicate the located target geographic attribute and the water source usage of the target water source, and each weight set includes a weight corresponding to each machine learning algorithm.
And determining the designated water source attribute information with the highest similarity to the target water source from all the candidate water source attribute information, and determining the target weight corresponding to each machine learning algorithm according to the weight group corresponding to the designated water source attribute information.
By the method in the embodiment, the method for weighting the primary calibrated subdata according to the target weight corresponding to each machine learning algorithm is provided, so that the finally obtained calibrated data can integrate the advantages of different machine learning algorithms, and the accuracy is improved.
As an alternative embodiment, as in the foregoing method, the step S102 of preprocessing the raw monitoring data to obtain the target monitoring data includes the following steps:
obtaining the preprocessed monitoring data by performing the following operations of step S401, step S402, and step S403 on the raw monitoring data: step S401, removing outliers under the condition that a first anomaly check is carried out on original monitoring data and the outliers exist in the original monitoring data are determined; step S402, performing second anomaly check on the original monitoring data to determine that missing parameters exist in the original monitoring data, and predicting to obtain a predicted value of the missing parameters of a first specified characteristic type under the condition that all the missing parameters include the missing parameters of the first specified characteristic type, wherein the missing parameters are the characteristic parameters without parameter values, the first specified characteristic type is one of all the characteristic types of the target monitoring data, and the quantity of the missing parameters corresponding to the first specified characteristic type is less than or equal to a preset quantity within a preset time period; step S403, performing second anomaly check on the original monitoring data to determine that missing parameters exist in the original monitoring data, and eliminating all parameters corresponding to a second characteristic type under the condition that all the missing parameters include the missing parameters of the second specified characteristic type, wherein the second specified characteristic type is one of all the characteristic types of the target monitoring data, and the quantity of the missing parameters corresponding to the second specified characteristic type is greater than the preset quantity within a preset time period;
after the original monitoring data is obtained, an outlier in the original monitoring data can be determined by performing a first anomaly check, where the outlier can be a value whose value obviously deviates from normal: when the original monitoring data are acquired at a certain time point, the original monitoring data can be compared with the latest X historical monitoring data to determine whether an outlier exists or not; or, when the original monitoring data is acquired multiple times in a certain time period, each feature has a plurality of corresponding feature parameters, and each feature parameter has a corresponding parameter value when the acquisition is completed (for example, if the feature parameter is a water temperature T, the feature parameter may include T1=20 ℃, T2=21 ℃, T3=20 ℃, where the temperature is the parameter value), so that the feature parameters corresponding to the same feature in the original monitoring data may be compared to determine whether an outlier exists; and rejecting outliers if it is determined that outliers exist.
In addition, second anomaly check can be carried out on the original monitoring data to determine all missing parameters in the original monitoring data, wherein the missing parameters are characteristic parameters without parameter values; for example, characteristic of water temperature T, the presence of one characteristic parameter T2= null, indicating that T2 has no corresponding parameter value, and therefore T2 is a missing parameter.
After obtaining all the missing parameters, the number of the missing parameters corresponding to each feature type may be determined, so as to determine a first specified feature type in all the feature types, where the first specified feature type is a feature type in which the number of the missing parameters corresponding to a preset time period is less than or equal to a preset number (for example, 1, 2, and the like, and may be adjusted according to the length of the preset time period). The preset time period may be a time period (for example, 1min, 1hour, and the like, which may be adjusted according to an actual application situation) before the parameter value of the missing parameter is acquired.
In a preset time period, under the condition that the number of the missing parameters corresponding to the first specified feature type is smaller than or equal to the preset number, namely the number of the missing parameters is indicated to be less, the missing values are predicted, and a predicted value corresponding to each missing parameter is obtained.
For any missing parameter, the predicted value of the missing parameter is obtained through prediction, and the predicted value can be obtained through calculation of an average value of two parameters which are adjacent to the missing parameter before and after the acquisition time of the missing parameter under the first specified feature type corresponding to the missing parameter, or through determination of a change function between all parameters of the first specified feature type corresponding to the missing parameter, and determination of the predicted value of the missing parameter based on the acquisition time of the missing parameter.
When all the missing parameters include the missing parameter corresponding to the second specified feature type, and the second specified feature type is the feature type in which the number of the corresponding missing parameters is greater than the preset number within the preset time period, that is, it indicates that the missing parameters of the second specified feature type are more and mostly continuous missing, the second specified feature type needs to be removed.
And after removing the outlier, predicting to obtain a predicted value of the missing parameter of the first specified characteristic type, and removing all parameters corresponding to the second specified characteristic type, the preprocessed monitoring data can be obtained.
Optionally, under the condition that all the missing parameters include the missing parameter of the second specified feature type, current target space-time information can be further determined, wherein the target space-time information includes target time information, target position information and target weather information; when appointed spatio-temporal information meeting preset similarity with target spatio-temporal information is inquired in all historical spatio-temporal information, determining historical monitoring data corresponding to the appointed spatio-temporal information as appointed monitoring data; replacing missing parameters corresponding to a second specified feature type in the original monitoring data by parameters corresponding to the second specified feature type in the specified monitoring data to obtain supplemented parameters corresponding to the second specified feature type in the original monitoring data, and removing outliers to obtain preprocessed monitoring data; wherein each historical spatiotemporal information has corresponding historical monitoring data. And then, under the condition that the quantity of the missing parameters corresponding to the second specified characteristic type in the original monitoring data is larger than the preset quantity, the missing parameters corresponding to the second specified characteristic type can be preprocessed in a historical data supplementing mode, so that the quantity of the characteristic types finally used for determining the accurate monitoring data can be increased, and the accuracy of the accurate monitoring data can be further improved.
And S404, performing feature extraction on the preprocessed monitoring data to obtain target monitoring data.
After the preprocessed monitoring data is obtained, feature extraction may be performed on the preprocessed monitoring data, so that the obtained target monitoring data may include data corresponding to features that need to be corrected.
According to the method in the embodiment, the original monitoring data are subjected to the first anomaly detection and the second anomaly detection, the outlier and the missing parameter in the original monitoring data are determined, and then the original monitoring data can be adjusted, so that the obtained preprocessed monitoring data can not include target features with more outliers and missing parameters, the accuracy of the obtained target monitoring data can be improved, and the purpose of finally improving the accuracy of the accurate monitoring data is achieved.
As an alternative embodiment, as in the foregoing method, before performing feature extraction on the preprocessed monitoring data in step S404 to obtain target monitoring data, the method further includes the following steps:
step S501, determining the characteristic type in monitoring data acquired by target water quality monitoring equipment after water quality monitoring is carried out on a candidate water source;
step S502, determining a target characteristic type in all characteristic types by analyzing the characteristic correlation between the characteristic types and the real water quality of the candidate water source, wherein the correlation between the parameter value of the target characteristic type and the real water quality of the candidate water source meets the preset correlation requirement;
step S503, a new characteristic type is determined in all the characteristic types by analyzing the characteristic correlation between the characteristic types and the real water quality of the candidate water source, wherein the new characteristic type is obtained according to the characteristic parameter structure corresponding to at least one characteristic type, and the correlation between the parameter value of the new characteristic type and the real water quality of the candidate water source meets the preset correlation requirement.
After the target water quality monitoring device is determined, the feature types in the monitoring data acquired after the water quality monitoring of the candidate water source by the water quality monitoring device is performed can be determined, and the feature types may include but are not limited to: PH, conductivity, dissolved oxygen, turbidity, temperature.
The candidate water source is a water source used for providing reference data for finally determining the target characteristic type and the new characteristic type. Candidate water sources may include one or more and candidate water sources may or may not include target water sources.
After all the characteristic types are obtained, a target characteristic type can be determined in all the characteristic types through characteristic correlation analysis between the characteristic types and the real water quality of the candidate water source, the correlation between the parameter value of the target characteristic type and the real water quality of the candidate water source meets a preset correlation requirement, the preset correlation requirement can be that the correlation between the parameter value of the target characteristic type and the real water quality of the candidate water source is larger than a preset correlation lower limit threshold value, and the correlation between the parameter value of the target characteristic type and the real water quality of the candidate water source is larger than the correlation between any characteristic type except the target characteristic type and the real water quality in all the characteristic types. Alternatively, the real water quality may be monitored by a high-precision monitoring device.
In addition, a new characteristic type can be determined in all the characteristic types by analyzing the characteristic correlation between the characteristic types and the real water quality of the candidate water source.
The new characteristic type is obtained according to a characteristic parameter structure corresponding to at least one characteristic type, and the correlation between the parameter value of the new characteristic type and the real water quality of the candidate water source meets the preset correlation requirement; namely: meanwhile, the new feature type obtained by construction is a new feature which may affect the calibration value, for example, a time sequence feature, a differential feature and the like obtained by construction through feature parameter values corresponding to the same feature type; furthermore, a new feature type may be constructed based on at least two feature types.
By the method in the embodiment, the target characteristic type with high correlation with the water quality can be determined from all the existing characteristic types, and meanwhile, a new characteristic type can be determined from all the characteristic types by performing characteristic correlation analysis on the characteristic types and the real water quality of the candidate water source; and then the correlation between the characteristic parameters finally used for calibration and the water quality is improved, so that the aim of improving the accuracy of finally obtained accurate monitoring data is fulfilled.
As an alternative embodiment, as in the foregoing method, the step S404 of performing feature extraction on the preprocessed monitoring data to obtain target monitoring data includes the following steps:
step S601, extracting a first target parameter corresponding to a target feature type from the preprocessed monitoring data, and taking the first target parameter corresponding to the target feature type as first sub-target monitoring data;
step S602, extracting the existing characteristic type corresponding to the new characteristic type from the preprocessed monitoring data; constructing the characteristic parameters corresponding to the existing characteristic types to obtain the characteristic parameters corresponding to the new characteristic types, and obtaining second sub-target monitoring data;
step S603, obtaining target monitoring data based on the first sub-target monitoring data and the second sub-target monitoring data.
After the target feature type and the preprocessed monitoring data are determined, all feature parameters corresponding to the target feature type can be determined in the preprocessed monitoring data and are used as first sub-target monitoring data.
After determining the new feature type and the preprocessed monitoring data, the existing feature type corresponding to the new feature type can be extracted from the preprocessed monitoring data, and as is known from the foregoing embodiment, the new feature type is constructed according to the feature parameter corresponding to at least one feature type, and thus, the new feature type corresponds to at least one existing feature type; and determining the characteristic parameters corresponding to the existing characteristic types, and further, constructing the characteristic parameters corresponding to the existing characteristic types to obtain the characteristic parameters corresponding to the new characteristic types, so as to obtain the second sub-target monitoring data.
After first sub-target monitoring data corresponding to the target feature type and second sub-target monitoring data corresponding to the new feature type are obtained, target monitoring data consisting of all the first sub-target monitoring data and the second sub-target monitoring data can be obtained; that is, the target monitoring data includes first sub-target monitoring data and second sub-target monitoring data.
As an alternative embodiment, as in the foregoing method, before the step S101 acquires original monitoring data acquired after the target water source is monitored by the target water quality monitoring device, the method further includes the following steps:
step S701, determining target water source attribute information corresponding to a target water source, wherein the target water source attribute information is used for indicating the target geographical attribute and the water source use of the target water source.
Before monitoring the target water source, target water source attribute information corresponding to the target water source can be predetermined, and the target water source attribute information is used for indicating the target geographical attribute and the water source use of the target water source.
The target geographic attributes may include, but are not limited to: latitude and longitude, altitude, climate type, soil type, etc. may have an impact on water quality.
Water source uses may include, but are not limited to: drinking water, protected area water sources, aquaculture and the like.
For example, the target water source attribute information I may be: longitude (x 1), latitude (x 2), altitude (x 3), climate type (temperate monsoon climate), soil type (red soil), drinking water.
Step S702, obtaining the designated water source attribute information with the highest matching degree with the target water source attribute information through matching in all the candidate water source attribute information, and taking the candidate water quality monitoring equipment corresponding to the designated water source attribute information as target water quality monitoring equipment, wherein each candidate water source attribute information is preset with corresponding candidate water quality monitoring equipment, and for each candidate water source attribute information, the characteristic correlation between the characteristic type of the parameter collected by the candidate water quality monitoring equipment corresponding to the candidate water source attribute information and the water quality of the water source indicated by the candidate water source attribute information meets the preset correlation requirement.
The corresponding relation between different candidate water source attribute information and the candidate water quality monitoring equipment can be preset, and the corresponding relation can be stored in a data table mode.
After the target water source attribute information is determined, each candidate water source attribute information can be compared with the target water source attribute information in sequence, the matching degree between each candidate water source attribute information and the target water source attribute information is determined, the weight of each attribute type (such as longitude and latitude, altitude, climate type, soil type and water source application) can be set, and the weighted matching degree is obtained.
After the designated water source attribute information with the highest matching degree is obtained, the candidate water quality monitoring equipment corresponding to the designated water source attribute information can be used as target water quality monitoring equipment.
By the method in the embodiment, the target water quality monitoring equipment with the highest matching degree with the target water source can be intelligently analyzed, and the characteristic correlation between the characteristic type of the parameter acquired by the candidate water quality monitoring equipment corresponding to the candidate water source attribute information and the water quality of the water source indicated by the candidate water source attribute information meets the preset correlation requirement, so that the characteristic correlation between the characteristic type in the original monitoring data acquired by the target water quality monitoring equipment and the water quality of the target water source meets the preset correlation requirement, and the accuracy of the accurate monitoring data acquired by later analysis can be improved.
As shown in fig. 3, the present application further provides an application example applying any of the foregoing embodiments:
1. deploying a water quality monitoring micro station (namely, an optional target water quality monitoring device) at a target water source needing to be monitored;
2. data acquisition: the raw monitoring data is obtained by monitoring through the water quality monitoring micro station and the relevant raw monitoring data is transmitted to a background data processing program (i.e. a program for realizing the method of the embodiment).
3. Data processing: processing received original monitoring data through a background preprocessing program, wherein the main content comprises exception processing (outliers in the monitoring data are removed, missing parameters are removed or linear interpolation is carried out, if the missing values corresponding to one feature type are less and continuous missing is less, linear interpolation processing is adopted, if the missing values corresponding to one feature type are more and continuous missing mostly, the feature type is marked as a second feature type, and all parameters of the second feature type are removed); extracting features (carrying out correlation analysis on data features, reserving parameter values of target feature types with high correlation, and constructing new feature types possibly influencing calibration values, such as time sequence features, differential features and the like); and obtaining the monitoring data after pretreatment.
4. Multi-model calibration: and inputting the preprocessed monitoring data into the machine learning models to obtain a primary calibration result output by each machine learning model, wherein the selected machine learning models comprise methods such as but not limited to linear regression, decision trees, random forests, gradient lifting trees, XGboost and the like.
5. Inputting the candidate weighting result output by the machine learning algorithm into a data fusion module, wherein the specific technical process of the data fusion module comprises the following steps: (i) initializing to obtain initialized accurate monitoring data corresponding to the target water source and initial weights of all machine learning algorithms; a weighted error value (i.e., the initial verification result described in step S1) at a given initial accurate monitor data and each initial weight is calculated by equation (1). (ii) The current weight (i.e., the initial weight in step S2 or the candidate weight in step S4) of each machine learning algorithm is calculated and updated by formula (2), resulting in the latest weight (i.e., the candidate weight in step S2 or the adjusted weight in step S4). (iii) And calculating to obtain the latest current accurate monitoring data according to the latest weight, and calculating to obtain the current verification result. (iv) And judging whether convergence occurs or not based on the current verification result and the historical verification result. (v) Repeating steps (ii) (iii) (iv) until the variation of the weighted error (i.e., the current verification result described in step S3) calculated according to formula (1) is less than the specified threshold, i.e., determining convergence, and obtaining the optimal weight (i.e., the target weight described in step S5) and the final true value (i.e., accurate monitoring data).
6. And obtaining the weighted value of each machine learning algorithm, carrying out weighted aggregation on the weighted values to obtain final accurate monitoring data, and determining a target water source treatment scheme based on the accurate monitoring data.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (e.g., a ROM (Read-Only Memory)/RAM (Random Access Memory), a magnetic disk, an optical disk) and includes several instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the methods according to the embodiments of the present application.
According to another aspect of the embodiments of the present application, there is also provided an apparatus for determining a water source treatment plan for implementing the method for determining a water source treatment plan described above. FIG. 4 is a block diagram of an alternative apparatus for determining water source treatment programs according to embodiments of the present application, and as shown in FIG. 4, the apparatus may include:
the system comprises an acquisition module 1, a data acquisition module and a data processing module, wherein the acquisition module is used for acquiring original monitoring data acquired after water quality monitoring is carried out on a target water source by target water quality monitoring equipment, and the accuracy of the water quality monitoring carried out by the target water quality monitoring equipment is lower than the lower limit of a preset accurate value;
the preprocessing module 2 is used for preprocessing the original monitoring data to obtain target monitoring data;
the calibration module 3 is configured to calibrate the target monitoring data through each of the at least two machine learning algorithms to obtain calibrated data obtained by calibrating the target monitoring data, where accuracy of calibration of the target monitoring data by any two machine learning algorithms in the at least two machine learning algorithms is different from each other;
the obtaining module 4 is used for obtaining accurate monitoring data for indicating an accurate water quality monitoring result of the target water source based on all the calibrated data;
the target pollution information determining module 5 is used for determining target pollution information of the target water source according to the accurate monitoring data;
and the scheme determining module 6 is configured to determine a target water source treatment scheme based on the pollution indexes indicated in the target pollution information, where each target pollution information includes at least one pollution index, and the target water source treatment scheme includes target treatment equipment and/or target treatment agents for treating pollutants corresponding to each pollution index.
It should be noted that the obtaining module 1 in this embodiment may be configured to execute the step S101, the preprocessing module 2 in this embodiment may be configured to execute the step S102, the calibrating module 3 in this embodiment may be configured to execute the step S103, the obtaining module 4 in this embodiment may be configured to execute the step S104, the target contamination information determining module 5 in this embodiment may be configured to execute the step S105, and the scheme determining module 6 in this embodiment may be configured to execute the step S106.
It should be noted here that the modules described above are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the disclosure of the above embodiments. It should be noted that the modules described above as a part of the apparatus may be run in a hardware environment for implementing the method shown in fig. 1, and may be implemented by software or hardware, where the hardware environment includes a network environment.
According to yet another aspect of the embodiments of the present application, there is also provided an electronic device for implementing the method for determining a water source treatment plan, which may be a server, a terminal, or a combination thereof.
According to another embodiment of the present application, there is also provided an electronic apparatus, as shown in fig. 5, the electronic apparatus may include: a processor 602, a communication interface 604, a memory 606, and a communication bus 608, wherein the processor 602, the communication interface 604, and the memory 606 communicate with each other through the communication bus 608.
A memory 606 for storing computer programs;
the processor 602 is configured to implement the following steps when executing the program stored in the memory 606:
step S101, acquiring original monitoring data acquired after water quality monitoring is carried out on a target water source by target water quality monitoring equipment, wherein the accuracy of the target water quality monitoring equipment in water quality monitoring is lower than a preset lower limit of an accurate value;
step S102, preprocessing original monitoring data to obtain target monitoring data;
step S103, calibrating the target monitoring data through each of at least two machine learning algorithms to obtain calibrated data after the target monitoring data are calibrated, wherein the accuracy of the target monitoring data calibration performed by any two machine learning algorithms in the at least two machine learning algorithms is different from each other;
step S104, obtaining accurate monitoring data for indicating the accurate water quality monitoring result of the target water source based on all the calibrated data;
step S105, determining target pollution information of a target water source according to the accurate monitoring data;
and S106, determining a target water source treatment scheme based on the pollution indexes indicated in the target pollution information, wherein each target pollution information comprises at least one pollution index, and the target water source treatment scheme comprises target treatment equipment and/or target treatment agents for treating pollutants corresponding to each pollution index.
Alternatively, in this embodiment, the communication bus may be a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus. The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The processor may be a general-purpose processor, and may include but is not limited to: a CPU (Central Processing Unit), an NP (Network Processor), and the like; but also a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component.
The embodiment of the present application further provides a computer-readable storage medium, where the storage medium includes a stored program, and when the program runs, the method steps of the above method embodiment are executed.
Optionally, in this embodiment, the storage medium may include, but is not limited to: various media capable of storing program codes, such as a U disk, a ROM, a RAM, a removable hard disk, a magnetic disk, or an optical disk.
The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.
The integrated unit in the above embodiments, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or a part of or all or part of the technical solution contributing to the prior art may be embodied in the form of a software product stored in a storage medium, and including instructions for causing one or more computer devices (which may be personal computers, servers, network devices, or the like) to execute all or part of the steps of the method described in the embodiments of the present application.
In the above embodiments of the present application, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed client may be implemented in other manners. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, and may also be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution provided in this embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The foregoing is only a preferred embodiment of the present application and it should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims (8)

1. A method of determining a water source remediation program, comprising:
acquiring original monitoring data acquired after water quality monitoring is carried out on a target water source by target water quality monitoring equipment, wherein the accuracy of the target water quality monitoring equipment for carrying out the water quality monitoring is lower than a preset lower limit of an accurate value;
preprocessing the original monitoring data to obtain target monitoring data;
calibrating the target monitoring data through each of at least two machine learning algorithms to obtain calibrated data after the target monitoring data are calibrated, wherein the accuracy of the target monitoring data calibrated by any two machine learning algorithms in the at least two machine learning algorithms is different from each other;
obtaining accurate monitoring data used for indicating an accurate water quality monitoring result of the target water source based on all the calibrated data;
determining target pollution information of the target water source according to the accurate monitoring data;
determining a target water source treatment scheme based on pollution indexes indicated in the target pollution information, wherein each target pollution information comprises at least one pollution index, and the target water source treatment scheme comprises target treatment equipment and/or target treatment agents for treating pollutants corresponding to each pollution index;
before the calibrating the target monitoring data by each of the at least two machine learning algorithms to obtain calibrated data after calibrating the target monitoring data, the method further includes:
step S1, according to the initial weight corresponding to each machine learning algorithm, weighting the deviation between the initial calibration data corresponding to each machine learning algorithm with respect to the initial accurate monitoring data to obtain an initial weighted deviation corresponding to each machine learning algorithm, and accumulating all the initial weighted deviations to obtain an initial verification result, wherein the initial calibration data corresponding to each machine learning algorithm is obtained by calibrating the target monitoring data by each machine learning algorithm, and the initial accurate monitoring data is determined according to the initial calibration data;
step S2, for each machine learning algorithm, adjusting the initial weight corresponding to each machine learning algorithm by a preset weight adjusting method to obtain a candidate weight, weighting all the preliminary calibration data according to the candidate weight corresponding to each machine learning algorithm to obtain a first weighting result corresponding to each machine learning algorithm, and accumulating all the first weighting results to obtain adjusted accurate monitoring data;
step S3, according to the candidate weight corresponding to each machine learning algorithm, weighting the deviation of the preliminary calibration data corresponding to each machine learning algorithm relative to the adjusted accurate monitoring data to obtain a candidate weighted deviation corresponding to each machine learning algorithm, and accumulating all the candidate weighted deviations to obtain a current verification result;
step S4, under the condition that the preset convergence requirement is not satisfied between the current check result and the historical check result, for each machine learning algorithm, adjusting the candidate weight corresponding to each machine learning algorithm by a preset weight adjusting method to obtain an adjusted weight, weighting all the preliminary calibration data according to the adjusted weight corresponding to each machine learning algorithm to obtain a second weighting result corresponding to each machine learning algorithm, accumulating all the second weighting results to obtain current accurate monitoring data, updating the candidate weight in the step S3 by the adjusted weight, updating the adjusted accurate monitoring data in the step S3 by the current accurate monitoring data, jumping to the step S3, and circulating, until the current verification result and the historical verification result meet the preset convergence requirement, wherein the historical verification result at least comprises a verification result obtained before the current verification result is obtained;
step S5, under the condition that the preset convergence requirement is met between the current verification result and the historical verification result, determining that all the candidate weights meet a preset accuracy requirement, and determining the candidate weight corresponding to each machine learning algorithm as a target weight corresponding to each machine learning algorithm;
the calibrating the target monitoring data by each of at least two machine learning algorithms to obtain calibrated data after calibrating the target monitoring data includes: for any one machine learning algorithm, inputting the target monitoring data into the machine learning algorithm to obtain primary calibrated subdata obtained by calibrating the target monitoring data by the machine learning algorithm; and weighting all the primary calibrated subdata according to the target weight corresponding to each machine learning algorithm to obtain a target weighting result corresponding to each machine learning algorithm, and adding all the target weighting results to obtain the calibrated data.
2. The method of claim 1, wherein the pre-processing the raw monitoring data to obtain target monitoring data comprises:
obtaining preprocessed monitoring data by performing the following operations on the raw monitoring data: performing first anomaly check on the original monitoring data, and removing outliers under the condition that the outliers exist in the original monitoring data; performing second anomaly check on the original monitoring data to determine that missing parameters exist in the original monitoring data, and predicting to obtain a predicted value of the missing parameters of a first specified characteristic type under the condition that all the missing parameters include the missing parameters of the first specified characteristic type, wherein the missing parameters are characteristic parameters without parameter values, the first specified characteristic type is one of all the characteristic types of the target monitoring data, and in a preset time period, the number of the missing parameters corresponding to the first specified characteristic type is less than or equal to a preset number; performing second anomaly check on the original monitoring data to determine that missing parameters exist in the original monitoring data, and removing all parameters corresponding to a second feature type under the condition that all the missing parameters include the missing parameters of the second specified feature type, wherein the second specified feature type is one of all the feature types of the target monitoring data, and the number of the missing parameters corresponding to the second specified feature type is larger than the preset number within a preset time period;
and performing feature extraction on the preprocessed monitoring data to obtain the target monitoring data.
3. The method of claim 2, wherein before said performing feature extraction on said preprocessed monitoring data to obtain said target monitoring data, said method further comprises:
determining the characteristic type in monitoring data acquired by the target water quality monitoring equipment after the target water quality monitoring equipment monitors the water quality of the candidate water source;
determining a target characteristic type in all the characteristic types by performing characteristic correlation analysis between the characteristic types and the real water quality of the candidate water source, wherein the correlation between the parameter value of the target characteristic type and the real water quality of the candidate water source meets the preset correlation requirement;
and determining a new characteristic type in all the characteristic types by performing characteristic correlation analysis between the characteristic types and the real water quality of the candidate water source, wherein the new characteristic type is obtained according to a characteristic parameter corresponding to at least one characteristic type, and the correlation between the parameter value of the new characteristic type and the real water quality of the candidate water source meets the preset correlation requirement.
4. The method of claim 3, wherein the performing feature extraction on the preprocessed monitoring data to obtain the target monitoring data comprises:
extracting a first target parameter corresponding to the target feature type from the preprocessed monitoring data, and taking the first target parameter corresponding to the target feature type as first sub-target monitoring data;
extracting the existing feature type corresponding to the new feature type from the preprocessed monitoring data; constructing the characteristic parameters corresponding to the existing characteristic types to obtain the characteristic parameters corresponding to the new characteristic types, and obtaining second sub-target monitoring data;
and obtaining the target monitoring data based on the first sub-target monitoring data and the second sub-target monitoring data.
5. The method of claim 1, wherein prior to said obtaining raw monitoring data collected after water quality monitoring of a target water source by a target water quality monitoring device, the method further comprises:
determining target water source attribute information corresponding to the target water source, wherein the target water source attribute information is used for indicating the target geographical attribute and the water source use of the target water source;
obtaining appointed water source attribute information with the highest matching degree with the target water source attribute information through matching in all candidate water source attribute information, and taking candidate water quality monitoring equipment corresponding to the appointed water source attribute information as the target water quality monitoring equipment, wherein each candidate water source attribute information is preset with corresponding candidate water quality monitoring equipment, and for each candidate water source attribute information, the characteristic correlation between the characteristic type of the parameter acquired by the candidate water quality monitoring equipment corresponding to the candidate water source attribute information and the water quality of a water source indicated by the candidate water source attribute information meets the preset correlation requirement.
6. An apparatus for determining a water source remediation program, comprising:
the system comprises an acquisition module, a data acquisition module and a data processing module, wherein the acquisition module is used for acquiring original monitoring data acquired after water quality monitoring is carried out on a target water source by target water quality monitoring equipment, and the accuracy of the water quality monitoring carried out by the target water quality monitoring equipment is lower than the lower limit of a preset accurate value;
the preprocessing module is used for preprocessing the original monitoring data to obtain target monitoring data;
the calibration module is used for calibrating the target monitoring data through each of at least two machine learning algorithms to obtain calibrated data after the target monitoring data are calibrated, wherein the accuracy of the calibration of the target monitoring data by any two machine learning algorithms in the at least two machine learning algorithms is different from each other;
the obtaining module is used for obtaining accurate monitoring data used for indicating an accurate water quality monitoring result of the target water source based on all the calibrated data;
the target pollution information determining module is used for determining the target pollution information of the target water source according to the accurate monitoring data;
the system comprises a scheme determining module, a target water source treatment scheme and a control module, wherein the scheme determining module is used for determining a target water source treatment scheme based on pollution indexes indicated in the target pollution information, each target pollution information comprises at least one pollution index, and the target water source treatment scheme comprises target treatment equipment and/or target treatment agents for treating pollutants corresponding to each pollution index;
the method also comprises a module for executing the following steps: step S1, according to the initial weight corresponding to each machine learning algorithm, weighting the deviation between the initial calibration data corresponding to each machine learning algorithm with respect to the initial accurate monitoring data to obtain an initial weighted deviation corresponding to each machine learning algorithm, and accumulating all the initial weighted deviations to obtain an initial verification result, wherein the initial calibration data corresponding to each machine learning algorithm is obtained by calibrating the target monitoring data by each machine learning algorithm, and the initial accurate monitoring data is determined according to the initial calibration data; step S2, for each machine learning algorithm, adjusting the initial weight corresponding to each machine learning algorithm by a preset weight adjusting method to obtain a candidate weight, weighting all the preliminary calibration data according to the candidate weight corresponding to each machine learning algorithm to obtain a first weighting result corresponding to each machine learning algorithm, and accumulating all the first weighting results to obtain adjusted accurate monitoring data; step S3, according to the candidate weight corresponding to each machine learning algorithm, weighting the deviation of the preliminary calibration data corresponding to each machine learning algorithm relative to the adjusted accurate monitoring data to obtain a candidate weighted deviation corresponding to each machine learning algorithm, and accumulating all the candidate weighted deviations to obtain a current verification result; step S4, under the condition that the preset convergence requirement is not met between the current verification result and the historical verification result, for each machine learning algorithm, the candidate weight corresponding to each machine learning algorithm is adjusted through a preset weight adjusting method to obtain an adjusted weight, all the preliminary calibration data are weighted according to the adjusted weight corresponding to each machine learning algorithm to obtain a second weighting result corresponding to each machine learning algorithm, all the second weighting results are accumulated to obtain current accurate monitoring data, the candidate weight in the step S3 is updated through the adjusted weight, the adjusted accurate monitoring data in the step S3 are updated through the current accurate monitoring data, the step S3 is skipped to, and the process is repeated, until the current verification result and the historical verification result meet the preset convergence requirement, wherein the historical verification result at least comprises a verification result obtained before the current verification result is obtained; step S5, under the condition that the preset convergence requirement is met between the current verification result and the historical verification result, determining that all the candidate weights meet a preset accuracy requirement, and determining the candidate weight corresponding to each machine learning algorithm as a target weight corresponding to each machine learning algorithm;
the calibration module is further configured to input the target monitoring data into the machine learning algorithm for any one of the machine learning algorithms to obtain primary calibrated sub-data obtained by calibrating the target monitoring data by the machine learning algorithm; and weighting all the primary calibrated subdata according to the target weight corresponding to each machine learning algorithm to obtain a target weighting result corresponding to each machine learning algorithm, and adding all the target weighting results to obtain the calibrated data.
7. An electronic device comprising a processor, a communication interface, a memory and a communication bus, wherein said processor, said communication interface and said memory communicate with each other via said communication bus,
the memory for storing a computer program;
the processor for performing the method steps of any of claims 1 to 5 by running the computer program stored on the memory.
8. A computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to carry out the method steps as claimed in any of claims 1 to 5 when executed.
CN202210777656.9A 2022-07-04 2022-07-04 Method and device for determining water source treatment scheme, electronic equipment and storage medium Active CN114881540B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210777656.9A CN114881540B (en) 2022-07-04 2022-07-04 Method and device for determining water source treatment scheme, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210777656.9A CN114881540B (en) 2022-07-04 2022-07-04 Method and device for determining water source treatment scheme, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN114881540A CN114881540A (en) 2022-08-09
CN114881540B true CN114881540B (en) 2022-09-27

Family

ID=82683462

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210777656.9A Active CN114881540B (en) 2022-07-04 2022-07-04 Method and device for determining water source treatment scheme, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN114881540B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117076991B (en) * 2023-10-16 2024-01-02 云境商务智能研究院南京有限公司 Power consumption abnormality monitoring method and device for pollution control equipment and computer equipment

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111053540A (en) * 2019-12-23 2020-04-24 浙江大学 CRRT computer-patient body temperature correction system based on machine learning
AU2020100700A4 (en) * 2020-05-05 2020-06-11 Li, Jinglin Miss A Correction Method for Gas Sensor Based on Machine Learning
AU2020102518A4 (en) * 2020-09-30 2020-11-19 Lu, Junjie Mr A method of gas sensor calibration based on linear optimization
CN112146761A (en) * 2020-08-14 2020-12-29 上海数川数据科技有限公司 Human body temperature measurement compensation method based on machine learning
CN214668555U (en) * 2021-01-25 2021-11-09 广东盈峰科技有限公司 Water quality on-line monitoring system
CA3127938A1 (en) * 2020-08-12 2022-02-12 Marine Thinking Inc. Holding tank monitoring system based on wireless sensor network and monitoring method

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3519636B1 (en) * 2016-09-29 2021-08-25 Cywat Technologies Ltd. Method for constant online water quality & safety monitoring of a fluid system
US20180181876A1 (en) * 2016-12-22 2018-06-28 Intel Corporation Unsupervised machine learning to manage aquatic resources
WO2020213614A1 (en) * 2019-04-15 2020-10-22 国立研究開発法人理化学研究所 Device, method and program for environmental factor estimation, learned model and recording medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111053540A (en) * 2019-12-23 2020-04-24 浙江大学 CRRT computer-patient body temperature correction system based on machine learning
AU2020100700A4 (en) * 2020-05-05 2020-06-11 Li, Jinglin Miss A Correction Method for Gas Sensor Based on Machine Learning
CA3127938A1 (en) * 2020-08-12 2022-02-12 Marine Thinking Inc. Holding tank monitoring system based on wireless sensor network and monitoring method
CN112146761A (en) * 2020-08-14 2020-12-29 上海数川数据科技有限公司 Human body temperature measurement compensation method based on machine learning
AU2020102518A4 (en) * 2020-09-30 2020-11-19 Lu, Junjie Mr A method of gas sensor calibration based on linear optimization
CN214668555U (en) * 2021-01-25 2021-11-09 广东盈峰科技有限公司 Water quality on-line monitoring system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
丹江口水库湖北库区水质分区及长期变化趋势;张煦等;《中国环境监测》;20160215(第01期);第65-69页 *

Also Published As

Publication number Publication date
CN114881540A (en) 2022-08-09

Similar Documents

Publication Publication Date Title
EP3499375B1 (en) Predicting quality of an information system using system dynamics modelling and machine learning
CN112735094B (en) Geological disaster prediction method and device based on machine learning and electronic equipment
CN113935497A (en) Intelligent operation and maintenance fault processing method, device and equipment and storage medium thereof
CN109905696A (en) A kind of recognition methods of the Video service Quality of experience based on encryption data on flows
CN114881540B (en) Method and device for determining water source treatment scheme, electronic equipment and storage medium
CN110928957A (en) Data clustering method and device
CN111523683A (en) Method and system for predicting technological parameters in tobacco processing
CN111797320A (en) Data processing method, device, equipment and storage medium
CN112966014A (en) Method and device for searching target object
CN113313280A (en) Cloud platform inspection method, electronic equipment and nonvolatile storage medium
CN113536066A (en) Data anomaly detection algorithm determination method and device and computer equipment
CN112860676A (en) Data cleaning method applied to big data mining and business analysis and cloud server
CN105681088B (en) Reliability and performance comprehensive evaluation method for Internet of things
CN115932144B (en) Chromatograph performance detection method, chromatograph performance detection device, chromatograph performance detection equipment and computer medium
CN109657907B (en) Quality control method and device for geographical national condition monitoring data and terminal equipment
CN116822366A (en) Construction of runoff pollution load calculation model and runoff pollution load calculation method
CN114757391B (en) Network data space design and application method oriented to service quality prediction
KR102500534B1 (en) Recurrent neural network based water resource information generating device and method
CN114139643B (en) Monoglyceride quality detection method and system based on machine vision
JP6587950B2 (en) Program, apparatus, and method capable of detecting time series change point by scalar feature
CN114943273A (en) Data processing method, storage medium, and computer terminal
CN115689320A (en) Health management method and device for base station operation and computer readable storage medium
CN117591283B (en) Cloud cutting equipment management method and system based on cross-platform data fusion
CN111103333B (en) Method and device for calibrating soil water content sensor
KR20230165996A (en) Method for Providing Time Series Data Optimized Analysis/Prediction Based on Artificial Intelligence

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant