CN117686447A - Water quality monitoring method, device, equipment and medium based on multichannel model - Google Patents
Water quality monitoring method, device, equipment and medium based on multichannel model Download PDFInfo
- Publication number
- CN117686447A CN117686447A CN202410130354.1A CN202410130354A CN117686447A CN 117686447 A CN117686447 A CN 117686447A CN 202410130354 A CN202410130354 A CN 202410130354A CN 117686447 A CN117686447 A CN 117686447A
- Authority
- CN
- China
- Prior art keywords
- water
- data
- water quality
- gene
- latest
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 title claims abstract description 587
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000012544 monitoring process Methods 0.000 title claims abstract description 35
- 108090000623 proteins and genes Proteins 0.000 claims abstract description 118
- 230000003595 spectral effect Effects 0.000 claims abstract description 40
- 238000012549 training Methods 0.000 claims abstract description 23
- 238000001228 spectrum Methods 0.000 claims description 23
- 238000010521 absorption reaction Methods 0.000 claims description 20
- 238000002835 absorbance Methods 0.000 claims description 12
- 239000011159 matrix material Substances 0.000 claims description 12
- 230000002159 abnormal effect Effects 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 9
- 238000012806 monitoring device Methods 0.000 claims description 8
- 238000007781 pre-processing Methods 0.000 claims description 6
- 238000001514 detection method Methods 0.000 claims description 5
- 238000007637 random forest analysis Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000012706 support-vector machine Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 230000005180 public health Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A20/00—Water conservation; Efficient water supply; Efficient water use
- Y02A20/152—Water filtration
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The embodiment of the invention provides a water quality monitoring method, a device, equipment and a medium based on a multichannel model, which relate to the technical field of water quality monitoring and comprise the following steps: acquiring water quality data of water samples in different areas in continuous time periods and corresponding water-based data; obtaining a plurality of water gene types based on the spectral characteristics of each water gene data; training the model by taking water quality data corresponding to all water-based factor types as sample data to obtain a multichannel water quality prediction model; acquiring the latest water quality data and the latest water gene data corresponding to the latest water quality data of water samples in any region, and determining the latest water gene type corresponding to the latest water gene data; updating the water quality prediction model by using the latest water quality data; and inputting the water quality data of the water sample of the region to be monitored in a preset time period before the current time point into the updated water quality prediction model to obtain the water quality data of the region to be monitored in the current time point. The scheme improves the timeliness and the robustness of model updating, and the efficiency and the accuracy of water quality prediction.
Description
Technical Field
The invention relates to the technical field of water quality monitoring, in particular to a water quality monitoring method, device, equipment and medium based on a multichannel model.
Background
In order to better manage and protect water resources, it is important to monitor water quality. By collecting and analyzing various indexes and data in the water body, the water quality condition and the development trend can be mastered in time, the future water quality change condition can be predicted, and scientific basis is provided for water resource management, pollution control and environment planning. Meanwhile, the water quality monitoring can also improve the disaster prevention and reduction effect, guide the reasonable allocation and use of water resources, and promote the continuous utilization of the water resources and the improvement of ecological environment. Therefore, the water quality monitoring has important significance in protecting water resources, guaranteeing public health, promoting ecological balance and realizing sustainable development.
In the current water quality detection method, detection can be performed through a water quality prediction model, but for different time periods and different environmental conditions, the prediction model needs to be updated continuously, the current updating mode has poor timeliness, high consumption resources, low updating frequency and high uncertainty of model performance.
Disclosure of Invention
In view of the above, the embodiment of the invention provides a water quality monitoring method based on a multichannel model, which aims to solve the technical problems of low accuracy and low efficiency of water quality monitoring caused by low model update timeliness, high resource consumption and high model performance uncertainty in the prior art of water quality monitoring. The method comprises the following steps:
acquiring a plurality of water quality data and a plurality of water gene data corresponding to the water quality data in historical continuous time periods of water samples in different areas, wherein the water gene data are spectrum data of the water samples;
clustering a plurality of water-based data based on the spectral characteristics of each water gene data to obtain a plurality of water gene types;
training a model by taking water quality data corresponding to water-based factors of all the water-based factor types as sample data to obtain a multi-channel water quality prediction model, wherein one channel of the water quality prediction model corresponds to one water-based factor type, and the water quality prediction model comprises a first parameter respectively corresponding to each water-based factor type and a second parameter commonly corresponding to all the water-based factor types;
acquiring the latest water quality data of a water sample in any region at the current time point and the latest water gene data corresponding to the latest water quality data, and determining the latest water gene type corresponding to the latest water gene data;
updating a first parameter and the second parameter corresponding to the latest water gene type in the water quality prediction model by utilizing the latest water quality data to obtain an updated water quality prediction model;
and inputting the water quality data of the regional water sample to be monitored in a first preset time period before the current time point of the regional water sample to be monitored into the updated water quality prediction model to obtain the water quality data of the current time point of the regional water sample to be monitored.
The embodiment of the invention also provides a water quality monitoring device based on the multichannel model, which is used for solving the technical problems of low accuracy and low efficiency of water quality monitoring caused by low model update timeliness, high resource consumption and high model performance uncertainty in the prior art of water quality monitoring. The device comprises:
the first acquisition module is used for acquiring a plurality of water quality data and a plurality of water gene data corresponding to the water quality data in historical continuous time periods of water samples in different areas, wherein the water gene data are spectrum data of the water samples;
the clustering module is used for clustering a plurality of water-based data based on the spectral characteristics of each water gene data to obtain a plurality of water gene types;
the model training module is used for training a model by taking water quality data corresponding to water-based factor types as sample data to obtain a multi-channel water quality prediction model, wherein one channel of the water quality prediction model corresponds to one water-based factor type, and the water quality prediction model comprises a first parameter corresponding to each water-based factor type and a second parameter corresponding to all water-based factor types;
the second acquisition module is used for acquiring the latest water quality data of the current time point of the water sample in any region and the latest water gene data corresponding to the latest water quality data, and determining the latest water gene type corresponding to the latest water gene data;
the model updating module is used for updating the first parameter and the second parameter corresponding to the latest water gene type in the water quality prediction model by utilizing the latest water quality data to obtain an updated water quality prediction model;
the water quality prediction module is used for inputting the water quality data of the water sample in the area to be monitored in the first preset time period before the current time point of the water sample in the area to be monitored into the updated water quality prediction model, and obtaining the water quality data of the water sample in the area to be monitored in the current time point.
The embodiment of the invention also provides computer equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes any water quality monitoring method based on the multi-channel model when executing the computer program so as to solve the technical problems of low water quality monitoring accuracy and low efficiency caused by poor model updating timeliness, high resource consumption and high model performance uncertainty in the prior art.
The embodiment of the invention also provides a computer readable storage medium which stores a computer program for executing any of the water quality monitoring methods based on the multi-channel model, so as to solve the technical problems of low accuracy and low efficiency of water quality monitoring caused by poor model updating timeliness, high resource consumption and high model performance uncertainty in the prior art of water quality monitoring.
Compared with the prior art, the beneficial effects that above-mentioned at least one technical scheme that this description embodiment adopted can reach include at least: acquiring a plurality of water quality data and a plurality of water gene data corresponding to the water quality data in a historical continuous time period of water samples in different areas, wherein the water gene data is spectrum data of the water samples; clustering a plurality of water gene data based on the spectral characteristics of each water gene data to obtain a plurality of water gene types; training a model by taking water quality data corresponding to water-based factor data of all water-based factor types as sample data to obtain a multi-channel water quality prediction model, wherein one channel of the water quality prediction model corresponds to one water-based factor type, and the water quality prediction model comprises a first parameter respectively corresponding to each water-based factor type and a second parameter commonly corresponding to all water-based factor types; acquiring the latest water quality data of the current time point of the water sample in any region and the latest water gene data corresponding to the latest water quality data, and determining the latest water gene type corresponding to the latest water gene data; updating a first parameter and a second parameter corresponding to the latest water gene type in the water quality prediction model by using the latest water quality data to obtain an updated water quality prediction model; and inputting the water quality data of the regional water sample to be monitored in a first preset time period before the current time point of the regional water sample to be monitored into the updated water quality prediction model to obtain the water quality data of the current time point of the regional water sample to be monitored. According to the method, water quality data and water gene data of different areas are collected, the water-based data are classified based on spectral characteristics of the water gene data to obtain a plurality of water-based factor types, a multi-channel water quality prediction model corresponding to the water-based factor types is built by combining historical water quality data, and when updating is carried out, new water-based data are utilized to determine the water-based factor types corresponding to the new water-based factor types, so that model parameters corresponding to the water-based factor types in the model are updated. Therefore, when the model is updated, only a single model parameter corresponding to the water-based factor type is required to be updated, the model can be updated in time according to different changes of the environment, the timeliness of the model updating is improved, the updating time cost is effectively reduced, and meanwhile the robustness of the model updating is guaranteed. And the updated model is used for water quality prediction, so that the efficiency and accuracy of water quality prediction are improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a water quality monitoring method based on a multi-channel model provided by an embodiment of the invention;
FIG. 2 is a schematic diagram of a multi-channel water quality prediction model provided by an embodiment of the present invention;
FIG. 3 is a block diagram of a computer device according to an embodiment of the present invention;
FIG. 4 is a block diagram of a water quality monitoring device based on a multi-channel model according to an embodiment of the present invention.
Detailed Description
Embodiments of the present application are described in detail below with reference to the accompanying drawings.
Other advantages and effects of the present application will become apparent to those skilled in the art from the present disclosure, when the following description of the embodiments is taken in conjunction with the accompanying drawings. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present application. The present application may be embodied or carried out in other specific embodiments, and the details of the present application may be modified or changed from various points of view and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments may be combined with each other without conflict. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.
At present, for updating the water quality prediction model, there are two main modes of retraining and fine tuning. The model is retrained, the time cost is high, the required computing resource is large, the updating frequency is low, the timeliness is poor, and the uncertainty of the model performance is high. The mode of fine tuning can effectively reduce the updated time cost and calculate the resource quantity, but the existing fine tuning mainly adjusts the regression part of the model, has high uncertainty in water quality prediction after environmental change, and greatly reduces the prediction accuracy.
In an embodiment of the present invention, in order to solve the problems in the prior art, a water quality monitoring method based on a multi-channel model is provided, as shown in fig. 1, and the method includes the following steps:
s101, acquiring a plurality of water quality data and a plurality of water gene data corresponding to the water quality data in historical continuous time periods of water samples in different areas, wherein the water gene data are spectrum data of the water samples;
step S102, clustering a plurality of water-based data based on the spectral characteristics of each water gene data to obtain a plurality of water gene types;
step S103, training a model by taking water quality data corresponding to water-based factor types of all the water-based factors as sample data to obtain a multi-channel water quality prediction model, wherein one channel of the water quality prediction model corresponds to one water-based factor type, and the water quality prediction model comprises a first parameter corresponding to each water-based factor type and a second parameter corresponding to all the water-based factor types;
step S104, acquiring the latest water quality data of the current time point of the water sample in any region and the latest water gene data corresponding to the latest water quality data, and determining the latest water gene type corresponding to the latest water gene data;
step S105, updating a first parameter and a second parameter corresponding to the latest water gene type in the water quality prediction model by utilizing the latest water quality data to obtain an updated water quality prediction model;
and S106, inputting the water quality data of the regional water sample to be monitored in a first preset time period before the current time point of the regional water sample to be monitored into the updated water quality prediction model, and obtaining the water quality data of the regional water sample to be monitored in the current time point.
According to the method, water quality data and water gene data of different areas are collected, the water-based data are classified based on spectral characteristics of the water gene data to obtain a plurality of water-based factor types, a multi-channel water quality prediction model corresponding to the water-based factor types is built by combining historical water quality data, and when updating is carried out, new water-based data are utilized to determine the water-based factor types corresponding to the new water-based factor types, so that model parameters corresponding to the water-based factor types in the model are updated. Therefore, when the model is updated, only a single model parameter corresponding to the water-based factor type is required to be updated, the model can be updated in time according to different changes of the environment, the timeliness of the model updating is improved, the updating time cost is effectively reduced, and meanwhile the robustness of the model updating is guaranteed. And the updated model is used for water quality prediction, so that the efficiency and accuracy of water quality prediction are improved.
In particular, water quality data of different regions, historical continuous time periods and water-based data corresponding to the water quality data are collected, for example, the water quality data are expressed as x [ x1, x2, …, xi, … xm ], the water-based data corresponding to the water quality data are expressed as p [ p1, p2, …, pi, … pm ], i is the data at the i-th moment, the water gene data are the spectral data of the water sample at 200-710nm, and the spectral data detect a value every 2nm, so each spectral data is 256 dimensions, i.e., pi= [ pi200, pi202, …, pij, …, pi710], and j represents the wavelength.
In one embodiment, the clustering of the plurality of water-based data based on the spectral characteristics of each of the water gene data to obtain a plurality of water gene types includes the steps of:
after preprocessing each water-based data, extracting spectral characteristics of each water-based data;
constructing a spectrum characteristic matrix based on each water gene data and the spectrum characteristic corresponding to each water-based data, and obtaining a plurality of spectrum characteristic matrixes corresponding to the plurality of water gene data one by one;
clustering a plurality of the spectrum feature matrixes, and taking each clustered category as one water-based factor type.
In specific implementation, the water-based data p [ p1, p2, …, pi, … pm ] are preprocessed, and spectral characteristics of each water-based data are extracted respectively, wherein the spectral characteristics comprise a peak position lambda M, an absorbance value rho M of a peak, an absorption left shoulder position lambda L, an absorption right shoulder position lambda R, an absorbance value rho L of an absorption left shoulder, an absorbance value rho R of an absorption right shoulder and an absorption intensity H of the peak; when constructing a spectral feature matrix, constructing a spectral feature matrix gi by taking a water-based data pi and lambda M, rho M, lambda L, lambda R, rho L, rho R and H of the water-based data as a group of data; and taking the spectral feature matrixes g1, g2, …, gi, … and gm corresponding to each water-based data as input, clustering the water-based data by using a clustering model, and taking one class after clustering as a water-based factor type. For example, in this embodiment, m water-based factors are clustered to obtain n water-based factor types, namely water-based factor type 1, water-based factor type 2, water-based factor type n-1, and water-based factor type n, where n is less than m. In this embodiment, the waveform characteristics of the water-based data curve are clustered, and the matrices with similar waveforms are classified into the same type.
In one embodiment, the determining the latest water gene type corresponding to the latest water gene data includes:
constructing corresponding relations between a plurality of spectral feature matrixes and a plurality of water gene types by using an SVM method;
acquiring a latest spectrum characteristic matrix corresponding to the latest water gene data;
and determining the latest water gene type corresponding to the latest spectral feature matrix based on the corresponding relation.
In specific implementation, the corresponding relation between a plurality of spectral feature matrixes and a plurality of water gene types is constructed by utilizing an SVM method, and the step is to construct a water gene type identification model. Specifically, by using an SVM method, a plurality of spectral feature matrices corresponding to a water-based factor type are used as one input sample data of a model, the water-based factor type is used as one output sample data of the model, all the water-based factor types and the corresponding spectral feature matrices are used as sample data for model training to train the model, and a water-based factor type identification model is obtained. The water factor type identification model can be used for identifying the water factor type of any water factor data and preparing for subsequent water quality prediction.
The SVM (support vector machines, SVM) support vector machine is a two-class model, and the basic idea of the SVM algorithm is to map data into a high-dimensional space and find a hyperplane in the space so that the distance from various data points to the hyperplane is maximized. Specifically, for a given training data set, the SVM determines the best decision boundary by calculating the distance between each sample point and the hyperplane. In order to avoid overfitting and improve generalization performance, the SVM also introduces a kernel function, which can map linear inseparable data into a high-dimensional space, thereby realizing nonlinear classification.
The SVM method mainly comprises the following steps:
(1) Data preprocessing: the method comprises the steps of data cleaning, feature extraction, feature scaling and the like;
(2) Feature mapping: mapping data into a high-dimensional space using a kernel function;
(3) Calculating a hyperplane: searching a hyperplane in the high-dimensional space, so that the distance from each data point to the hyperplane is the largest;
(4) And (3) predicting: the new samples are classified using the learned model.
In one embodiment, the training the model by using the water quality data corresponding to all the water-based factor type water-based data as sample data to obtain a multi-channel water quality prediction model includes:
for each time point, taking the water quality data of a second preset time period before the time point as one input data of a water quality prediction model to be trained, taking the water quality data of the time point as one output data of the water quality prediction model to be trained, wherein the one input data and the one output data form sample data of the water quality data of the time point;
obtaining sample data of water quality data at all time points in the historical continuous time period;
dividing the sample data of the water quality data at all time points into a plurality of groups of sample data based on each water factor type, wherein one water factor type corresponds to one group of sample data;
and respectively inputting each group of sample data into one channel of the water quality prediction model to be trained for training, and obtaining the water quality prediction model of multiple channels.
In the specific implementation, water quality data q [ q1, q2, …, qi, … qm ] in the historical continuous time period are taken as an example, and one water quality data is taken every hour, namely qi is the water quality data of the ith hour in the historical continuous time period.
In this embodiment, the second preset period is set to 4 hours, the continuous water quality data is slid to construct input data, the water quality data of the next hour is predicted by the water quality data of the first 4 hours, that is, the water quality data of the continuous 4 hours is used as one input data, the water quality data of the next hour of the continuous 4 hours is used as one input data, for example, [ q1, q2, q3, q4] is one input data, and [ q5] is the corresponding output data, that is, x= [ q1, q2, q3, q4], and y= [ q5] is one sample data corresponding to the water quality data of one time point.
The input data and the output data for all hours are constructed in a sliding manner, and are specifically shown as follows:
x=[[q1,q2,q3,q4],[q2,q3,q4,q5],…,[qt-4,qt-3,qt-2,qt-1],…,[qm-4,qm-3,qm-2,qm-1]]；
y = [q5,q6,…,qt,…,qm]。
by processing the water quality data, input data and output data of the water quality data at all time points in the historical continuous time period are obtained, and all sample data for training of the water quality prediction model are obtained.
Then, according to the water gene data included in each water gene type, the sample data corresponding to the water quality data corresponding to the water gene data in the same water gene type is used as a group of sample data, so as to obtain the sample data of the water quality data corresponding to each water gene type respectively, for example, the number of the water gene types is n, and n groups of sample data of the water quality data are obtained, and the method is shown in fig. 2. The obtained n groups of sample data corresponding to the water-based factor types are respectively input into n channels of a model to be trained for model training, fig. 2 is a schematic structural diagram of a water quality prediction model, each box on the left side of the diagram represents a first parameter corresponding to each water-based factor type one by one, the first parameter is a unique parameter of water quality data under each water-based factor type, the large box on the right side of the diagram represents a second parameter corresponding to all water-based factor types together, and the second parameter is a common parameter of water quality data under all water-based factor types.
In the specific implementation, in the step of inputting the water quality data of the to-be-monitored area water sample in the first preset time period before the current time point of the to-be-monitored area water sample into the updated water quality prediction model to obtain the water quality data of the to-be-monitored area water sample at the current time point, the first preset time period can be set to be 4 hours, namely, 4 small data before the current time point of the to-be-monitored area water sample is used as input data and is input into the water quality prediction model, so that the water quality data of the to-be-monitored area water sample at the current time point can be obtained, and the monitoring of the to-be-monitored area water sample is realized.
In one embodiment, the method further comprises:
searching null values existing in a plurality of water quality data in the historical continuous time period, and obtaining a first null value position;
detecting abnormal values in a plurality of water quality data in the historical continuous time period by adopting a random forest abnormal value detection method, and eliminating the abnormal values to obtain a second null value position;
filling the first null position by using the average value of a plurality of water quality data adjacent to the first null position, and filling the second null position by using the average value of a plurality of water quality data adjacent to the second null position.
In this embodiment, the water quality data is preprocessed to remove abnormal data, and the empty data is filled to ensure the continuity of the data. In general, before the steps of sliding construction of input data and output data for all hours, pretreatment of water quality data is performed, construction of sample data is performed after continuous effective data is obtained, effectiveness of the data is guaranteed, and accuracy of a water quality prediction model is further improved.
In one embodiment, said pre-processing each of said water-based data comprises:
and normalizing each water-based data.
In specific implementation, water gene data p [ p1, p2, …, pi, … pm ] are taken as an example for illustration, one of the water gene data pi is selected, and the water gene data pi is normalized, and the normalized calculation formula is as follows:
Pij=（pij-min(pij)）/(max(pij)-min(pij)),
pij is a spectral value at a single wavelength after normalization of the water gene data Pi, i.e., pi, pi= [ Pi200, pi202, …, pij, …, pi710] is expressed after normalization of the water gene data Pi. The water gene data P [ P1, P2, …, pi, … Pm ] are normalized to P [ P1, P2, …, pi, … Pm ], and then the spectral feature matrix is constructed based on the water gene data P [ P1, P2, …, pi, … Pm ].
In one embodiment, the spectral features include: the position of the peak, the absorbance value of the peak, the position of the left absorption shoulder, the position of the right absorption shoulder, the absorbance value of the left absorption shoulder, the absorbance value of the right absorption shoulder and the absorption intensity of the peak.
According to the water quality monitoring method based on the multi-channel model, the water quality data and the water gene data of different areas are collected, the water-based factor data are classified based on the spectral characteristics of the water gene data to obtain a plurality of water-based factor types, the multi-channel water quality prediction model corresponding to the water-based factor types is built by combining the historical water quality data, and when updating is carried out, the new water-based factor data are utilized to determine the water-based factor type corresponding to the new water-based factor type so as to update model parameters corresponding to the water-based factor type in the model. Therefore, when the model is updated, only a single model parameter corresponding to the water-based factor type is required to be updated, the model can be updated in time according to different changes of the environment, the timeliness of the model updating is improved, the updating time cost is effectively reduced, and meanwhile the robustness of the model updating is guaranteed. And the updated model is used for water quality prediction, so that the efficiency and accuracy of water quality prediction are improved.
In this embodiment, a computer device is provided, as shown in fig. 3, including a memory 301, a processor 302, and a computer program stored in the memory and capable of running on the processor, where the processor implements any of the above-mentioned multi-channel model-based water quality monitoring methods when executing the computer program.
In particular, the computer device may be a computer terminal, a server or similar computing means.
In this embodiment, a computer-readable storage medium storing a computer program for executing any of the above water quality monitoring methods is provided.
In particular, computer-readable storage media, including both permanent and non-permanent, removable and non-removable media, may be used to implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer-readable storage media include, but are not limited to, phase-change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable storage media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
Based on the same inventive concept, the embodiment of the invention also provides a water quality monitoring device based on a multi-channel model, as described in the following embodiment. Because the principle of solving the problem of the water quality monitoring device based on the multi-channel model is similar to that of the water quality monitoring method based on the multi-channel model, the implementation of the water quality monitoring device based on the multi-channel model can be referred to the implementation of the water quality monitoring method based on the multi-channel model, and the repeated parts are not repeated. As used below, the term "unit" or "module" may be a combination of software and/or hardware that implements the intended function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
FIG. 4 is a block diagram of a multi-channel model-based water quality monitoring device according to an embodiment of the present invention, as shown in FIG. 4, comprising: the first acquisition module 401, the clustering module 402, the model training module 403, the second acquisition module 404, the model updating module 405, and the water quality prediction module 406 are described below.
The first acquisition module is used for acquiring a plurality of water quality data and a plurality of water gene data corresponding to the water quality data in historical continuous time periods of water samples in different areas, wherein the water gene data are spectrum data of the water samples;
the clustering module is used for clustering a plurality of water-based data based on the spectral characteristics of each water gene data to obtain a plurality of water gene types;
the model training module is used for training a model by taking water quality data corresponding to water-based factor types as sample data to obtain a multi-channel water quality prediction model, wherein one channel of the water quality prediction model corresponds to one water-based factor type, and the water quality prediction model comprises a first parameter corresponding to each water-based factor type and a second parameter corresponding to all water-based factor types;
the second acquisition module is used for acquiring the latest water quality data of the current time point of the water sample in any region and the latest water gene data corresponding to the latest water quality data, and determining the latest water gene type corresponding to the latest water gene data;
the model updating module is used for updating the first parameter and the second parameter corresponding to the latest water gene type in the water quality prediction model by utilizing the latest water quality data to obtain an updated water quality prediction model;
the water quality prediction module is used for inputting the water quality data of the water sample in the area to be monitored in the first preset time period before the current time point of the water sample in the area to be monitored into the updated water quality prediction model, and obtaining the water quality data of the water sample in the area to be monitored in the current time point.
In one embodiment, the clustering module 402 is further configured to:
after preprocessing each water-based data, extracting spectral characteristics of each water-based data;
constructing a spectrum characteristic matrix based on each water gene data and the spectrum characteristic corresponding to each water-based data, and obtaining a plurality of spectrum characteristic matrixes corresponding to the plurality of water gene data one by one;
clustering a plurality of the spectrum feature matrixes, and taking each clustered category as one water-based factor type.
In one embodiment, the second obtaining module 404 is further configured to:
constructing corresponding relations between a plurality of spectral feature matrixes and a plurality of water gene types by using an SVM method;
acquiring a latest spectrum characteristic matrix corresponding to the latest water gene data;
and determining the latest water gene type corresponding to the latest spectral feature matrix based on the corresponding relation.
In one embodiment, the model training module 403 is further configured to:
for each time point, taking the water quality data of a second preset time period before the time point as one input data of a water quality prediction model to be trained, taking the water quality data of the time point as one output data of the water quality prediction model to be trained, wherein the one input data and the one output data form sample data of the water quality data of the time point;
obtaining sample data of water quality data at all time points in the historical continuous time period;
dividing the sample data of the water quality data at all time points into a plurality of groups of sample data based on each water factor type, wherein one water factor type corresponds to one group of sample data;
and respectively inputting each group of sample data into one channel of the water quality prediction model to be trained for training, and obtaining the water quality prediction model of multiple channels.
In one embodiment, the multi-channel model-based water quality monitoring device further comprises:
the searching module is used for searching null values existing in a plurality of water quality data in the historical continuous time period and acquiring a first null value position;
the abnormal value eliminating module is used for detecting abnormal values in the plurality of water quality data in the historical continuous time period by adopting a random forest abnormal value detection method and eliminating the abnormal values to obtain a second null value position;
and the numerical value filling module is used for filling the first null value position by using the average value of the plurality of water quality data adjacent to the first null value position and filling the second null value position by using the average value of the plurality of water quality data adjacent to the second null value position.
In one embodiment, the clustering module 402 is further configured to:
and normalizing each water-based data.
In one embodiment, the clustering module 402 is further configured to:
the spectral features include: the position of the peak, the absorbance value of the peak, the position of the left absorption shoulder, the position of the right absorption shoulder, the absorbance value of the left absorption shoulder, the absorbance value of the right absorption shoulder and the absorption intensity of the peak.
The embodiment of the invention realizes the following technical effects: acquiring a plurality of water quality data and a plurality of water gene data corresponding to the water quality data in a historical continuous time period of water samples in different areas, wherein the water gene data is spectrum data of the water samples; clustering a plurality of water gene data based on the spectral characteristics of each water gene data to obtain a plurality of water gene types; training a model by taking water quality data corresponding to water-based factor data of all water-based factor types as sample data to obtain a multi-channel water quality prediction model, wherein one channel of the water quality prediction model corresponds to one water-based factor type, and the water quality prediction model comprises a first parameter respectively corresponding to each water-based factor type and a second parameter commonly corresponding to all water-based factor types; acquiring the latest water quality data of the current time point of the water sample in any region and the latest water gene data corresponding to the latest water quality data, and determining the latest water gene type corresponding to the latest water gene data; updating a first parameter and a second parameter corresponding to the latest water gene type in the water quality prediction model by using the latest water quality data to obtain an updated water quality prediction model; and inputting the water quality data of the regional water sample to be monitored in a first preset time period before the current time point of the regional water sample to be monitored into the updated water quality prediction model to obtain the water quality data of the current time point of the regional water sample to be monitored. According to the method, water quality data and water gene data of different areas are collected, the water-based data are classified based on spectral characteristics of the water gene data to obtain a plurality of water-based factor types, a multi-channel water quality prediction model corresponding to the water-based factor types is built by combining historical water quality data, and when updating is carried out, new water-based data are utilized to determine the water-based factor types corresponding to the new water-based factor types, so that model parameters corresponding to the water-based factor types in the model are updated. Therefore, when the model is updated, only a single model parameter corresponding to the water-based factor type is required to be updated, the model can be updated in time according to different changes of the environment, the timeliness of the model updating is improved, the updating time cost is effectively reduced, and meanwhile the robustness of the model updating is guaranteed. And the updated model is used for water quality prediction, so that the efficiency and accuracy of water quality prediction are improved.
It will be apparent to those skilled in the art that the modules or steps of the embodiments of the invention described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may alternatively be implemented in program code executable by computing devices, so that they may be stored in a storage device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than what is shown or described, or they may be separately fabricated into individual integrated circuit modules, or a plurality of modules or steps in them may be fabricated into a single integrated circuit module. Thus, embodiments of the invention are not limited to any specific combination of hardware and software.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, and various modifications and variations can be made to the embodiments of the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A water quality monitoring method based on a multichannel model is characterized by comprising the following steps:
acquiring a plurality of water quality data and a plurality of water gene data corresponding to the water quality data in historical continuous time periods of water samples in different areas, wherein the water gene data are spectrum data of the water samples;
clustering a plurality of water-based data based on the spectral characteristics of each water gene data to obtain a plurality of water gene types;
training a model by taking water quality data corresponding to water-based factors of all the water-based factor types as sample data to obtain a multi-channel water quality prediction model, wherein one channel of the water quality prediction model corresponds to one water-based factor type, and the water quality prediction model comprises a first parameter respectively corresponding to each water-based factor type and a second parameter commonly corresponding to all the water-based factor types;
acquiring the latest water quality data of a water sample in any region at the current time point and the latest water gene data corresponding to the latest water quality data, and determining the latest water gene type corresponding to the latest water gene data;
updating a first parameter and the second parameter corresponding to the latest water gene type in the water quality prediction model by utilizing the latest water quality data to obtain an updated water quality prediction model;
and inputting the water quality data of the regional water sample to be monitored in a first preset time period before the current time point of the regional water sample to be monitored into the updated water quality prediction model to obtain the water quality data of the current time point of the regional water sample to be monitored.
2. The multi-channel model based water quality monitoring method of claim 1, wherein the clustering the plurality of water-based data based on the spectral characteristics of each of the water gene data to obtain a plurality of water gene types comprises:
after preprocessing each water-based data, extracting spectral characteristics of each water-based data;
constructing a spectrum characteristic matrix based on each water gene data and the spectrum characteristic corresponding to each water-based data, and obtaining a plurality of spectrum characteristic matrixes corresponding to the plurality of water gene data one by one;
clustering a plurality of the spectrum feature matrixes, and taking each clustered category as one water-based factor type.
3. The multi-channel model based water quality monitoring method of claim 2, wherein the determining the latest water gene type corresponding to the latest water gene data comprises:
constructing corresponding relations between a plurality of spectral feature matrixes and a plurality of water gene types by using an SVM method;
acquiring a latest spectrum characteristic matrix corresponding to the latest water gene data;
and determining the latest water gene type corresponding to the latest spectral feature matrix based on the corresponding relation.
4. The multi-channel model-based water quality monitoring method of claim 1, wherein training the model with water quality data corresponding to all the water-based factor types of water-based factors as sample data to obtain a multi-channel water quality prediction model comprises:
for each time point, taking the water quality data of a second preset time period before the time point as one input data of a water quality prediction model to be trained, taking the water quality data of the time point as one output data of the water quality prediction model to be trained, wherein the one input data and the one output data form sample data of the water quality data of the time point;
obtaining sample data of water quality data at all time points in the historical continuous time period;
dividing the sample data of the water quality data at all time points into a plurality of groups of sample data based on each water factor type, wherein one water factor type corresponds to one group of sample data;
and respectively inputting each group of sample data into one channel of the water quality prediction model to be trained for training, and obtaining the water quality prediction model of multiple channels.
5. The multi-channel model based water quality monitoring method of claim 1, further comprising:
searching null values existing in a plurality of water quality data in the historical continuous time period, and obtaining a first null value position;
detecting abnormal values in a plurality of water quality data in the historical continuous time period by adopting a random forest abnormal value detection method, and eliminating the abnormal values to obtain a second null value position;
filling the first null position by using the average value of a plurality of water quality data adjacent to the first null position, and filling the second null position by using the average value of a plurality of water quality data adjacent to the second null position.
6. The multi-channel model based water quality monitoring method of claim 2, wherein said preprocessing each of said water-based data comprises:
and normalizing each water-based data.
7. The multi-channel model based water quality monitoring method of claim 2, wherein the spectral features include: the position of the peak, the absorbance value of the peak, the position of the left absorption shoulder, the position of the right absorption shoulder, the absorbance value of the left absorption shoulder, the absorbance value of the right absorption shoulder and the absorption intensity of the peak.
8. A water quality monitoring device based on a multi-channel model, comprising:
the first acquisition module is used for acquiring a plurality of water quality data and a plurality of water gene data corresponding to the water quality data in historical continuous time periods of water samples in different areas, wherein the water gene data are spectrum data of the water samples;
the clustering module is used for clustering a plurality of water-based data based on the spectral characteristics of each water gene data to obtain a plurality of water gene types;
the model training module is used for training a model by taking water quality data corresponding to water-based factor types as sample data to obtain a multi-channel water quality prediction model, wherein one channel of the water quality prediction model corresponds to one water-based factor type, and the water quality prediction model comprises a first parameter corresponding to each water-based factor type and a second parameter corresponding to all water-based factor types;
the second acquisition module is used for acquiring the latest water quality data of the current time point of the water sample in any region and the latest water gene data corresponding to the latest water quality data, and determining the latest water gene type corresponding to the latest water gene data;
the model updating module is used for updating the first parameter and the second parameter corresponding to the latest water gene type in the water quality prediction model by utilizing the latest water quality data to obtain an updated water quality prediction model;
the water quality prediction module is used for inputting the water quality data of the water sample in the area to be monitored in the first preset time period before the current time point of the water sample in the area to be monitored into the updated water quality prediction model, and obtaining the water quality data of the water sample in the area to be monitored in the current time point.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the multichannel model based water quality monitoring method of any of claims 1 to 7 when the computer program is executed.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program for executing the multichannel model-based water quality monitoring method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410130354.1A CN117686447B (en) | 2024-01-31 | 2024-01-31 | Water quality monitoring method, device, equipment and medium based on multichannel model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410130354.1A CN117686447B (en) | 2024-01-31 | 2024-01-31 | Water quality monitoring method, device, equipment and medium based on multichannel model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117686447A true CN117686447A (en) | 2024-03-12 |
| CN117686447B CN117686447B (en) | 2024-05-03 |
Family
ID=90135612
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410130354.1A Active CN117686447B (en) | 2024-01-31 | 2024-01-31 | Water quality monitoring method, device, equipment and medium based on multichannel model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117686447B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105092493A (en) * | 2015-06-19 | 2015-11-25 | 浙江大学 | Water body COD optical measurement method based on water sample type identification |
| WO2022134391A1 (en) * | 2020-12-25 | 2022-06-30 | 中国科学院西安光学精密机械研究所 | Fusion neuron model, neural network structure and training and inference methods therefor, storage medium, and device |
| CN115810403A (en) * | 2023-01-30 | 2023-03-17 | 北京英视睿达科技股份有限公司 | Method for evaluating water pollution based on environmental characteristic information |
| CN115824993A (en) * | 2023-02-14 | 2023-03-21 | 北京英视睿达科技股份有限公司 | Method and device for determining chemical oxygen demand of water body, computer equipment and medium |
| CN116128039A (en) * | 2023-01-04 | 2023-05-16 | 上海威派格智慧水务股份有限公司 | Construction method and prediction method of surface water quality prediction model |
| CN116362363A (en) * | 2021-12-27 | 2023-06-30 | 腾讯科技（深圳）有限公司 | Water quality prediction method, device, equipment and storage medium |
-
2024
- 2024-01-31 CN CN202410130354.1A patent/CN117686447B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105092493A (en) * | 2015-06-19 | 2015-11-25 | 浙江大学 | Water body COD optical measurement method based on water sample type identification |
| WO2022134391A1 (en) * | 2020-12-25 | 2022-06-30 | 中国科学院西安光学精密机械研究所 | Fusion neuron model, neural network structure and training and inference methods therefor, storage medium, and device |
| CN116362363A (en) * | 2021-12-27 | 2023-06-30 | 腾讯科技（深圳）有限公司 | Water quality prediction method, device, equipment and storage medium |
| CN116128039A (en) * | 2023-01-04 | 2023-05-16 | 上海威派格智慧水务股份有限公司 | Construction method and prediction method of surface water quality prediction model |
| CN115810403A (en) * | 2023-01-30 | 2023-03-17 | 北京英视睿达科技股份有限公司 | Method for evaluating water pollution based on environmental characteristic information |
| CN115824993A (en) * | 2023-02-14 | 2023-03-21 | 北京英视睿达科技股份有限公司 | Method and device for determining chemical oxygen demand of water body, computer equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117686447B (en) | 2024-05-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wen et al. | Big data driven marine environment information forecasting: a time series prediction network | |
| CN101841435B (en) | Method, apparatus and system for detecting abnormality of DNS (domain name system) query flow | |
| Yu et al. | Trajectory similarity clustering based on multi-feature distance measurement | |
| Carmona et al. | Overview on evolutionary subgroup discovery: analysis of the suitability and potential of the search performed by evolutionary algorithms | |
| CN110503256A (en) | Short-term load forecasting method and system based on big data technology | |
| Song et al. | A match‐then‐predict method for daily traffic flow forecasting based on group method of data handling | |
| CN115952429A (en) | Self-adaptive DBSCAN abnormal battery identification method based on Euclidean distance without prior weight | |
| CN117686447B (en) | Water quality monitoring method, device, equipment and medium based on multichannel model | |
| CN117517230A (en) | Water quality monitoring method, device, equipment and medium based on ultraviolet-visible spectrum | |
| CN117664888B (en) | Water quality monitoring method, device, equipment and medium based on water quality prediction model library | |
| CN111209105A (en) | Capacity expansion processing method, capacity expansion processing device, capacity expansion processing equipment and readable storage medium | |
| CN110131593A (en) | Pipe leakage assisted detection system based on big data platform | |
| CN105701027B (en) | The prediction technique and prediction meanss of data storage capacity | |
| CN114925905A (en) | Industrial energy consumption allocation method, equipment and medium based on identification analysis | |
| Guerrero et al. | Intelligent Information System as a Tool to Reach Unaproachable Goals for Inspectors | |
| Guo | Modeling short-term energy load with continuous conditional random fields | |
| CN104574137A (en) | Method for generating expected probability distribution data of business data | |
| CN113673574B (en) | Soft measurement method, device and medium for water outlet variable prediction | |
| Liu et al. | Short-term Load Forecasting Approach with SVM and Similar Days Based on United Data Mining Technology | |
| Al Junaibi et al. | Evaluating skills dimensions: Case study on occupational changes in the UAE | |
| CN112990362B (en) | Mine car driving grade recognition model training method and device and terminal equipment | |
| CN117495109B (en) | Power stealing user identification system based on neural network | |
| Khan | Decoding Urban Inequality: The Applications of Machine Learning for Mapping Inequality in Cities of the Global South | |
| Khafaf et al. | Optimal Clustering of Energy Consumers based on Entropy of the Correlation Matrix between Clusters | |
| CN116882563A (en) | Industry prediction method and equipment based on dynamic strategy |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |