CN116735444B - Method and device for detecting concentration of suspended matters in water body based on infrared scattering spectrum - Google Patents
Method and device for detecting concentration of suspended matters in water body based on infrared scattering spectrum Download PDFInfo
- Publication number
- CN116735444B CN116735444B CN202310679526.6A CN202310679526A CN116735444B CN 116735444 B CN116735444 B CN 116735444B CN 202310679526 A CN202310679526 A CN 202310679526A CN 116735444 B CN116735444 B CN 116735444B
- Authority
- CN
- China
- Prior art keywords
- data
- detection model
- concentration detection
- water body
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 title claims abstract description 120
- 238000000034 method Methods 0.000 title claims abstract description 28
- 238000001228 spectrum Methods 0.000 title claims abstract description 24
- 238000001514 detection method Methods 0.000 claims abstract description 177
- 238000012549 training Methods 0.000 claims abstract description 91
- 238000010606 normalization Methods 0.000 claims description 21
- 238000012545 processing Methods 0.000 claims description 15
- 239000007787 solid Substances 0.000 claims description 10
- 238000005070 sampling Methods 0.000 claims description 9
- 238000004140 cleaning Methods 0.000 claims description 7
- 238000012546 transfer Methods 0.000 claims description 6
- 238000010276 construction Methods 0.000 claims description 4
- 238000012216 screening Methods 0.000 claims description 4
- 230000011218 segmentation Effects 0.000 claims description 4
- 238000012795 verification Methods 0.000 claims description 3
- 238000013473 artificial intelligence Methods 0.000 abstract description 6
- 230000006870 function Effects 0.000 description 9
- 238000005259 measurement Methods 0.000 description 4
- 239000000126 substance Substances 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 238000007405 data analysis Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 241000251468 Actinopterygii Species 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 231100000481 chemical toxicant Toxicity 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 239000000356 contaminant Substances 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000013501 data transformation Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 239000010842 industrial wastewater Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000010865 sewage Substances 0.000 description 1
- 239000003440 toxic substance Substances 0.000 description 1
- 238000003911 water pollution Methods 0.000 description 1
- 238000005303 weighing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume, or surface-area of porous materials
- G01N15/06—Investigating concentration of particle suspensions
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
- G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
- G01N21/31—Investigating relative effect of material at wavelengths characteristic of specific elements or molecules, e.g. atomic absorption spectrometry
- G01N21/35—Investigating relative effect of material at wavelengths characteristic of specific elements or molecules, e.g. atomic absorption spectrometry using infrared light
- G01N21/3577—Investigating relative effect of material at wavelengths characteristic of specific elements or molecules, e.g. atomic absorption spectrometry using infrared light for analysing liquids, e.g. polluted water
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The invention relates to the technical field of artificial intelligence, and discloses a method and a device for detecting the concentration of suspended matters in a water body based on infrared scattering spectrum, wherein the method comprises the following steps: performing multi-factor classification on the obtained infrared experimental data of the target water body to obtain classified data of the infrared experimental data; determining concentration indexes corresponding to the classification data according to the infrared experimental data, and generating training data of the target water body according to the concentration indexes and the classification data; normalizing the training data to obtain normalized data of the training data; generating an original concentration detection model of the target water body by using a preset concentration prediction algorithm; training an original concentration detection model by using the normalized data to obtain a suspended matter concentration detection model; and acquiring real-time data of the target water body, and generating the real-time suspended matter concentration of the target water body by using the suspended matter concentration detection model and the real-time data. The invention can improve the efficiency of detecting the suspended matter concentration of the water body.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a method and a device for detecting the concentration of suspended matters in a water body based on infrared scattering spectrum.
Background
As with other water environment contaminants, suspended matter in water can have adverse effects on the water ecosystem. The suspended matters in the water body can cause light attenuation, the depth of a light transmission area is shortened, the vertical layering of heat in the water body is changed, the existence of a large amount of suspended matters is an important factor for limiting fish habitat, meanwhile, the suspended matters in the water can be caused to carry toxic chemical substances due to the discharge of industrial wastewater and sewage, the substances can be suspended in the water for a long time and are not easy to deposit at the bottom of the water, the suspended matters can become potential sources of water pollution in the downstream transportation process along the water flow, the polluted suspended matters threaten the water environment, and the risks are brought to human health.
At present, the filtering weighing method is used for detecting suspended matters in water, time and effort are wasted, the workload is high, the automatic sampler reduces labor consumption to a certain extent, the automatic sampler is limited by the limited storage space of the sampler, in addition, the manual or automatic sampling lacks of the rapid concentration of suspended matters and the acquisition capability of instantaneous information during heavy rain, the peak of the concentration of suspended matters is usually missed, and most of the total suspended matters are transported in the peak period, so that the detection efficiency of the concentration of the suspended matters in water is improved, and the problem to be solved is urgent.
Disclosure of Invention
The invention provides a method and a device for detecting the concentration of suspended matters in a water body based on infrared scattering spectrum, and mainly aims to solve the problem of low efficiency in detecting the concentration of suspended matters in the water body.
In order to achieve the above purpose, the invention provides a method for detecting the concentration of suspended matters in a water body based on infrared scattering spectrum, which comprises the following steps:
acquiring infrared experimental data of a target water body, and classifying the infrared experimental data by multiple factors to obtain classification data of the infrared experimental data;
determining a concentration index corresponding to the classification data according to the infrared experimental data, and generating training data of the target water body according to the concentration index and the classification data;
normalizing the training data to obtain normalized data of the training data;
generating an original concentration detection model of the target water body by using a preset concentration prediction algorithm, wherein the preset concentration prediction algorithm is as follows:
wherein Y is the output value of the original concentration detection model, V j Is the connection weight between the output layer and the jth intermediate layer of the original concentration detection model, W ij Is the weight transferred from the ith input layer to the jth intermediate layer of the original concentration detection model, Y i Is the input value of the jth intermediate layer of the original concentration detection model, B ij Is the bias weight of the ith input layer and the jth intermediate layer, B 2 Is the bias weight of the intermediate layer and the output layer, M is the total number of nodes of the input layer of the original concentration detection model, N is the total number of nodes of the intermediate layer of the original concentration detection model, i is the node identifier of the input layer of the original concentration detection model, j is the node identifier of the intermediate layer of the original concentration detection model, and g (is the intermediate layer transfer function;
training the original concentration detection model by using the normalization data to obtain a suspended matter concentration detection model;
and acquiring real-time data of the target water body, and generating the real-time suspended matter concentration of the target water body by using the suspended matter concentration detection model and the real-time data.
Optionally, the multi-factor classifying the infrared experimental data to obtain classified data of the infrared experimental data includes:
performing data cleaning on the infrared experimental data to obtain standard data of the infrared experimental data;
performing field segmentation on the standard data to obtain segmented data of the standard data;
performing field screening on the segmented data to obtain target data of the segmented data;
And carrying out multi-factor classification on the target data to obtain factor data of the target data, and determining the factor data as classification data of the infrared experimental data.
Optionally, the multi-factor classifying the target data to obtain factor data of the target data includes:
generating a target object set of the target data according to a preset clustering parameter threshold and a preset distance algorithm, wherein the preset distance algorithm is as follows:
wherein dist is the distance between the target data, x a Is the a element, y of the selected target data a Is the a-th element of the target data corresponding to the selected target data, a is the element identification of the selected target data, and a is the total number of elements of the selected target data;
and selecting target objects in the target object set one by one as core objects, and generating factor data of the target data by using the core objects and the target object set.
Optionally, the determining, according to the infrared experimental data, a concentration index corresponding to the classification data includes:
generating an association value of the classification data according to the infrared experimental data, and determining a unique factor identifier of the classification data according to the association value;
Determining a unique concentration identifier of the target water body according to the infrared experimental data, and calculating the identifier similarity between the unique concentration identifier and the unique factor identifier;
and generating a concentration index corresponding to the classified data according to the identification similarity and a preset similarity threshold.
Optionally, the generating training data of the target water body according to the concentration index and the classification data includes:
generating a sample set of the target water body according to the corresponding relation between the concentration index and the classification data;
and carrying out layered sampling on the sample set by utilizing a preset sampling proportion to obtain training data of the sample set, wherein the training data comprises a training set and a verification set.
Optionally, the normalizing the training data to obtain normalized data of the training data includes:
normalizing the training data by using the following normalization algorithm to obtain normalized data of the training data:
wherein x' is the normalized data of the training data, y max Is the range maximum value, y, of the data range of the normalized data min Is the range minimum value, x, of the data range of the normalized data max Is the maximum value of the data in the training data, x min Is the minimum value of the data in the training data and x is the training data.
Optionally, the generating the original concentration detection model of the target water body by using a preset concentration prediction algorithm includes:
acquiring a blank model of a target water body, and performing unit configuration on the blank model by using a preset concentration prediction algorithm to obtain a primary unit of the blank model;
sequentially connecting and configuring the primary units to obtain a primary structure of the primary units;
initializing the layer number parameters of the primary structure to obtain a topological structure of the primary structure, and generating an original concentration detection model of the target water body according to the topological structure.
Optionally, training the original concentration detection model by using the normalized data to obtain a suspended matter concentration detection model, including:
s11, carrying out weight initialization on the original concentration detection model to obtain an original concentration detection model of the original concentration detection model;
s12, inputting the normalized data into the initial concentration detection model to obtain an initial intermediate value of the initial concentration detection model, and generating an initial output value of the initial concentration detection model according to the initial intermediate value;
S13, generating an output error mean square value of the initial concentration detection model according to the initial output value;
s14, when the mean square value of the output error is greater than or equal to a preset mean square value threshold value of the error, generating an intermediate layer error of the initial concentration detection model according to the initial intermediate value, updating the weight of the original concentration detection model by using an error gradient of the intermediate layer error to obtain an updated concentration detection model, and returning to the step S12;
and S15, when the output error mean square value is smaller than a preset error mean square value threshold, determining a suspended matter concentration detection model according to the weight of the original concentration detection model.
Optionally, the generating the mean square value of the output error of the initial concentration detection model according to the initial output value includes:
generating an output error mean square value of the initial concentration detection model according to the initial output value and the following error mean square value algorithm:
wherein MSE is the mean square value of the output errors of the initial concentration detection model, B is the total number of the output values of the initial concentration detection model, B is the identification of the output values of the initial concentration detection model, T b Is the b-th output value, P, of the initial concentration detection model b Is the expected value corresponding to the b-th output value of the initial concentration detection model.
In order to solve the above problems, the present invention further provides a device for detecting the concentration of suspended matters in a water body based on infrared scattering spectrum, the device comprising:
the multi-factor classification module is used for acquiring infrared experimental data of a target water body, and performing multi-factor classification on the infrared experimental data to obtain classification data of the infrared experimental data;
the training data module is used for determining concentration indexes corresponding to the classification data according to the infrared experimental data and generating training data of the target water body according to the concentration indexes and the classification data;
the normalization processing module is used for carrying out normalization processing on the training data to obtain normalized data of the training data;
the model construction module is used for generating an original concentration detection model of the target water body by using a preset concentration prediction algorithm, wherein the preset concentration prediction algorithm is as follows:
wherein Y is the output value of the original concentration detection model, V j Is the connection weight between the output layer and the jth intermediate layer of the original concentration detection model, W ij Is the weight transferred from the ith input layer to the jth intermediate layer of the original concentration detection model, Y i Is the input value of the jth intermediate layer of the original concentration detection model, B ij Is the bias weight of the ith input layer and the jth intermediate layer, B 2 Is the bias weight of the intermediate layer and the output layer, M is the total number of nodes of the input layer of the original concentration detection model, N is the total number of nodes of the intermediate layer of the original concentration detection model, i is the node identifier of the input layer of the original concentration detection model, j is the node identifier of the intermediate layer of the original concentration detection model, and g (is the intermediate layer transfer function;
the model training module is used for training the original concentration detection model by utilizing the normalization data to obtain a suspended matter concentration detection model;
the concentration detection module is used for acquiring real-time data of the target water body and generating the real-time suspended matter concentration of the target water body by utilizing the suspended matter concentration detection model and the real-time data.
According to the method and the device for detecting the suspended solids concentration of the water body based on the infrared scattering spectrum, the infrared experimental data of the target water body is obtained through the infrared detection equipment, the measurement is relatively simple, the infrared experimental data are subjected to multi-factor classification, the classification data and the concentration index of the target water body are determined, the training data of the target water body are determined, the normalization processing is carried out on the training data, the problem that the format and the measurement of the training data are not uniform is solved, the obtained normalization data are utilized to carry out model training on an original concentration detection model of the target water body, the suspended solids concentration detection model is obtained, the precision of the model is improved, and the suspended solids concentration of the target water body is rapidly generated through the suspended solids concentration detection model, so that the problem that the suspended solids concentration detection efficiency of the water body is low can be solved.
Drawings
FIG. 1 is a schematic flow chart of a method for detecting the concentration of suspended matters in a water body based on infrared scattering spectrum according to an embodiment of the invention;
FIG. 2 is a flow chart illustrating multi-factor classification of data according to an embodiment of the present invention;
FIG. 3 is a flow chart of generating concentration indicators according to an embodiment of the present invention;
FIG. 4 is a functional block diagram of a device for detecting the concentration of suspended substances in a water body based on infrared scattering spectrum according to an embodiment of the present invention;
the achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The embodiment of the application provides a water body suspended matter concentration detection method based on infrared scattering spectrum. The main execution body of the method for detecting the concentration of the water body suspended matter based on the infrared scattering spectrum comprises at least one of an electronic device, a server, a terminal and the like, which can be configured to execute the method provided by the embodiment of the application. In other words, the method for detecting the concentration of the water body suspended matters based on the infrared scattering spectrum can be executed by software or hardware installed on the terminal equipment or the server equipment. The service end includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like. The server may be an independent server, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.
Referring to fig. 1, a flow chart of a method for detecting the concentration of suspended substances in a water body based on infrared scattering spectrum according to an embodiment of the invention is shown. In this embodiment, the method for detecting the concentration of the suspended matter in the water body based on the infrared scattering spectrum includes:
s1, acquiring infrared experimental data of a target water body, and classifying the infrared experimental data by multiple factors to obtain classified data of the infrared experimental data.
In the embodiment of the invention, the target water body is the water body needing to be subjected to suspended matter concentration detection; the infrared experimental data refer to experimental data obtained by detecting the concentration of the target water body by using infrared detection equipment, and the infrared experimental data can be infrared wavelength, an included angle between an infrared light and the target water body, water body parameters of the target water body and the like; the classification data is determined based on the distance between the data, and characterizes the relationship between the suspended matter concentration of the target water body and the influencing factors.
In an embodiment of the present invention, referring to fig. 2, the multi-factor classification is performed on the infrared experimental data to obtain classified data of the infrared experimental data, where the classifying data includes:
S21, carrying out data cleaning on the infrared experimental data to obtain standard data of the infrared experimental data;
s22, field segmentation is carried out on the standard data to obtain segmented data of the standard data;
s23, performing field screening on the segmented data to obtain target data of the segmented data;
s24, multi-factor classification is carried out on the target data to obtain factor data of the target data, and the factor data is determined to be classified data of the infrared experimental data.
In detail, since the quality of the data determines whether the data analysis and the data mining can obtain the ideal result, the data cleaning is performed on the infrared experimental data, wherein the data cleaning is to pre-process the infrared experimental data, and the main task of the data cleaning of the infrared experimental data is to improve the usability of the infrared experimental data, namely, to remove data noise, irrelevant data, null values and the like in the infrared experimental data, and to consider the dynamic change of the infrared experimental data, the data cleaning can be performed by using a machine learning technology, namely, the missing and wrong data is modified by using a specific algorithm detection database.
In detail, the field segmentation of the standard data may be performed based on a fusion feature between a background feature of the standard data and a data feature of the standard data; the field screening of the divided data means that data related to concentration influence in the divided data is determined as target data.
In detail, the multi-factor classification of the target data is determined according to the clustering result of the target data, and different clustering categories represent different concentration influencing factors.
In detail, the multi-factor classification is performed on the target data to obtain factor data of the target data, including:
generating a target object set of the target data according to a preset clustering parameter threshold and a preset distance algorithm, wherein the preset distance algorithm is as follows:
wherein dist is the distance between the target data, x a Is the a element, y of the selected target data a Is the a-th element of the target data corresponding to the selected target data, a is the element identification of the selected target data, and a is the total number of elements of the selected target data;
And selecting target objects in the target object set one by one as core objects, and generating factor data of the target data by using the core objects and the target object set.
In detail, the preset clustering parameter threshold is used for judging whether the target data belongs to the target object set, and when the distance between the target data is smaller than the preset clustering parameter threshold, the target data belongs to the target object set; and when the distance between the target data is greater than or equal to the preset clustering parameter threshold value, the target data does not belong to the target object set.
In detail, the step of selecting the target objects in the target object set one by one as the core objects is to classify each target object in the target object set, so as to ensure that the target objects in the target object set are considered.
In detail, the factor data of the target data generated by using the core object and the target object set refers to element judgment on a neighborhood set of the target object, when the target object exists in the neighborhood set, the category of the neighborhood set is consistent with the category of the core object, and when the target object does not exist in the neighborhood set, the category of the neighborhood set is inconsistent with the category of the core object.
Further, factor categories of the target data are generated according to the categories of the core objects, and factor data of the target data are determined according to the factor categories.
S2, determining concentration indexes corresponding to the classified data according to the infrared experimental data, and generating training data of the target water body according to the concentration indexes and the classified data.
In an embodiment of the present invention, referring to fig. 3, the determining, according to the infrared experimental data, a concentration index corresponding to the classification data includes:
s31, generating an association value of the classification data according to the infrared experimental data, and determining a unique factor identifier of the classification data according to the association value;
s32, determining a unique concentration identifier of the target water body according to the infrared experimental data, and calculating the identifier similarity between the unique concentration identifier and the unique factor identifier;
s33, generating a concentration index corresponding to the classified data according to the identification similarity and a preset similarity threshold.
In detail, the association value is used to characterize the association between the classification data, and if the association of a data with a data is stronger than the association of a data with b data, the association value of a data with a data is greater than the association value of a data with b data.
Further, the determining the unique factor identifier of the classification data according to the association value means that the classification data can be subjected to data association according to the size of the association value, and the associated data is subjected to unique identifier, so that the association value determines the unique factor identifier of the classification data.
In detail, the unique concentration identification of the target water body is determined according to the infrared experimental data, because the concentrations of different target water bodies are different, the concentration label can be used for identifying the target water body, and the concentration label is determined to be the unique concentration identification of the target water body.
In detail, the calculating the identification similarity between the unique concentration identification and the unique factor identification may use a euclidean distance formula or a chebyshev distance formula to perform the characterization of the similarity between the unique concentration identification and the unique factor identification; the step of generating the concentration index corresponding to the classified data according to the identification similarity and a preset similarity threshold value means that when the identification similarity is greater than the preset similarity threshold value, a concentration label corresponding to the classified data can be determined, so that the concentration index corresponding to the classified data is determined.
In an embodiment of the present invention, the generating training data of the target water body according to the concentration index and the classification data includes:
generating a sample set of the target water body according to the corresponding relation between the concentration index and the classification data;
and carrying out layered sampling on the sample set by utilizing a preset sampling proportion to obtain training data of the sample set, wherein the training data comprises a training set and a verification set.
In detail, the sample set of the target water body is generated according to the correspondence between the concentration index and the classification data, wherein the correspondence between the concentration index and the classification data refers to the relationship between the input quantity and the output quantity of the target water body, for example: the concentration index corresponding to the C group data is 1, and the concentration index corresponding to the D group data is 2.
In detail, the preset sampling proportion is set according to experience, can be 2:1 or 7:3, and the like, and the hierarchical sampling refers to that the elements of the sample set are divided into a plurality of layers according to the characteristics of the elements, sub-samples are extracted in each layer by simple random extraction or system extraction, and finally the sub-samples are combined to form training data of the sample set.
And S3, carrying out normalization processing on the training data to obtain normalized data of the training data.
In the embodiment of the present invention, the normalization processing is performed on the training data because the training data often have different dimensions and dimension units, which may affect the result of data analysis, so that in order to eliminate the dimension effect between the training data, the normalization processing is performed on the training data to solve the comparability between the training data.
In an embodiment of the present invention, the normalizing the training data to obtain normalized data of the training data includes:
normalizing the training data by using the following normalization algorithm to obtain normalized data of the training data:
wherein x' is the normalized data of the training data, y max Is the range maximum value, y, of the data range of the normalized data min Is the range minimum value, x, of the data range of the normalized data max Is the maximum value of the data in the training data, x min Is the minimum value of the data in the training data and x is the training data.
Typically, the normalized data is limited to a range of (0, 1), i.e., the range minimum value of the data range of the normalized data is 0 and the range maximum value of the data range of the normalized data is 1.
S4, generating an original concentration detection model of the target water body by using a preset concentration prediction algorithm.
In the embodiment of the present invention, the generating the original concentration detection model of the target water body by using a preset concentration prediction algorithm includes:
acquiring a blank model of a target water body, and performing unit configuration on the blank model by using a preset concentration prediction algorithm to obtain a primary unit of the blank model;
sequentially connecting and configuring the primary units to obtain a primary structure of the primary units;
initializing the layer number parameters of the primary structure to obtain a topological structure of the primary structure, and generating an original concentration detection model of the target water body according to the topological structure.
In detail, the unit configuration refers to configuration of an input layer, an output layer and an intermediate layer of the blank model; the sequential connection configuration is configured according to the sequence that data arrives at the middle layer from the input layer and then arrives at the output layer; the number of layers parameter initialization can set the number of nodes of the input layer to be 6, the number of nodes of the middle layer to be 18, and the number of nodes of the output layer to be 1.
In detail, the topology is used to represent the data transformation order of training data, and the processing of the training data by the raw concentration detection model.
In detail, the preset concentration prediction algorithm is as follows:
wherein Y is the output value of the original concentration detection model, V j Is the connection weight between the output layer and the jth intermediate layer of the original concentration detection model, W ij Is the weight transferred from the ith input layer to the jth intermediate layer of the original concentration detection model, Y i Is the input value of the jth intermediate layer of the original concentration detection model, B ij Is the bias weight of the ith input layer and the jth intermediate layer, B 2 Is the bias weight of the intermediate layer and the output layer, M is the total number of nodes of the input layer of the original concentration detection model, N is the total number of nodes of the intermediate layer of the original concentration detection model, i is the node identifier of the input layer of the original concentration detection model, j is the node identifier of the intermediate layer of the original concentration detection model, and g (x is the intermediate layer transfer function).
And S5, training the original concentration detection model by utilizing the normalized data to obtain a suspended matter concentration detection model.
In an embodiment of the present invention, training the original concentration detection model by using the normalized data to obtain a suspended matter concentration detection model includes:
s11, carrying out weight initialization on the original concentration detection model to obtain an original concentration detection model of the original concentration detection model;
S12, inputting the normalized data into the initial concentration detection model to obtain an initial intermediate value of the initial concentration detection model, and generating an initial output value of the initial concentration detection model according to the initial intermediate value;
s13, generating an output error mean square value of the initial concentration detection model according to the initial output value;
s14, when the mean square value of the output error is greater than or equal to a preset mean square value threshold value of the error, generating an intermediate layer error of the initial concentration detection model according to the initial intermediate value, updating the weight of the original concentration detection model by using an error gradient of the intermediate layer error to obtain an updated concentration detection model, and returning to the step S12;
and S15, when the output error mean square value is smaller than a preset error mean square value threshold, determining a suspended matter concentration detection model according to the weight of the original concentration detection model.
In detail, the weight initialization refers to determining an initial weight transmitted from an ith input layer to a jth middle layer of the original concentration detection model, wherein the initial weight changes according to training of the original concentration detection model by the normalized data.
In detail, the initial intermediate value refers to output data of an intermediate layer of the initial concentration detection model, and generating an initial output value of the initial concentration detection model according to the initial intermediate value refers to performing data processing on an output layer of the initial concentration detection model by using the initial intermediate value as an input value to obtain the output value of the initial concentration detection model.
In detail, the generating the mean square value of the output error of the initial concentration detection model according to the initial output value includes:
generating an output error mean square value of the initial concentration detection model according to the initial output value and the following error mean square value algorithm:
wherein MSE is the mean square value of the output errors of the initial concentration detection model, B is the total number of the output values of the initial concentration detection model, B is the identification of the output values of the initial concentration detection model, T b Is the b-th output value, P, of the initial concentration detection model b Is the expected value corresponding to the b-th output value of the initial concentration detection model.
In detail, the intermediate layer error that generates the initial concentration detection model according to the initial intermediate value may also use the error mean square value algorithm described above, take the initial intermediate value as the input of the error mean square value algorithm, obtain the output value of the error mean square value algorithm, and take this output value as the intermediate layer error of the initial concentration detection model.
S6, acquiring real-time data of the target water body, and generating the real-time suspended matter concentration of the target water body by using the suspended matter concentration detection model and the real-time data.
In the embodiment of the invention, the real-time data of the target water body refers to data obtained by detecting the water body of the target water body according to the infrared detection equipment, the real-time data is input into the suspended matter concentration detection model, and the output result is the real-time suspended matter concentration of the target book body.
According to the method, the infrared experimental data of the target water body are obtained through the infrared detection equipment, the measurement is relatively simple, the infrared experimental data are subjected to multi-factor classification, the classification data and the concentration index of the target water body are determined, so that the training data of the target water body are determined, the training data are subjected to normalization processing, the problem that the format and the measurement of the training data are not uniform is solved, the obtained normalization data are utilized to carry out model training on an original concentration detection model of the target water body, a suspended matter concentration detection model is obtained, the precision of the model is improved, and the suspended matter concentration of the target water body is rapidly generated by utilizing the suspended matter concentration detection model, so that the problem of low detection efficiency of suspended matter concentration of the water body can be solved.
Fig. 4 is a functional block diagram of a device for detecting the concentration of suspended matters in a water body based on infrared scattering spectrum according to an embodiment of the present invention.
The device 100 for detecting the concentration of the suspended matters in the water body based on the infrared scattering spectrum can be installed in electronic equipment. Depending on the functions implemented, the device for detecting the concentration of the suspended matter in the water body 100 based on the infrared scattering spectrum may include a multi-factor classification module 101, a training data module 102, a normalization processing module 103, a model construction module 104, a model training module 105 and a concentration detection module 106. The module of the invention, which may also be referred to as a unit, refers to a series of computer program segments, which are stored in the memory of the electronic device, capable of being executed by the processor of the electronic device and of performing a fixed function.
In the present embodiment, the functions concerning the respective modules/units are as follows:
the multi-factor classification module 101 is configured to obtain infrared experimental data of a target water body, perform multi-factor classification on the infrared experimental data, and obtain classification data of the infrared experimental data;
the training data module 102 is configured to determine a concentration index corresponding to the classification data according to the infrared experimental data, and generate training data of the target water body according to the concentration index and the classification data;
The normalization processing module 103 is configured to perform normalization processing on the training data to obtain normalized data of the training data;
the model construction module 104 is configured to generate an original concentration detection model of the target water body by using a preset concentration prediction algorithm, where the preset concentration prediction algorithm is:
wherein Y is the output value of the original concentration detection model, V j Is the output layer and the first of the original concentration detection modelConnection rights between j intermediate layers, W ij Is the weight transferred from the ith input layer to the jth intermediate layer of the original concentration detection model, Y i Is the input value of the jth intermediate layer of the original concentration detection model, B ij Is the bias weight of the ith input layer and the jth intermediate layer, B 2 Is the bias weight of the intermediate layer and the output layer, M is the total number of nodes of the input layer of the original concentration detection model, N is the total number of nodes of the intermediate layer of the original concentration detection model, i is the node identifier of the input layer of the original concentration detection model, j is the node identifier of the intermediate layer of the original concentration detection model, and g (is the intermediate layer transfer function;
the model training module 105 is configured to train the original concentration detection model by using the normalized data to obtain a suspended matter concentration detection model;
The concentration detection module 106 is configured to obtain real-time data of a target water body, and generate a real-time suspended matter concentration of the target water body using the suspended matter concentration detection model and the real-time data.
In the several embodiments provided in the present invention, it should be understood that the disclosed method and apparatus may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be other manners of division when actually implemented.
The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional module in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units can be realized in a form of hardware or a form of hardware and a form of software functional modules.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.
The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
The embodiment of the application can acquire and process the related data based on the artificial intelligence technology. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application that uses a digital computer or a digital computer-controlled machine to simulate, extend and expand human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results.
Furthermore, it is evident that the word "comprising" does not exclude other elements or steps, and that the singular does not exclude a plurality. A plurality of units or means recited in the apparatus claims can also be implemented by means of one unit or means in software or hardware. The terms first, second, etc. are used to denote a name, but not any particular order.
Finally, it should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made to the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention.
Claims (6)
1. The method for detecting the concentration of the water body suspended matters based on the infrared scattering spectrum is characterized by comprising the following steps of:
acquiring infrared experimental data of a target water body, and classifying the infrared experimental data by multiple factors to obtain classification data of the infrared experimental data;
determining a concentration index corresponding to the classification data according to the infrared experimental data, and generating training data of the target water body according to the concentration index and the classification data;
normalizing the training data to obtain normalized data of the training data;
generating an original concentration detection model of the target water body by using a preset concentration prediction algorithm, wherein the preset concentration prediction algorithm is as follows:
wherein Y is the output value of the original concentration detection model, V j Is the connection weight between the output layer and the jth intermediate layer of the original concentration detection model, W ij Is the weight transferred from the ith input layer to the jth intermediate layer of the original concentration detection model, Y i Is the input value of the jth intermediate layer of the original concentration detection model, B ij Is the bias weight of the ith input layer and the jth intermediate layer, B 2 The bias weight is the bias weight of the middle layer and the output layer, M is the total number of nodes of the input layer of the original concentration detection model, N is the total number of nodes of the middle layer of the original concentration detection model, i is the node identifier of the input layer of the original concentration detection model, j is the node identifier of the middle layer of the original concentration detection model, and g is the transfer function of the middle layer;
training the original concentration detection model by using the normalization data to obtain a suspended matter concentration detection model;
acquiring real-time data of a target water body, and generating real-time suspended matter concentration of the target water body by using the suspended matter concentration detection model and the real-time data;
the multi-factor classification is performed on the infrared experimental data to obtain classification data of the infrared experimental data, and the method comprises the following steps:
Performing data cleaning on the infrared experimental data to obtain standard data of the infrared experimental data;
performing field segmentation on the standard data to obtain segmented data of the standard data;
performing field screening on the segmented data to obtain target data of the segmented data;
carrying out multi-factor classification on the target data to obtain factor data of the target data, and determining the factor data as classification data of the infrared experimental data;
the multi-factor classification is performed on the target data to obtain factor data of the target data, including:
generating a target object set of the target data according to a preset clustering parameter threshold and a preset distance algorithm, wherein the preset distance algorithm is as follows:
wherein dist is the distance between the target data, x a Is the a element, y of the selected target data a Is the a-th element of the target data corresponding to the selected target data, a is the element identification of the selected target data, and a is the total number of elements of the selected target data;
selecting target objects in the target object set one by one as core objects, and generating factor data of the target data by using the core objects and the target object set;
Wherein, the determining the concentration index corresponding to the classification data according to the infrared experimental data includes:
generating an association value of the classification data according to the infrared experimental data, and determining a unique factor identifier of the classification data according to the association value;
determining a unique concentration identifier of the target water body according to the infrared experimental data, and calculating the identifier similarity between the unique concentration identifier and the unique factor identifier;
generating a concentration index corresponding to the classified data according to the identification similarity and a preset similarity threshold;
wherein the generating training data of the target water body according to the concentration index and the classification data comprises:
generating a sample set of the target water body according to the corresponding relation between the concentration index and the classification data;
and carrying out layered sampling on the sample set by utilizing a preset sampling proportion to obtain training data of the sample set, wherein the training data comprises a training set and a verification set.
2. The method for detecting the concentration of suspended solids in a water body based on infrared scattering spectra as claimed in claim 1, wherein said normalizing the training data to obtain normalized data of the training data comprises:
Normalizing the training data by using the following normalization algorithm to obtain normalized data of the training data:
wherein x is ′ Is the normalized data of the training data, y max Is the range maximum value, y, of the data range of the normalized data min Is the range minimum value, x, of the data range of the normalized data max Is the maximum value of the data in the training data, x min Is the minimum value of the data in the training data and x is the training data.
3. The method for detecting the concentration of suspended solids in a water body based on infrared scattering spectra as claimed in claim 1, wherein said generating an original concentration detection model of the target water body by using a preset concentration prediction algorithm comprises:
acquiring a blank model of a target water body, and performing unit configuration on the blank model by using a preset concentration prediction algorithm to obtain a primary unit of the blank model;
sequentially connecting and configuring the primary units to obtain a primary structure of the primary units;
initializing the layer number parameters of the primary structure to obtain a topological structure of the primary structure, and generating an original concentration detection model of the target water body according to the topological structure.
4. The method for detecting the concentration of suspended solids in a water body based on infrared scattering spectra as claimed in claim 1, wherein said training said original concentration detection model by using said normalized data to obtain a suspended solids concentration detection model comprises:
s11, carrying out weight initialization on the original concentration detection model to obtain an original concentration detection model of the original concentration detection model;
s12, inputting the normalized data into the initial concentration detection model to obtain an initial intermediate value of the initial concentration detection model, and generating an initial output value of the initial concentration detection model according to the initial intermediate value;
s13, generating an output error mean square value of the initial concentration detection model according to the initial output value;
s14, when the mean square value of the output error is greater than or equal to a preset mean square value threshold value of the error, generating an intermediate layer error of the initial concentration detection model according to the initial intermediate value, updating the weight of the original concentration detection model by using an error gradient of the intermediate layer error to obtain an updated concentration detection model, and returning to the step S12;
and S15, when the output error mean square value is smaller than a preset error mean square value threshold, determining a suspended matter concentration detection model according to the weight of the original concentration detection model.
5. The method for detecting the concentration of suspended solids in water based on infrared scattering spectra as claimed in claim 4, wherein said generating an output error mean square value of said initial concentration detection model from said initial output value comprises:
generating an output error mean square value of the initial concentration detection model according to the initial output value and the following error mean square value algorithm:
wherein MSE is the mean square value of the output errors of the initial concentration detection model, B is the total number of the output values of the initial concentration detection model, B is the identification of the output values of the initial concentration detection model, T b Is the b-th output value, P, of the initial concentration detection model b Is the expected value corresponding to the b-th output value of the initial concentration detection model.
6. An infrared scattering spectrum-based water body suspended matter concentration detection device, which is used for realizing the water body suspended matter concentration detection method as claimed in claim 1, and comprises the following steps:
the multi-factor classification module is used for acquiring infrared experimental data of a target water body, and performing multi-factor classification on the infrared experimental data to obtain classification data of the infrared experimental data;
the training data module is used for determining concentration indexes corresponding to the classification data according to the infrared experimental data and generating training data of the target water body according to the concentration indexes and the classification data;
The normalization processing module is used for carrying out normalization processing on the training data to obtain normalized data of the training data;
the model construction module is used for generating an original concentration detection model of the target water body by using a preset concentration prediction algorithm, wherein the preset concentration prediction algorithm is as follows:
wherein Y is the output value of the original concentration detection model, V j Is the connection weight between the output layer and the jth intermediate layer of the original concentration detection model, W ij Is the weight transferred from the ith input layer to the jth intermediate layer of the original concentration detection model, Y i Is the input value of the jth intermediate layer of the original concentration detection model, B ij Is the bias weight of the ith input layer and the jth intermediate layer, B 2 The bias weight is the bias weight of the middle layer and the output layer, M is the total number of nodes of the input layer of the original concentration detection model, N is the total number of nodes of the middle layer of the original concentration detection model, i is the node identifier of the input layer of the original concentration detection model, j is the node identifier of the middle layer of the original concentration detection model, and g is the transfer function of the middle layer;
the model training module is used for training the original concentration detection model by utilizing the normalization data to obtain a suspended matter concentration detection model;
The concentration detection module is used for acquiring real-time data of the target water body and generating the real-time suspended matter concentration of the target water body by utilizing the suspended matter concentration detection model and the real-time data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310679526.6A CN116735444B (en) | 2023-06-08 | 2023-06-08 | Method and device for detecting concentration of suspended matters in water body based on infrared scattering spectrum |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310679526.6A CN116735444B (en) | 2023-06-08 | 2023-06-08 | Method and device for detecting concentration of suspended matters in water body based on infrared scattering spectrum |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116735444A CN116735444A (en) | 2023-09-12 |
| CN116735444B true CN116735444B (en) | 2024-04-02 |
Family
ID=87916247
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310679526.6A Active CN116735444B (en) | 2023-06-08 | 2023-06-08 | Method and device for detecting concentration of suspended matters in water body based on infrared scattering spectrum |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116735444B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110146455A (en) * | 2019-05-27 | 2019-08-20 | 安徽大学 | Laser spectrum gas concentration measuring method based on deep learning |
| CN112669915A (en) * | 2020-11-06 | 2021-04-16 | 西安理工大学 | Pear nondestructive testing method based on neural network and near infrared spectrum |
| CN112858208A (en) * | 2019-11-27 | 2021-05-28 | 国能生物发电集团有限公司 | Biomass potassium content measurement and modeling method based on infrared spectrum principal component and neural network |
| CN112861413A (en) * | 2019-11-27 | 2021-05-28 | 国能生物发电集团有限公司 | Biomass water content measurement and modeling method based on near infrared spectrum principal component and neural network |
| CN113484257A (en) * | 2021-06-24 | 2021-10-08 | 北京市生态环境监测中心 | Water quality concentration calculation system and method based on neural network and full-spectrum absorbance |
| CN113916817A (en) * | 2021-10-21 | 2022-01-11 | 深圳市中科云驰环境科技有限公司 | Spectroscopy chromaticity online measurement method for urban drinking water |
| CN114923873A (en) * | 2022-05-27 | 2022-08-19 | 江苏蓝创智能科技股份有限公司 | Gas concentration detection method and device, electronic equipment and storage medium |
| KR20230011110A (en) * | 2021-07-13 | 2023-01-20 | 주식회사 엘지화학 | Concentration measuring device and method based on spectra learning |
-
2023
- 2023-06-08 CN CN202310679526.6A patent/CN116735444B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110146455A (en) * | 2019-05-27 | 2019-08-20 | 安徽大学 | Laser spectrum gas concentration measuring method based on deep learning |
| CN112858208A (en) * | 2019-11-27 | 2021-05-28 | 国能生物发电集团有限公司 | Biomass potassium content measurement and modeling method based on infrared spectrum principal component and neural network |
| CN112861413A (en) * | 2019-11-27 | 2021-05-28 | 国能生物发电集团有限公司 | Biomass water content measurement and modeling method based on near infrared spectrum principal component and neural network |
| CN112669915A (en) * | 2020-11-06 | 2021-04-16 | 西安理工大学 | Pear nondestructive testing method based on neural network and near infrared spectrum |
| CN113484257A (en) * | 2021-06-24 | 2021-10-08 | 北京市生态环境监测中心 | Water quality concentration calculation system and method based on neural network and full-spectrum absorbance |
| KR20230011110A (en) * | 2021-07-13 | 2023-01-20 | 주식회사 엘지화학 | Concentration measuring device and method based on spectra learning |
| CN113916817A (en) * | 2021-10-21 | 2022-01-11 | 深圳市中科云驰环境科技有限公司 | Spectroscopy chromaticity online measurement method for urban drinking water |
| CN114923873A (en) * | 2022-05-27 | 2022-08-19 | 江苏蓝创智能科技股份有限公司 | Gas concentration detection method and device, electronic equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 基于实测光谱的杭州湾悬浮物浓度遥感反演模式;王繁;周斌;徐建明;凌在盈;;环境科学(11);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116735444A (en) | 2023-09-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Abarenkov et al. | Protax‐fungi: a web‐based tool for probabilistic taxonomic placement of fungal internal transcribed spacer sequences | |
| CN114816909B (en) | Real-time log detection early warning method and system based on machine learning | |
| CN111881290A (en) | Distribution network multi-source grid entity fusion method based on weighted semantic similarity | |
| CN108921301A (en) | A kind of machine learning model update method and system based on self study | |
| CN111160432A (en) | Automatic classification method and system for panel production defects | |
| CN113869052B (en) | AI-based house address matching method, storage medium and equipment | |
| Albatayneh et al. | Image retraining using TensorFlow implementation of the pretrained inception-v3 model for evaluating gravel road dust | |
| CN113554175B (en) | Knowledge graph construction method and device, readable storage medium and terminal equipment | |
| CN112288247A (en) | Soil heavy metal risk identification method based on space interaction relation | |
| CN111126820A (en) | Electricity stealing prevention method and system | |
| CN111160959A (en) | User click conversion estimation method and device | |
| CN112288700A (en) | Rail defect detection method | |
| CN115311205A (en) | Industrial equipment fault detection method based on pattern neural network federal learning | |
| CN115659966A (en) | Rumor detection method and system based on dynamic heteromorphic graph and multi-level attention | |
| CN109977131A (en) | A kind of house type matching system | |
| CN116932523B (en) | Platform for integrating and supervising third party environment detection mechanism | |
| CN116735444B (en) | Method and device for detecting concentration of suspended matters in water body based on infrared scattering spectrum | |
| CN113726558A (en) | Network equipment flow prediction system based on random forest algorithm | |
| CN113704389A (en) | Data evaluation method and device, computer equipment and storage medium | |
| CN112579777A (en) | Semi-supervised classification method for unlabelled texts | |
| CN112182441A (en) | Method and device for detecting violation data | |
| CN115913992A (en) | Anonymous network traffic classification method based on small sample machine learning | |
| CN114139636B (en) | Abnormal operation processing method and device | |
| CN116842936A (en) | Keyword recognition method, keyword recognition device, electronic equipment and computer readable storage medium | |
| CN114818353A (en) | Train control vehicle-mounted equipment fault prediction method based on fault characteristic relation map |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |