Disclosure of Invention
The present invention has been made in view of the above problems. The invention provides a method for realizing ultrafiltration membrane pollution evaluation through a neural network, which comprises the following steps: preprocessing the original data, training a neural network model by adopting a preprocessed data set, and evaluating the membrane pollution condition under a new working condition based on the obtained neural network model. On one hand, the neural network evaluation model has a self-learning function, so that the condition of large operation effect difference caused by the difference of artificial experience is avoided, the evaluation accuracy is improved, on the other hand, the strict requirement of a theoretical formula on parameters can be avoided, and the method is easy to implement.
According to one aspect of the invention, the method for realizing the pollution evaluation of the ultrafiltration membrane through the neural network is characterized by comprising the following steps:
step S1, the raw data is preprocessed,
step S2, training the neural network model by adopting the preprocessed data set,
and step S3, evaluating the membrane pollution condition under the new working condition based on the obtained neural network model.
Illustratively, the step S1 of preprocessing the raw data includes: missing value processing and abnormal value processing;
illustratively, the missing value processing method includes: deletion method, filling method, etc.;
illustratively, before processing the outlier, further comprising: judging abnormal values; the method for distinguishing the abnormal value mainly comprises the following steps: statistical analysis methods, 3 sigma principles, boxplot analysis methods, and the like;
illustratively, the step S1 of preprocessing the raw data further includes: data standardization, variable screening, and the like;
illustratively, before the preprocessing the raw data, the method further comprises: acquiring ultrafiltration water quality data and other operation data and the like;
illustratively, the acquiring water quality data of ultrafiltration and other operational data includes: acquiring data under different working conditions; acquiring data under different working conditions by sampling, wherein the sampling corresponding variables are selected from last cleaning time, COD, ammonia nitrogen, sludge concentration, inflow rate, temperature, inflow pressure, produced water flow, concentrated water pressure and the like;
illustratively, the step S2 of training the neural network model with the preprocessed data set further includes:
step S21, dividing the preprocessed data set into a training set and a testing set;
step S22, training the neural network model by adopting a training set, and evaluating the established model by adopting a test set;
and S23, repeating S21 and S22 to optimize the parameters of the model, so that the model result is expected, and obtaining the model reaching the model result expectation.
Illustratively, the parameter process of the optimization model of step S23 includes using a gradient descent method.
Wherein, the formula adopted by the gradient descent method is as follows:
wherein theta isiThe pending coefficients of the solution are represented, α the step size, and J (theta) the loss function that minimizes the variance with respect to theta.
Illustratively, the obtained neural network model comprises a three-layer network structure;
wherein the three-tier network structure comprises: an input layer, a hidden layer, and an output layer.
According to another aspect of the present invention, there is provided an apparatus for performing ultrafiltration membrane contamination evaluation through a neural network, the apparatus comprising:
a preprocessing module for preprocessing the original data,
a training module for training the neural network model by adopting the preprocessed data set,
and the evaluation module is used for evaluating the membrane pollution condition under the new working condition based on the obtained neural network model.
According to another aspect of the present invention, there is provided a system for performing ultrafiltration membrane contamination evaluation through a neural network, the system comprising a memory and a processor, the memory having stored thereon a computer program for execution by the processor, the computer program, when executed by the processor, performing the method for performing ultrafiltration membrane contamination evaluation through a neural network of the present invention.
According to another aspect of the present invention, there is provided a storage medium on which program instructions are stored, which when executed by a computer or a processor, are used for executing the steps of the method for realizing pollution evaluation of an ultrafiltration membrane through a neural network according to the present invention, and are used for realizing the modules in the device for realizing pollution evaluation of an ultrafiltration membrane through a neural network according to the embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. It is to be understood that the described embodiments are merely a subset of embodiments of the invention and not all embodiments of the invention, with the understanding that the invention is not limited to the example embodiments described herein.
In the prior art, the following two methods are mainly adopted for evaluating membrane pollution based on transmembrane pressure difference:
the first method is to use the formula:
wherein P represents the degree of membrane fouling; qtAfter the operation time t of the membrane component is shown, the water yield in unit time is shown; q0Indicates the initial water yield, prediff, of the membrane module per unit time0And predifftRespectively representing the membrane differential pressure at the initial operation time of the membrane module and the membrane differential pressure after the operation time t.
For the case where the transmembrane pressure difference is not measured, we assume that the transmembrane pressure difference is constant and use the water production to make the calculation:
another approach is to use the formula:
1/J′sp=1+(FI)Vsp
wherein J represents the membrane flux in L/(m)2·h);
J′spRelative unit membrane flux is expressed, and the formula is calculated: j'sp=Jsp/Jsp0No dimensional quantity;
Jspexpressing the unit membrane flux, the formula is calculated: j. the design is a squarespJ/p, unit L/(m)2·h·Pa);
p represents the transmembrane pressure difference in Pa;
Jsp0expressing the unit membrane flux at the initial stage of filtration, and calculating the formula: j. the design is a squaresp0=J0/p0Unit L/(m)2·h·Pa);
J0Represents the initial membrane flux for filtration in L/(m)2·h);
p0Represents the initial transmembrane pressure difference of filtration in Pa;
Vsprepresents the water yield per unit area of the membrane, unit L/m2;
FI denotes the membrane fouling index in m2/L。
And, according to J'spDifferent FI may be calculated. FI can be used to characterize the membrane fouling rate, i.e., the smaller the FI, the slower the fouling; conversely, the faster the contamination. FI does not distinguish a specific pollution mechanism, the calculation method is concise and concise, and the membrane pollution condition can be better described, and the pollution rate and the pollution degree can be represented. When the flux is constant, the FI can directly determine the final head loss; while at constant transmembrane pressure, FI is directly related to the final water production.
Next, a method for evaluating contamination of an ultrafiltration membrane by a neural network according to an embodiment of the present invention will be described with reference to fig. 1 to 2. Illustratively, the neural network structure of the present invention is a three-layer structure, where fig. 2 shows a schematic structural diagram of the neural network in an embodiment of the present invention, and fig. 2 shows a schematic flow chart of a method for implementing ultrafiltration membrane contamination evaluation by the neural network in an embodiment of the present invention.
Referring to fig. 2, the method for evaluating the contamination of the ultrafiltration membrane through the neural network mainly comprises the following steps:
step S1, the raw data is preprocessed,
step S2, training the neural network model by adopting the preprocessed data set,
and step S3, evaluating the membrane pollution condition under the new working condition based on the obtained neural network model.
Illustratively, the step S1 of preprocessing the raw data includes: according to the actual process, if a device for measuring the water pressure is not installed, the water outlet pressure of the ultrafiltration membrane is equal to the water inlet pressure minus the concentrated water pressure according to the principle, so that the difference value of the ultrafiltration membrane pressure difference equal to the water inlet pressure and the water outlet pressure is calculated.
The method comprises the steps of selecting the best condition of the operation working condition as an initial state, and using the corresponding membrane pressure difference and membrane flux as the initial membrane pressure difference and the initial membrane flux; the condition with the best operating condition is selected as the initial state, so that the method is more referential and scientific, and the evaluation accuracy can be improved.
Wherein, the membrane flux calculation formula is as follows:
J=V/(T×A)
where J is the membrane flux, V is the sample volume L, T is the sample time h, and A is the membrane effective area.
Illustratively, the step S1 of preprocessing the raw data includes: missing value processing and abnormal value processing;
in the actual operation process, situations such as short-time faults of the data acquisition terminal and the like may occur to cause data loss, so that a missing value needs to be processed, and all suitable methods known by a person skilled in the art can be adopted to realize the missing value processing;
illustratively, the missing value processing method includes: deletion method, filling method, etc.;
the deleting method comprises the steps of directly deleting the data samples with missing data or the variables with excessive missing data; the filling method includes substitution methods such as mean value interpolation, mode interpolation, near filling method and the like, and model prediction methods such as regression method, maximum likelihood estimation method, gray scale theory method, random forest method and the like. The above-mentioned padding method is exemplary, and other methods capable of solving the missing value problem in the art can be applied herein.
Preferably, the missing value is processed by adopting a regression filling method, on one hand, because the change and fluctuation of the ultrafiltration inflow water quality can be found to have certain regularity according to the experimental condition, the missing data can be reduced as much as possible according to the regression filling method; on the other hand, the missing value is filled up by a regression filling-up method, so that the effectiveness of subsequent neural network model training can be improved, and the accuracy of the neural network model in membrane pollution evaluation is further improved.
Because data collection is usually performed through the meter equipment, and the meter equipment is inevitably subject to failure and other conditions, individual data abnormality, namely a data abnormal value, occurs in the collected data.
In order to improve the accuracy of the acquired data, abnormal values need to be processed;
illustratively, prior to processing the outlier comprises: judging abnormal values; and firstly, finding out abnormal values of the data from the acquired data, and further processing the abnormal values.
The method for distinguishing the abnormal value mainly comprises the following steps: statistical analysis methods, 3 sigma principles, boxplot analysis methods, and the like;
the statistical analysis method comprises the steps of carrying out descriptive statistics on data of each attribute value so as to judge an abnormal value; in the 3 sigma principle, sigma represents a standard deviation, firstly, a group of detection data is supposed to only contain random errors, the detection data is calculated to obtain a standard deviation, an interval is determined according to a certain probability, the errors exceeding the interval are considered not to belong to the random errors but to be coarse errors (namely abnormal values), and the data containing the coarse errors are removed to obtain 3 sigma; the Box diagram (Box-plot), also called Box-whisker diagram, Box diagram or Box diagram, is a statistical diagram used for displaying a group of data dispersion situation data, the drawing of the Box diagram depends on actual data, does not need to assume in advance that the data obeys a specific distribution form, does not make any restrictive requirement on the data, and only truly and intuitively represents the original appearance of the data shape, therefore, the Box diagram has certain superiority in identifying abnormal values. Of course, other suitable methods for determining data outliers may be used herein.
After the abnormal value is determined by the abnormal value determination method, the abnormal value is processed, and the processing of the abnormal value mainly comprises the following steps: directly deleting the abnormal value; or the abnormal value is regarded as a missing value, namely the abnormal value can be processed by adopting the method for processing the missing value; or the average value is used for correction. Other methods of solving the outlier problem in the art may be used herein.
Illustratively, the step S1 of preprocessing the raw data further includes: data standardization, variable screening, and the like;
because the dimensions of each parameter have differences, data standardization is needed to convert the parameters into dimensionless ones, and further calculation is facilitated.
In the present application, parameters of the model may be determined according to data, correlation coefficients, information entropy, and the like within a sampling time, where the input parameters include: apart from last cleaning time, COD, ammonia nitrogen, sludge concentration and temperature, output parameter includes: membrane fouling index.
Illustratively, before the preprocessing the raw data, the method further comprises: acquiring ultrafiltration water quality data and other operation data and the like;
in many cases, the power is usually turned on to full frequency during operation to meet the requirements of design parameters, which results in less working conditions of historical data, and poor results if the data is directly used for modeling. Therefore, experiments are needed for a period of time to acquire data under different working conditions for training, so that the accuracy of the model is improved.
Illustratively, the acquiring water quality data of ultrafiltration and other operational data includes: acquiring data under different working conditions; acquiring data under different working conditions by sampling, wherein the sampling corresponding variables are selected from last cleaning time, COD, ammonia nitrogen, sludge concentration, inflow rate, temperature, inflow pressure, produced water flow, concentrated water pressure and the like;
illustratively, the present invention samples ultrafiltration data in the leachate;
illustratively, the step S2 of training the neural network model with the preprocessed data set further includes:
step S21, dividing the preprocessed data set into a training set and a testing set;
step S22, training the neural network model by adopting a training set, and evaluating the established model by adopting a test set;
and S23, repeating S21 and S22 to optimize the parameters of the model, so that the model result is expected, and obtaining the model reaching the model result expectation.
Illustratively, the parameter process of the optimization model of step S23 includes using a gradient descent method.
Wherein, the formula adopted by the gradient descent method is as follows:
wherein theta isiThe pending coefficients of the solution are represented, α the step size, and J (theta) the loss function that minimizes the variance with respect to theta.
Illustratively, the obtained neural network model comprises a three-layer network structure; referring to fig. 2, wherein the three-layer network structure comprises: an input layer, a hidden layer, and an output layer.
Wherein the nodes of each layer, except the input layer, contain a non-linear transformation.
Illustratively, the neural network model in the invention includes 6 nodes of the input layer, 1 node of the output layer, one layer of the hidden layer and 7 nodes of the hidden layer.
Specifically, the number of layers and the number of nodes of the neural network model can be designed and selected according to actual needs.
The invention adopts a general Sigmoid function as the number of the activation functions.
For example, the neural network model may need to be updated according to the specific process and the specific sampling time so as to meet the change of the working condition.
According to another aspect of the present invention, there is provided an apparatus for performing ultrafiltration membrane contamination evaluation through a neural network, the apparatus comprising:
a preprocessing module for preprocessing the original data,
a training module for training the neural network model by adopting the preprocessed data set,
and the evaluation module is used for evaluating the membrane pollution condition under the new working condition based on the obtained neural network model.
According to another aspect of the present invention, there is provided a system for performing ultrafiltration membrane contamination evaluation through a neural network, the system comprising a memory and a processor, the memory having stored thereon a computer program for execution by the processor, the computer program, when executed by the processor, performing the method for performing ultrafiltration membrane contamination evaluation through a neural network of the present invention.
According to another aspect of the present invention, there is provided a storage medium on which program instructions are stored, which when executed by a computer or a processor, are used for executing the steps of the method for realizing pollution evaluation of an ultrafiltration membrane through a neural network according to the present invention, and are used for realizing the modules in the device for realizing pollution evaluation of an ultrafiltration membrane through a neural network according to the embodiment of the present invention.
Many water treatment today are determined empirically for when the membrane is cleaned and the human flow factors inevitably lead to poor experience for the operators. The evaluation is also carried out according to a theoretical formula, but the requirements on parameters are severe, and the feasibility in actual operation is not good. The invention adopts the neural network method to give scientific membrane pollution evaluation, on one hand, the invention has self-learning function, avoids the condition of larger operation effect difference caused by the difference of artificial experience, thereby improving the evaluation accuracy, on the other hand, the invention can avoid the harsh requirement of a theoretical formula on parameters, and is easy to implement.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the units is only one logical functional division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another device, or some features may be omitted, or not executed.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the invention and aiding in the understanding of one or more of the various inventive aspects. However, the method of the present invention should not be construed to reflect the intent: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
It will be understood by those skilled in the art that all of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where such features are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.
The various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It will be appreciated by those skilled in the art that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functionality of some of the modules in an item analysis apparatus according to embodiments of the present invention. The present invention may also be embodied as apparatus programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
The above description is only for the specific embodiment of the present invention or the description thereof, and the protection scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and the changes or substitutions should be covered within the protection scope of the present invention. The protection scope of the present invention shall be subject to the protection scope of the claims.