CN115326749A - Method and apparatus for measuring contaminants - Google Patents
Method and apparatus for measuring contaminants Download PDFInfo
- Publication number
- CN115326749A CN115326749A CN202211007383.6A CN202211007383A CN115326749A CN 115326749 A CN115326749 A CN 115326749A CN 202211007383 A CN202211007383 A CN 202211007383A CN 115326749 A CN115326749 A CN 115326749A
- Authority
- CN
- China
- Prior art keywords
- absorbance
- pollutant
- vector
- pollutants
- content
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000000356 contaminant Substances 0.000 title claims abstract description 62
- 238000000034 method Methods 0.000 title claims abstract description 41
- 239000003344 environmental pollutant Substances 0.000 claims abstract description 111
- 231100000719 pollutant Toxicity 0.000 claims abstract description 111
- 238000002329 infrared spectrum Methods 0.000 claims abstract description 37
- 238000001228 spectrum Methods 0.000 claims abstract description 37
- 238000003860 storage Methods 0.000 claims abstract description 20
- 239000000126 substance Substances 0.000 claims abstract description 19
- 239000000463 material Substances 0.000 claims abstract description 15
- 239000013598 vector Substances 0.000 claims description 75
- 238000002835 absorbance Methods 0.000 claims description 73
- 239000000446 fuel Substances 0.000 claims description 39
- 238000013145 classification model Methods 0.000 claims description 24
- 230000006870 function Effects 0.000 claims description 21
- RTZKZFJDLAIYFH-UHFFFAOYSA-N Diethyl ether Chemical compound CCOCC RTZKZFJDLAIYFH-UHFFFAOYSA-N 0.000 claims description 16
- 238000003556 assay Methods 0.000 claims description 12
- 239000003921 oil Substances 0.000 claims description 12
- 238000004088 simulation Methods 0.000 claims description 11
- 238000011176 pooling Methods 0.000 claims description 9
- 239000003208 petroleum Substances 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 7
- 239000003502 gasoline Substances 0.000 claims description 6
- 239000003350 kerosene Substances 0.000 claims description 5
- 238000012549 training Methods 0.000 claims description 5
- 239000002283 diesel fuel Substances 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 claims description 4
- 238000000611 regression analysis Methods 0.000 claims description 4
- 238000010224 classification analysis Methods 0.000 claims description 3
- 239000010779 crude oil Substances 0.000 claims description 3
- 239000010687 lubricating oil Substances 0.000 claims description 3
- 239000012264 purified product Substances 0.000 claims description 2
- 239000001993 wax Substances 0.000 claims description 2
- 239000013014 purified material Substances 0.000 claims 1
- 238000004451 qualitative analysis Methods 0.000 abstract description 6
- 238000004445 quantitative analysis Methods 0.000 abstract description 6
- 230000003595 spectral effect Effects 0.000 abstract description 6
- 238000005259 measurement Methods 0.000 abstract description 3
- 238000004891 communication Methods 0.000 description 11
- 238000012545 processing Methods 0.000 description 11
- 238000001514 detection method Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 8
- 238000013527 convolutional neural network Methods 0.000 description 5
- 230000007547 defect Effects 0.000 description 5
- 239000000203 mixture Substances 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000012417 linear regression Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000010606 normalization Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000004497 NIR spectroscopy Methods 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- -1 diesel Substances 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 239000002245 particle Substances 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 101150051392 UL91 gene Proteins 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000011109 contamination Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000011423 initialization method Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000004611 spectroscopical analysis Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
- G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
- G01N21/31—Investigating relative effect of material at wavelengths characteristic of specific elements or molecules, e.g. atomic absorption spectrometry
- G01N21/35—Investigating relative effect of material at wavelengths characteristic of specific elements or molecules, e.g. atomic absorption spectrometry using infrared light
- G01N21/359—Investigating relative effect of material at wavelengths characteristic of specific elements or molecules, e.g. atomic absorption spectrometry using infrared light using near infrared light
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
Abstract
The invention provides a method and a device for analyzing pollutants and a storage medium. The method for analyzing the pollutants comprises the following steps: acquiring the near infrared spectrum of pure substances and pollutants; acquiring a simulated spectrum set of the polluted substance according to the near infrared spectrums of the pure substance and the pollutant; obtaining a near infrared spectrum of an unknown contaminated material; and predicting the type and content of the contaminant in the unknown contaminated material based on the near infrared spectrum of the unknown contaminated material and the set of simulated spectra. By performing these steps, the method for analyzing the pollutants can perform qualitative and quantitative analysis only by performing one spectral measurement on the polluted substances, so as to determine the types and the contents of the pollutants in the polluted substances.
Description
Technical Field
The invention relates to the technical field of near-infrared modeling data processing, in particular to a pollutant determination method, a pollutant determination device and a corresponding computer readable storage medium.
Background
Aviation fuel is an indispensable energy source for the aviation industry. During the use of aviation fuel, there are many safety issues. Once these problems occur, severe aviation accidents often occur. For example, the rate of aviation accidents related to aviation fuel in China is about 10% to 15%, and about 500 primary aviation accidents occur each year in the world, wherein the accidents related to fuel account for about 10%, so the quality of aviation fuel is an important factor related to production safety.
During the whole period of production, storage, transportation and use of aviation fuel, the aviation fuel is polluted by other oil materials. For example, jet fuel No. 3 may be contaminated with motor gasoline, diesel fuel when used in airports. However, the conventional aviation fuel pollutant detection standard, such as MH/T6068-2017, has the defects of complicated sampling method, multiple operation steps, long time consumption, and incapability of analyzing the type and content of oil pollutants, and only one pollutant can be analyzed at a time.
In order to overcome the above-mentioned drawbacks of the prior art, there is a need in the art for a contaminant determination technique for quickly and conveniently performing qualitative and quantitative analysis of contaminants to determine the type and content of contaminants in contaminated materials.
Disclosure of Invention
The following presents a simplified summary of one or more aspects in order to provide a basic understanding of such aspects. This summary is not an extensive overview of all contemplated aspects, and is intended to neither identify key or critical elements of all aspects nor delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that is presented later.
In order to overcome the above-mentioned defects in the prior art, the present invention provides a method for analyzing a contaminant, an apparatus for analyzing a contaminant, and a computer-readable storage medium therefor, which can complete qualitative and quantitative analysis by performing only one spectroscopic measurement on a contaminated material, and can conveniently and rapidly determine the type and content of the contaminant in the contaminated material.
Specifically, the above-described contaminant determination method provided according to the first aspect of the present invention includes the steps of: acquiring a near infrared spectrum of a pollutant to be detected; determining an absorbance characteristic vector of the pollutant to be detected according to the near infrared spectrum; classifying and analyzing the absorbance characteristic vector according to a pre-established simulation spectrum set to determine one or more pollutants contained in the pollutants to be detected, wherein the simulation spectrum set is established according to the near infrared spectrum of the purified substances corresponding to the pollutants to be detected; and performing regression analysis on the absorbance characteristic vector according to the simulated spectrum set and the classification result to determine the content of each pollutant in the pollutants to be detected.
Further, in some embodiments of the present invention, the step of establishing the set of simulated spectra comprises: acquiring a first near infrared spectrum of the pure object; obtaining a second near infrared spectrum of the at least one contaminant sample; setting the type and content of each pollutant in the pollutant sample; and fitting the second absorbance vector of the second near infrared spectrum of each pollutant sample and the first absorbance vector of the first near infrared spectrum of the pure object respectively to establish the simulated spectrum set.
Further, in some embodiments of the present invention, the step of fitting the second absorbance vector of the second near infrared spectrum of each of the contaminant samples and the first absorbance vector of the first near infrared spectrum of the purified object respectively comprises: performing linear fitting calculation on the first absorbance vector and the third absorbance vector of each pollutant according to the set pollutant types and pollutant contents to determine a second absorbance vector of a second near infrared spectrum of each pollutant sample, wherein the calculation expression is as follows:
Absor mix ＝x％×Absor air +y％Absor pol
wherein, absor mix Is a second absorbance vector, absor, of the contaminant sample air Is the first absorbance vector, absor, of the purified product pol And the matrix is formed by third absorbance vectors of all the pollutants, wherein x% is the percentage content of the pure substances, and y% is the percentage content vector of all the pollutants.
Further, in some embodiments of the present invention, the step of performing a classification analysis on the absorbance feature vector according to a pre-established simulation spectrum set to determine one or more pollutants contained in the pollutant to be detected includes: and inputting the absorbance characteristic vector into a pre-established classification model trained according to the simulation spectrum set so as to determine one or more corresponding pollutants according to the first learning parameter of the classification model.
Further, in some embodiments of the present invention, the step of building and training the classification model comprises: establishing an initialized classification model by using an input layer, at least one convolution layer, a pooling layer and an output layer of one-dimensional convolution; sequentially importing each second absorbance vector in the simulated spectrum set into the input layer, and obtaining one or more corresponding class prediction labels through the output layer after the second absorbance vectors are processed by the initialized first learning parameters of the classification model; and adjusting the first learning parameter according to the error between the type prediction label and the sample expected value, and repeating the operations of inputting a second absorbance vector, calculating the type prediction label and adjusting the first learning parameter until the error between the type prediction label and the sample expected value meets an error threshold.
Further, in some embodiments of the present invention, the step of performing a regression analysis on the absorbance feature vector according to the simulated spectrum set and the classification result to determine the content of each pollutant in the pollutant to be detected includes: and inputting the absorbance feature vector and the classified result labels of the one or more pollutants into a pre-established regression model trained according to the simulation spectrum set so as to determine the content of each pollutant according to a second learning parameter of the regression model.
Further, in some embodiments of the present invention, the step of establishing and training the regression model comprises: establishing a regression model to establish the initialized correlation between the absorbance characteristic vector of the pollutant to be detected and the content of one or more pollutants corresponding to the absorbance characteristic vector; sequentially introducing each second absorbance vector in the simulated spectrum set and the corresponding one or more pollutant type labels into the regression model, and obtaining one or more corresponding content predicted values after the second absorbance vector is processed by initialized second learning parameters of the regression model; and adjusting the second learning parameter according to the error between the content predicted value and the sample expected value, and repeating the operations of inputting a second absorbance vector and a type label, calculating the content predicted value and adjusting the second learning parameter until the error between the one or more content predicted values and the sample expected value meets an error threshold value.
Further, in some embodiments of the present invention, an initialization correlation between the absorbance feature vector of the to-be-detected pollutant and the content of one or more pollutants corresponding to the absorbance feature vector is constructed; sequentially introducing each second absorbance vector in the simulated spectrum set and the corresponding type label of the one or more pollutants into the regression model, and obtaining one or more corresponding content predicted values after processing by initialized second learning parameters of the regression model; and adjusting the second learning parameter according to the error between the content predicted value and the sample expected value, and repeating the operations of inputting a second absorbance vector and a type label, calculating the content predicted value and adjusting the second learning parameter until the error between the one or more content predicted values and the sample expected value meets an error threshold value.
Further, in some embodiments of the invention, the step of integrating the second learning parameter comprises: setting different types of error parameters; and adjusting the second learning parameter according to the convergence speed of the error function.
Further, in some embodiments of the present invention, the purified substance is a purified aviation fuel, and the contaminant to be detected is an aviation fuel contaminant selected from at least one of crude oil, condensate, naphtha, gasoline, kerosene, diesel oil, wax oil, residual oil, petroleum ether, and lubricating oil.
In addition, the above-mentioned pollutant measuring device according to the second aspect of the present invention includes a memory and a processor. The processor is connected to the memory and is configured to implement the above-described contaminant determination method provided by the first aspect of the invention.
Further, the above computer-readable storage medium according to a third aspect of the present invention is provided, having computer instructions stored thereon. The computer instructions, when executed by a processor, implement the contaminant determination method provided by the first aspect of the invention.
Drawings
The above features and advantages of the present disclosure will be better understood upon reading the detailed description of embodiments thereof in conjunction with the following drawings. In the drawings, components are not necessarily drawn to scale, and components having similar relative characteristics or features may have the same or similar reference numerals.
FIG. 1 shows an architecture diagram of an assay device for contaminants provided in accordance with some embodiments of the invention;
FIG. 2 illustrates a flow diagram of a method of determining contaminants provided according to some embodiments of the invention;
FIG. 3 illustrates a schematic diagram of a convolutional neural network of a classification model provided in accordance with some embodiments of the present invention;
FIG. 4 illustrates a schematic near infrared spectrum of jet fuel provided in accordance with some embodiments of the invention;
FIG. 5 illustrates a near infrared spectrum schematic of naphtha provided in accordance with some embodiments of the invention;
FIG. 6 illustrates a near infrared spectrum schematic of a condensate provided in accordance with some embodiments of the present invention;
FIG. 7 illustrates a near infrared spectroscopy schematic of petroleum ether provided in accordance with some embodiments of the invention;
FIG. 8 is a graph illustrating the degree of deviation of predicted values from true values for a naphtha regression model provided in accordance with some embodiments of the invention;
FIG. 9 illustrates a graph of the degree of deviation of predicted values from true values for a regression model with condensate provided in accordance with some embodiments of the present invention; and
FIG. 10 illustrates a schematic representation of the degree of deviation of predicted values from true values for a petroleum ether regression model provided according to some embodiments of the invention.
Detailed Description
The following description of the embodiments of the present invention is provided for illustrative purposes, and other advantages and effects of the present invention will become apparent to those skilled in the art from the present disclosure. While the invention will be described in connection with the preferred embodiments, there is no intent to limit its features to those embodiments. On the contrary, the invention is described in connection with the embodiments for the purpose of covering alternatives or modifications that may be extended based on the claims of the present invention. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. The invention may be practiced without these particulars. Moreover, some of the specific details have been left out of the description in order to avoid obscuring or obscuring the focus of the present invention.
In the description of the present invention, it should be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in a specific case to those of ordinary skill in the art.
Also, the terms "upper," "lower," "left," "right," "top," "bottom," "horizontal," "vertical" and the like used in the following description shall be understood to refer to the orientation as it is drawn in this section and the associated drawings. The relative terms are used for convenience of description and do not imply that the described apparatus should be constructed or operated in the specific orientation and therefore should not be construed as limiting the invention.
It will be understood that, although the terms first, second, third, etc. may be used herein to describe various elements, regions, layers and/or sections, these elements, regions, layers and/or sections should not be limited by these terms, but rather are used to distinguish one element, region, layer and/or section from another element, region, layer and/or section. Thus, a first component, region, layer or section discussed below could be termed a second component, region, layer or section without departing from some embodiments of the present invention.
As mentioned above, during the entire cycle of production, storage, transportation, use of aviation fuel, there is a risk that aviation fuel will be contaminated with other oils. For example, jet fuel No. 3 may be contaminated with motor gasoline, diesel fuel when used in airports. However, the traditional aviation fuel pollutant detection standard, such as MH/T6068-2017, has the defects of complicated sampling method, multiple operation steps, long time consumption, capability of analyzing only one pollutant at a time, and incapability of analyzing the type and content of oil pollutants.
Due to the maturity of spectroscopy technology, the improvement of reliability and accuracy of computer science technology, and the development and application of chemometrics, the near infrared technology can complete a large amount of spectral data analysis and processing in a short time, so that the near infrared technology is rapidly developed.
In order to overcome the above-mentioned defects in the prior art, the present invention provides a method for determining a contaminant, a planning apparatus for determining a contaminant, and a computer-readable storage medium therefor, which can perform qualitative and quantitative analysis by performing a single spectroscopic determination on a contaminated material, thereby determining the type and content of the contaminant in the contaminated material.
In some non-limiting embodiments, the above-mentioned method for determining a contaminant provided by the first aspect of the present invention may be implemented by a contaminant determination apparatus provided by the second aspect of the present invention. Specifically, the planning device is configured with a memory and a processor. The memory includes, but is not limited to, the above-described computer-readable storage medium provided by the third aspect of the invention having computer instructions stored thereon. The processor is coupled to the memory and configured to execute computer instructions stored on the memory to perform the method of contaminant determination provided by the first aspect of the invention.
Please refer to fig. 1. FIG. 1 illustrates an architectural diagram of a contaminant assay device provided in accordance with some embodiments of the invention. The contaminant determination device includes an internal communication bus 301, a processor 302, a Read Only Memory (ROM) 303, a Random Access Memory (RAM) 304, a communication port 305, and a hard disk 307. The internal communication bus 301 may enable data communication between the assay device components of the contaminant. The processor 302 may make the determination and issue the prompt. In some embodiments, processor 302 may be comprised of one or more processors. The communication port 305 may enable data transfer and communication between the assay device of the contaminant and an external input/output device. In some embodiments, the contaminant assay device may send and receive information and data from the network through the communication port 305. In some embodiments, the contaminant detection device may communicate and transmit data via the input/output port 306 in a wired manner to an external input/output device. The contaminant detection device also includes various forms of program storage units and data storage units, such as a hard disk 307, read Only Memory (ROM) 303 and Random Access Memory (RAM) 304, capable of storing various data files for computer processing and/or communication, as well as possible program instructions for execution by the processor 302. The processor 302 executes these instructions to implement the main parts of the method. The results of the processing by the processor 302 are communicated to an external output device via the communication port 305 for display on a user interface of the output device.
The working principle of the above-described contaminant detection device will be described below with reference to some examples of contaminant detection methods. It will be appreciated by those skilled in the art that these examples of communication methods are merely provided as non-limiting examples of the present invention, and are intended to clearly illustrate the broad concepts of the present invention and to provide some specific details which may be readily implemented by the public rather than to limit the overall function or operation of the contamination assay device. Similarly, the contaminant measurement device is only a non-limiting embodiment of the present invention, and does not limit the main body of the contaminant measurement method.
As shown in step S1 of fig. 2, in the process of determining the pollutant, the pollutant determining apparatus may first obtain a near infrared spectrum of the pollutant to be determined, and determine an absorbance feature vector of the pollutant to be determined.
Alternatively, in some embodiments, the present invention may employ a brook Matrix-F near infrared spectrometer to collect spectral data for jet fuel No. 3 and naphtha, condensate, petroleum ether, with a near infrared spectrum as shown in fig. 3-6.
Further, as shown in step S2 of fig. 1, after the near infrared spectrum of the pollutant to be detected and the absorbance feature vector of the pollutant to be detected are obtained, the present invention can fit the absorbance vector of the near infrared spectrum of the pollutant and the absorbance vector of the near infrared spectrum of the purified substance by setting the type and the ratio of the pollutant in the polluted substance, so as to form the simulated spectrum set.
Specifically, the method can perform linear fitting calculation on the absorbance vector of the near infrared spectrum of the pollutant and the absorbance vector of the near infrared spectrum of the corresponding pure object according to the beer-Lambert law and the set pollutant types and proportions to form the polluted aviation fuel simulation spectrum set. The computational expression of the absorbance vector for this set of spectra is:
Absor mix ＝x％×Absor air +y％Absor pol
wherein, absor mix 、Absor air And Absor pol The absorbance vectors of the respective pollutants of the polluted aviation fuel, the pure aviation kerosene and the common oil material of the aviation fuel, and x% and y% are the respective pure aviation kerosene and the common oil material of the aviation fuelSimulated mass percent of material contaminants.
Optionally, in some embodiments, the contaminants to be tested include, but are not limited to, jet fuel and aviation piston engine fuel. The jet fuel includes, but is not limited to, at least one of jet fuel No. 1, jet fuel No. 2, jet fuel No. 3, jet fuel No. 4, jet fuel No. 5, jet fuel No. 6. The aviation gasoline includes, but is not limited to, at least one of aviation piston engine fuel No. 75, aviation piston engine fuel No. UL91, aviation piston engine fuel No. 95, aviation piston engine fuel No. 100L.
Optionally, in some embodiments, the contaminants include, but are not limited to, at least one of crude oil, condensate, naphtha, gasoline, kerosene, diesel, wax oil, resid, petroleum ether, lubricating oil.
Further, for the simulated spectrum set, the present invention may perform classification analysis on the absorbance feature vector according to a pre-established simulated spectrum set, so as to determine one or more pollutants contained in the to-be-detected pollutant.
Specifically, the method can analyze the convolutional neural network as a classification model, input the acquired absorbance characteristic vector into the convolutional neural network, and output the type of the pollutant after the treatment of the convolutional neural network.
In some embodiments of the invention, the substance to be tested is an aviation fuel. Because aviation fuel and common oil pollutants thereof have similar composition components, the near infrared spectrum concentrated by the simulated spectrum has the characteristics of high similarity, high data dimension and large redundancy. The invention can find the characteristics of pollutants in the spectrum of the polluted aviation fuel by utilizing the strong characteristic extraction capability of the convolutional neural network.
Furthermore, the invention can use a one-dimensional convolution input layer, at least one convolution layer, a pooling layer and an output layer to establish an initialized classification model, and then, each second absorbance vector concentrated by the simulated spectrum is sequentially led into the input layer of the classification model, and after the processing of the initialized first learning parameter of the classification model, one or more corresponding class prediction labels are obtained through the output layer. Then, the invention can also adjust the first learning parameter of the classification model according to the error between the obtained type prediction label and the sample expected value, and repeat the operations of inputting the second absorbance vector, calculating the type prediction label and adjusting the first learning parameter until the error between the obtained type prediction label and the sample expected value meets the error threshold.
Preferably, as shown in fig. 2, in the process of the pollutant classification result, the present invention may further extract the characteristics of the pollutant through two convolution layers, respectively act on the characteristics of the pollutant in the near infrared spectrum in the polluted material through a pooling layer and reduce the size of the pollutant, and continuously reduce the spatial size of the data to reduce the number and the amount of calculation of the parameters, thereby controlling the overfitting of the pollutant classification prediction result to a certain extent. In this embodiment, the non-linear pooling function in the pooling layer may adopt a maximum pooling function, divide the input into a plurality of rectangular regions, and output the maximum value for each sub-region to complete the pooling operation. The invention can then pass the data through both convolutional and pooling layers to further compress the size of the features of the contaminant in the near infrared spectrum in the contaminated material and further avoid over-fitting of the contaminant classification prediction results. Then, the invention can output the pollutant classification result through a flattening layer (flat layer) and a dense layer (dense layer).
Optionally, the present invention may further add a full-connectivity layer to the neural network connection structure to perform nonlinear combination on the extracted features to obtain an output, and complete a learning objective by using existing high-order features of the contaminants.
Further, in some embodiments of the present invention, by adjusting the neural network hyper-parameters such as the learning rate parameter, the regularization parameter, the selection of the loss function, the weight initialization method, and the like, the present invention can gradually reduce the errors of the class prediction labels and the sample expected values until the error threshold is satisfied, and make the convergence speed of the error function satisfy the convergence speed threshold.
Further, in some embodiments of the present invention, by using the accuracy, precision, and recall of the classification model as the evaluation indexes of the evaluation classification model, the present invention may further use regularization and normalization processing as the error function.
Optionally, in some embodiments of the invention, the contaminants may comprise at least one of naphtha, condensate, petroleum ether. Table 1 below shows the confusion matrix of three common contaminants and shows the accuracy and precision of the classification model.
TABLE 1 confusion matrix for classification models
Furthermore, the method can also establish a regression model for predicting the content of the pollutants in the polluted substances based on the simulation spectrum set obtained in the step S1.
In particular, the regression model may be an artificial neural network. The invention can carry out regularization normalization processing on the input absorbance vector and the pollutant types obtained by the classification model, and input the normalized normalization processing into a neural network model as a high-dimensional vector. In the high-dimensional vector input neural network model, input data are processed through three hidden layers, and the content of the polluted substances can be output through an output layer.
Further, the regression model may be a multiple linear regression model. The invention can take the input absorbance vector and the pollutant types obtained by the classification model as a plurality of independent variables, and simultaneously take the corresponding content of the pollutant as a dependent variable to be substituted into the multiple linear regression model to determine the content of the polluted substance. Here, the function of the multiple linear regression model is:
Z y ＝β 1 Z·1+β 2 Z·2+...+β k Z·k.
optionally, in some embodiments of the invention, the regression model is a partial least squares model. The invention can take the input absorbance vector and the pollutant types obtained by the classification model as a plurality of independent variables, simultaneously take the corresponding content of the pollutant as a dependent variable to be substituted into the partial least square model, and respectively project the prediction variable and the observation variable to a new space through projection to find a linear regression model to predict the content of the pollutant.
Furthermore, the invention can also construct the initialization correlation between the absorbance characteristic vector of the pollutant to be detected and the content of one or more pollutants corresponding to the absorbance characteristic vector, and randomly set the initialization parameters of the regression model. Specifically, the present invention can randomly set initialization parameters of the regression model by truncating the normal distribution.
Then, in the training process of the regression model, the invention can sequentially introduce each second absorbance vector in the simulated spectrum set and the corresponding type label of one or more pollutants into the regression model, and obtain one or more corresponding content predicted values after the second learning parameter processing of the initialization of the regression model. Here, the second learning parameter includes, but is not limited to, at least one of a kernel function, a loss function, and a maximum weak learner number.
Compared with the traditional single model prediction of the pollutant types and content, the method has the advantages that the traditional single model prediction is decoupled into double model prediction. The traditional single model prediction has the defects that the type of pollutants cannot be determined, and the required data processing amount and the calculation amount are large, so that the prediction time is too long, and the accuracy is low. The method comprises the steps of firstly rapidly classifying pollutants through a classification model, then inputting the types of the pollutants and the absorbance vectors of the pollutants to be detected into a regression model, and selecting a proper regression model under the condition of determining a regression term because the classification of the pollutants is determined, so that the regression model can also rapidly, timely and accurately predict.
Furthermore, the invention can also adjust the second learning parameter by setting different types of error parameters and according to the convergence rate of the error function.
Preferably, in some embodiments of the present invention, the present invention may use the root mean square error RMSEP as an error function and use the error function and the decision coefficient R-Squared to judge the performance of the regression model. Here, RMSEP is a predicted root mean square error, and a smaller value indicates a better fitting effect. Suppose y i Is the true value, f (x) i ) For predictive value, RMSEP can be expressed as follows:
the value of the coefficient R-Squared is determined to be between 0 and 1. The closer the R-Squared is to 1, the better the model is. The closer the R-Squared is to 0, the worse the model is. Of course, negative values of R-Squared also exist, indicating that the model is very ineffective. Suppose y i Is the true value, f (x) i ) For predictive value, the decision coefficient R-Squared can be expressed as follows:
specifically, in the embodiments shown in fig. 8 to 10, the Root Mean Square Error (RMSEP) of the three regression models is less than 0.01, and the coefficient of determination R-Squared is greater than 0.999, so that the three regression models have small prediction errors and good fitting effect.
Here, the contaminants include naphtha, condensate, and petroleum ether, and the comparison results of the reference values and predicted values of the kinds and contents of the contaminants based on the near infrared spectroscopy are shown in table 2. As can be seen from Table 2, the model has good classification accuracy and small prediction error of pollutant content.
TABLE 2 qualitative and quantitative analysis results of the type and content of aviation fuel contaminants
Compared with the existing pollutant determination technology in the field, the method can complete qualitative and quantitative analysis by performing spectral determination on the polluted substances once, so that the method has the advantages of convenience in operation, high analysis speed and the like, and can overcome the problems of high spectral similarity, high data dimensionality and large redundancy of the polluted substances and the pollutants, and further can efficiently extract spectral features to support the establishment of qualitative and quantitative models. In addition, the spectrum set established by the invention is easy to expand. If new species pollutants are encountered in practical application, the qualitative and quantitative models can be expanded by adding the new species pollutants into the spectrum set, so that the universality of the method and the system is continuously improved.
While, for purposes of simplicity of explanation, the methodologies are shown and described as a series of acts, it is to be understood and appreciated that the methodologies are not limited by the order of acts, as some acts may, in accordance with one or more embodiments, occur in different orders and/or concurrently with other acts from that shown and described herein or not shown and described herein, as would be understood by one skilled in the art.
Those of skill in the art would understand that information, signals, and data may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits (bits), symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
Although the contaminant detection means described in the above embodiments may be implemented by a combination of software and hardware. It is understood that the contaminant detection means may also be implemented in software, hardware. For a hardware implementation, the contaminant determination device may be implemented in one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, other electronic devices designed to perform the functions described herein, or a selected combination thereof. For software implementations, the contaminant determination means may be implemented by separate software modules, such as program modules (processes) and function modules (functions), running on a common chip, each of which performs one or more of the functions and operations described herein.
The various illustrative logical modules, and circuits described in connection with the embodiments disclosed herein may be implemented or performed with a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In the alternative, the processor and the storage medium may reside as discrete components in a user terminal.
In one or more exemplary embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a web site, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk (disk) and disc (disc), as used herein, includes Compact Disc (CD), laser disc, optical disc, digital Versatile Disc (DVD), floppy disk and blu-ray disc where disks (disks) usually reproduce data magnetically, while discs (discs) reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples and designs described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (11)
1. A method for determining contaminants, comprising the steps of:
acquiring a near infrared spectrum of a pollutant to be detected;
determining an absorbance characteristic vector of the pollutant to be detected according to the near infrared spectrum;
classifying and analyzing the absorbance characteristic vector according to a pre-established simulation spectrum set to determine one or more pollutants contained in the pollutants to be detected, wherein the simulation spectrum set is established according to the near infrared spectrum of the purified substances corresponding to the pollutants to be detected; and
and performing regression analysis on the absorbance characteristic vector according to the simulated spectrum set and the classification result to determine the content of each pollutant in the pollutant to be detected.
2. The assay method of claim 1, wherein the step of establishing the set of simulated spectra comprises:
acquiring a first near infrared spectrum of the pure object;
obtaining a second near infrared spectrum of the at least one contaminant sample;
setting the type and content of each pollutant in the pollutant sample; and
and respectively fitting the second absorbance vector of the second near infrared spectrum of each pollutant sample and the first absorbance vector of the first near infrared spectrum of the pure object to establish the simulated spectrum set.
3. The method of claim 2, wherein the step of separately fitting the second absorbance vector of the second near-infrared spectrum of each of the contaminant samples to the first absorbance vector of the first near-infrared spectrum of the purified material comprises:
performing linear fitting calculation on the first absorbance vector and the third absorbance vector of each pollutant according to the set pollutant species and pollutant content to determine a second absorbance vector of a second near infrared spectrum of each pollutant sample, wherein the calculation expression is as follows:
Absor mix ＝x％×Absoa air +y％×Absor pol
wherein, absor mix Is a second absorbance vector, absor, of the contaminant sample air Is the first absorbance vector, absor, of the purified product pol And the matrix is formed by third absorbance vectors of all the pollutants, wherein x% is the percentage content of the pure substances, and y% is the percentage content vector of all the pollutants.
4. The assay method according to claim 2, wherein the step of performing a classification analysis on the absorbance eigenvectors according to a pre-established set of simulated spectra to determine one or more contaminant substances contained in the contaminants to be detected comprises:
and inputting the absorbance characteristic vector into a pre-established classification model trained according to the simulation spectrum set so as to determine one or more corresponding pollutants according to the first learning parameter of the classification model.
5. The assay of claim 4, wherein the step of building and training the classification model comprises:
establishing an initialized classification model by using an input layer, at least one convolution layer, a pooling layer and an output layer of one-dimensional convolution;
sequentially importing each second absorbance vector in the simulated spectrum set into the input layer, and obtaining one or more corresponding class prediction labels through the output layer after the second absorbance vectors are processed by the initialized first learning parameters of the classification model; and
and adjusting the first learning parameter according to the error between the type prediction label and the sample expected value, and repeating the operations of inputting a second absorbance vector, calculating the type prediction label and adjusting the first learning parameter until the error between the type prediction label and the sample expected value meets an error threshold.
6. The method according to claim 2, wherein the step of performing a regression analysis on the absorbance eigenvectors according to the set of simulated spectra and the classification result to determine the content of each of the pollutants in the pollutant to be detected comprises:
and inputting the absorbance feature vector and the classified result labels of the one or more pollutants into a pre-established regression model trained according to the simulation spectrum set so as to determine the content of each pollutant according to a second learning parameter of the regression model.
7. The assay of claim 6, wherein the step of establishing and training the regression model comprises:
establishing an initialization correlation between the absorbance characteristic vector of the pollutant to be detected and the content of one or more pollutants corresponding to the absorbance characteristic vector;
sequentially introducing each second absorbance vector in the simulated spectrum set and the corresponding one or more pollutant type labels into the regression model, and obtaining one or more corresponding content predicted values after the second absorbance vector is processed by initialized second learning parameters of the regression model;
and adjusting the second learning parameter according to the error between the content predicted value and the sample expected value, and repeating the operations of inputting a second absorbance vector and a type label, calculating the content predicted value and adjusting the second learning parameter until the error between the one or more content predicted values and the sample expected value meets an error threshold value.
8. The assay method according to claim 7, wherein the step of adjusting the second learning parameter comprises:
setting different types of error parameters; and
and adjusting the second learning parameter according to the convergence speed of the error function.
9. The method of claim 1, wherein the clean material is clean aviation fuel, and the contaminant to be detected is an aviation fuel contaminant selected from at least one of crude oil, condensate, naphtha, gasoline, kerosene, diesel oil, wax oil, residual oil, petroleum ether, and lubricating oil.
10. An assay system for contaminants, comprising:
a memory; and
a processor coupled to the memory and configured to implement the method of determining a contaminant of any one of claims 1-9.
11. A computer readable storage medium having computer instructions stored thereon, wherein the computer instructions, when executed by a processor, perform a method of determining a contaminant according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211007383.6A CN115326749A (en) | 2022-08-22 | 2022-08-22 | Method and apparatus for measuring contaminants |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211007383.6A CN115326749A (en) | 2022-08-22 | 2022-08-22 | Method and apparatus for measuring contaminants |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115326749A true CN115326749A (en) | 2022-11-11 |
Family
ID=83926109
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211007383.6A Pending CN115326749A (en) | 2022-08-22 | 2022-08-22 | Method and apparatus for measuring contaminants |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115326749A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118483179A (en) * | 2024-07-12 | 2024-08-13 | 奥谱天成（厦门）光电有限公司 | Water quality detection method, device, medium and equipment |
-
2022
- 2022-08-22 CN CN202211007383.6A patent/CN115326749A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118483179A (en) * | 2024-07-12 | 2024-08-13 | 奥谱天成（厦门）光电有限公司 | Water quality detection method, device, medium and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110687072B (en) | Calibration set and verification set selection and modeling method based on spectral similarity | |
| CN110309867B (en) | Mixed gas identification method based on convolutional neural network | |
| CN102023137B (en) | Method for identifying white spirits | |
| CN105203498A (en) | Near infrared spectrum variable selection method based on LASSO | |
| CN107679569A (en) | Raman spectrum substance automatic identifying method based on adaptive hypergraph algorithm | |
| CN101339150A (en) | Method for determining octane number based on dielectric spectra technology | |
| CN102338743B (en) | Mid-infrared spectrum method for identifying engine fuel type and brand | |
| Yu et al. | Prediction of soil properties based on characteristic wavelengths with optimal spectral resolution by using Vis-NIR spectroscopy | |
| CN108827909B (en) | Rapid soil classification method based on visible near infrared spectrum and multi-target fusion | |
| CN102841069B (en) | Method for rapidly identifying types of crude oil by using mid-infrared spectrum | |
| CN115236044A (en) | Method and device for calculating concentration of soluble organic carbon in water environment by fluorescence spectrometry | |
| CN105223140A (en) | The method for quickly identifying of homology material | |
| Pillai | Estimating fuel adulteration in automobiles using robust optical fiber sensors | |
| CN113903407A (en) | Component identification method, component identification device, electronic equipment and storage medium | |
| CN110362911A (en) | A kind of agent model selection method of Design-Oriented process | |
| CN110910970B (en) | Method for predicting toxicity of chemicals by taking zebra fish embryos as receptors through building QSAR model | |
| Ye et al. | Water chemical oxygen demand prediction model based on the CNN and ultraviolet-visible spectroscopy | |
| CN116858822A (en) | Quantitative analysis method for sulfadiazine in water based on machine learning and Raman spectrum | |
| CN115326749A (en) | Method and apparatus for measuring contaminants | |
| CN117078960A (en) | Near infrared spectrum analysis method and system based on image feature extraction | |
| Liang et al. | Improved SVR based on CARS and BAS for hydrocarbon concentration detection | |
| CN115795225A (en) | Method and device for screening near infrared spectrum correction set | |
| EP2834624B1 (en) | A method for measuring performance of a spectroscopy system | |
| CN104297205A (en) | Method for quickly and nondestructively identifying edible oil | |
| CN104181125A (en) | Method for rapidly determining Kol-bach value of beer malt |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |