CN117372780A - For PM 2.5 Model training method, device and PM for prediction 2.5 Prediction method and device - Google Patents

For PM 2.5 Model training method, device and PM for prediction 2.5 Prediction method and device Download PDF

Info

Publication number
CN117372780A
CN117372780A CN202311467778.9A CN202311467778A CN117372780A CN 117372780 A CN117372780 A CN 117372780A CN 202311467778 A CN202311467778 A CN 202311467778A CN 117372780 A CN117372780 A CN 117372780A
Authority
CN
China
Prior art keywords
feature set
model
image data
initial
prediction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311467778.9A
Other languages
Chinese (zh)
Inventor
陶益凡
王立志
李�昊
贾京京
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Institute of Atmospheric Physics of CAS
Original Assignee
Institute of Atmospheric Physics of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Institute of Atmospheric Physics of CAS filed Critical Institute of Atmospheric Physics of CAS
Priority to CN202311467778.9A priority Critical patent/CN117372780A/en
Publication of CN117372780A publication Critical patent/CN117372780A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G06V10/765Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects using rules for classification or partitioning the feature space
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00Investigating characteristics of particles; Investigating permeability, pore-volume, or surface-area of porous materials
    • G01N15/06Investigating concentration of particle suspensions
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/211Selection of the most significant subset of features
    • G06F18/2115Selection of the most significant subset of features by evaluating different subsets according to an optimisation criterion, e.g. class separability, forward selection or backward elimination
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/27Regression, e.g. linear or logistic regression
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00Machine learning
    • G06N20/10Machine learning using kernel methods, e.g. support vector machines [SVM]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/0464Convolutional networks [CNN, ConvNet]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/13Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/30Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/35Determination of transform parameters for the alignment of images, i.e. image registration using statistical methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/46Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462Salient features, e.g. scale invariant feature transforms [SIFT]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/757Matching configurations of points or features
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20092Interactive image processing based on input by user
    • G06T2207/20104Interactive definition of region of interest [ROI]

Abstract

The application discloses a method for PM 2.5 Model training method for prediction and PM 2.5 The prediction method and the prediction device are used in the technical field of particulate matter concentration prediction. The PM is used for 2.5 The predicted model training method comprises the following steps: acquiring preprocessed image data of a long-time sequence and atmospheric environment observation data; acquiring an initial image basic feature set according to the image data of the long-time sequence; acquiring an initial atmospheric basic element feature set and an initial air quality basic element feature set according to the atmospheric environment observation data of the space corresponding to the long-time sequence; filtering the acquired initial image basic feature set, initial atmosphere basic element feature set and initial air quality basic element feature set so as to acquire a final feature set; obtaining a deep learning model; training the deep learning model through the final feature set. Under the condition of guaranteeing the optimal RMSE, the method effectively reduces the data dimension of deep learning, and realizes high-efficiency PM based on high-definition images 2.5 And (5) concentration prediction.

Description

For PM 2.5 Model training method, device and PM for prediction 2.5 Prediction method and device
Technical Field
The application relates to the technical field of particulate matter concentration prediction, in particular to a particle concentration prediction method for PM 2.5 Model training method for prediction, and PM 2.5 Model training device and PM for prediction 2.5 Prediction method and PM 2.5 And a prediction device.
Background
Current PM 2.5 The concentration data is mainly obtained by monitoring the air quality monitoring station on the ground by using professional instruments and equipment on an hour-by-hour basis. The method has the problems of sparse ground monitoring station distribution and high detection cost, wherein 1436 air quality monitoring stations are built in 338 cities in the grade of China and above, each city is only 4.25 on average, the station positions are unevenly distributed, the point location is always biased, and less ground air quality monitoring stations are distributed at the edges of the cities or villages, so that the sparse and uneven ground air quality monitoring stations cannot cover all corners of the cities, and the data provided by the sparse and uneven ground air quality monitoring stations are coarse-grained; the high construction cost of the air quality monitoring station limits the inability to increase the pollutant concentration monitoring density by increasing the number, since the monitoring station uses high precision professional equipment for foulingEstimation of the concentration of the dye therefore inevitably requires high costs in site creation, equipment acquisition, system operation and manual maintenance, making the data obtained thereof costly.
It is therefore desirable to have a solution that solves or at least alleviates the above-mentioned drawbacks of the prior art.
Disclosure of Invention
The invention aims to provide a method for PM 2.5 The predictive model training method solves at least one of the above-mentioned technical problems.
The invention provides the following scheme:
according to one aspect of the present invention, there is provided a method for PM 2.5 Model training method for prediction, the method for PM 2.5 The predicted model training method comprises the following steps:
step 1: acquiring preprocessed long-time-sequence image data and atmospheric environment observation data of a space corresponding to the long-time-sequence space which is matched with the long-time-sequence image data in a space-time manner;
step 2: acquiring an initial image basic feature set according to the image data of the long-time sequence;
step 3: acquiring an initial atmospheric basic element feature set and an initial air quality basic element feature set according to the atmospheric environment observation data of the space corresponding to the long-time sequence;
step 4: filtering the acquired initial image basic feature set, initial atmosphere basic element feature set and initial air quality basic element feature set so as to acquire a final feature set;
step 5: obtaining a deep learning model;
step 6: training the deep learning model through the final feature set.
Optionally, the method is used for PM before the preprocessed long-time series of image data and the atmospheric environmental observation data of the long-time series corresponding space which is matched with the long-time series of image data in a space-time mode are acquired 2.5 The predicted model training method further comprises:
preprocessing the long-time-series image data.
Optionally, the preprocessing the long-time-series image data includes:
extracting position calibration feature points of the image from each image data by using a SIFT algorithm;
searching a matching relation of two groups of characteristic points by combining a BFM algorithm and a K adjacent algorithm;
the RANSAS algorithm is used for carrying out position registration on the images by using a relation calculation transformation matrix of the matched characteristic points.
Optionally, the preprocessing the long-time-series image data further includes:
after the position registration, classifying the images according to the concentration threshold under the high-low concentration condition, and performing image edge detection based on a Sobel operator and an OSTU algorithm on each image data according to the high-low concentration classification;
marking a connected domain of the image data by using a Twopass algorithm, thereby selecting a range of the target region of interest;
extracting basic image characteristics of each image data corresponding to time, including transmissivity, contrast, information entropy and average gradient;
and extracting the atmospheric environment observation data for corresponding times, and performing quality control selection on the observation data according to a space consistency and extremum detection method, so as to obtain the preprocessed long-time sequence image data.
Optionally, the acquiring the deep learning model includes:
step 51: constructing a training model based on SVR;
step 52: constructing a deep learning training model based on classical CNN;
step 53: constructing a depth residual error network training model based on res Net-18;
step 54: and training the SVR-based training model, the classical CNN-based deep learning training model and the res Net-18-based deep residual error network training model respectively, and giving an optimal model as a deep learning model through training results of the three models.
Optionally, the filtering the acquired initial image basic feature set, the initial atmosphere basic feature set, and the initial air quality basic feature set, thereby acquiring a final feature set includes:
step 41: constructing a feature optimization model selected based on sequence backward features, combining a heuristic search strategy and a package type evaluation criterion, starting from inputting an initial image basic feature set, an initial atmosphere basic element feature set and an initial air quality basic element feature set, deleting each feature in sequence based on each feature dimension, calculating a model prediction error, and selecting a feature subset with the best prediction effect as a best feature subset of the dimension according to model prediction performance;
step 42: repeating step 41, thereby obtaining an optimal feature subset for each dimension;
step 43: and selecting a group with the smallest prediction error as a final feature set of the SBFS algorithm.
The present application also provides a method for PM 2.5 Model training apparatus for prediction, the model training apparatus for PM 2.5 The predicted model training device comprises:
the data acquisition module is used for acquiring preprocessed long-time sequence image data and long-time sequence corresponding space atmospheric environment observation data which are subjected to space-time matching with the long-time sequence image data;
the initial image basic feature set acquisition module is used for acquiring an initial image basic feature set according to the image data of the long-time sequence;
the system comprises an initial atmosphere basic element feature set and an initial air quality basic element feature set acquisition module, wherein the initial atmosphere basic element feature set and the initial air quality basic element feature set acquisition module are used for acquiring an initial atmosphere basic element feature set and an initial air quality basic element feature set according to the atmospheric environment observation data of the long-time sequence corresponding space;
the final feature set acquisition module is used for filtering the acquired initial image basic feature set, the initial atmosphere basic element feature set and the initial air quality basic element feature set so as to acquire a final feature set;
the deep learning model acquisition module is used for acquiring a deep learning model;
and the training module is used for training the deep learning model through the final feature set.
The present application also provides a PM 2.5 Prediction method of the PM 2.5 The prediction method comprises the following steps:
acquiring image data to be predicted;
acquiring atmospheric environment observation data when shooting the image data to be predicted;
acquiring image characteristics to be predicted according to the image data to be predicted;
acquiring atmospheric characteristics and air quality characteristics according to the atmospheric environment observation data;
acquisition by the above-described PM 2.5 A deep learning model trained by a predicted model training method;
and inputting the atmospheric characteristics and the air quality characteristics into the deep learning model so as to obtain a prediction result.
The present application also provides a PM 2.5 Prediction device, the PM 2.5 The prediction apparatus includes:
the image data acquisition module to be predicted is used for acquiring the image data acquisition module to be predicted;
the atmospheric environment observation data acquisition module is used for acquiring atmospheric environment observation data when the image data to be predicted are shot;
the image characteristic obtaining module to be predicted is used for obtaining image characteristics to be predicted according to the image data to be predicted;
the air characteristic and air quality characteristic acquisition module is used for acquiring the air characteristic and the air quality characteristic according to the atmospheric environment observation data;
a deep learning model acquisition module for acquiring the PM for the PM by the above 2.5 A deep learning model trained by a predicted model training method;
and the prediction module is used for inputting the atmospheric characteristics and the air quality characteristics into the deep learning model so as to obtain a prediction result.
For PM of the present application 2.5 Predictive model training method PM is characterized from images by long time series of high definition images using deep learning techniques and Sequential Backward Feature Selection (SBFS) techniques 2.5 The concentration basic characteristic information, the historical air quality information and other large amount of characteristic data are effectively and rapidly screened, the data dimension of deep learning is effectively reduced under the condition of guaranteeing the optimal RMSE, and the high-efficiency PM based on high-definition images is realized 2.5 And (5) concentration prediction.
Drawings
FIG. 1 is a schematic illustration of a system for PM in an embodiment of the present application 2.5 A flow diagram of a predicted model training method;
FIG. 2 is a block diagram of an electronic device according to one embodiment of the present application;
FIG. 3 is a schematic structural diagram of a classical CNN-based deep learning training model in an embodiment of the present application;
FIG. 4 is a schematic diagram of an SBFS process in an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present invention will be made apparent and fully in view of the accompanying drawings, in which some, but not all embodiments of the invention are shown. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
FIG. 1 is a schematic illustration of a system for PM in an embodiment of the present application 2.5 A schematic flow chart of a predictive model training method.
For PM as shown in FIG. 1 2.5 Model training for predictionThe method comprises the following steps:
step 1: acquiring preprocessed long-time-sequence image data and atmospheric environment observation data of a space corresponding to the long-time-sequence space which is matched with the long-time-sequence image data in a space-time manner;
step 2: acquiring an initial image basic feature set according to the image data of the long-time sequence;
step 3: acquiring an initial atmospheric basic element feature set and an initial air quality basic element feature set according to the atmospheric environment observation data of the space corresponding to the long-time sequence;
step 4: filtering the acquired initial image basic feature set, initial atmosphere basic element feature set and initial air quality basic element feature set so as to acquire a final feature set;
step 5: obtaining a deep learning model;
step 6: training the deep learning model through the final feature set.
The present application characterizes PM from images by long time series of high definition images using deep learning techniques and Sequential Backward Feature Selection (SBFS) techniques 2.5 The concentration basic characteristic information, the historical air quality information and other large amount of characteristic data are effectively and rapidly screened, the data dimension of deep learning is effectively reduced under the condition of guaranteeing the optimal RMSE, and the high-efficiency PM based on high-definition images is realized 2.5 And (5) concentration prediction.
In this embodiment, the method for PM is performed before the acquisition of the preprocessed long-time-series image data and the atmospheric environmental observation data of the long-time-series corresponding space that is space-time matched with the long-time-series image data 2.5 The predicted model training method further comprises:
preprocessing the long-time-series image data.
In this embodiment, the preprocessing the long-time-series image data includes:
extracting position calibration feature points of the image from each image data by using a SIFT algorithm;
searching a matching relation of two groups of characteristic points by combining a BFM algorithm and a K adjacent algorithm;
the RANSAS algorithm is used for carrying out position registration on the images by using a relation calculation transformation matrix of the matched characteristic points.
In this embodiment, the preprocessing the long-time-series image data further includes:
after the position registration, classifying the images according to the concentration threshold under the high-low concentration condition, and performing image edge detection based on a Sobel operator and an OSTU algorithm on each image data according to the high-low concentration classification;
marking a connected domain of the image data by using a Twopass algorithm, thereby selecting a range of the target region of interest;
extracting basic image characteristics of each image data corresponding to time, including transmissivity, contrast, information entropy and average gradient;
and extracting the atmospheric environment observation data for corresponding times, and performing quality control selection on the observation data according to a space consistency and extremum detection method, so as to obtain the preprocessed long-time sequence image data.
In this embodiment, the acquiring the deep learning model includes:
step 51: constructing a training model based on SVR;
step 52: constructing a classical CNN-based deep learning training model (FIG. 3 is a schematic diagram of a classical CNN-based deep learning training model);
step 53: constructing a depth residual error network training model based on res Net-18;
step 54: and training the SVR-based training model, the classical CNN-based deep learning training model and the res Net-18-based deep residual error network training model respectively, and giving an optimal model as a deep learning model through training results of the three models. In this embodiment, training the SVR-based training model, the classical CNN-based deep learning training model, and the res Net-18-based deep residual network training model may use an initial image basic feature set.
In this embodiment, the filtering the acquired initial image basic feature set, the initial atmosphere basic element feature set, and the initial air quality basic element feature set, so as to acquire a final feature set includes:
step 41: constructing a feature optimization model selected based on sequence backward features, combining a heuristic search strategy and a package type evaluation criterion, starting from inputting an initial image basic feature set, an initial atmosphere basic element feature set and an initial air quality basic element feature set, deleting each feature in sequence based on each feature dimension, calculating a model prediction error, and selecting a feature subset with the best prediction effect as a best feature subset of the dimension according to model prediction performance;
step 42: repeating step 41, thereby obtaining an optimal feature subset for each dimension;
step 43: and selecting a group with the smallest prediction error as a final feature set of the SBFS algorithm.
Image processing and quality control are carried out on image data of a long time sequence, wherein the image processing and quality control comprise image positioning offset, picture watermark removal, interested target area selection and the like caused by camera shake, and basic image characteristics, such as transmittance, contrast, information entropy, average gradient and the like, are extracted on the basis; also, quality control is performed on the atmospheric environmental observation data of the corresponding space of the long-time sequence, including space-time consistency test, extremum test, etc., and then elements including wind, temperature, pressure, humidity, PM are extracted 2.5 、PM 10 、SO 2 、CO、O 3 And extracting the data and performing space-time matching with the image.
Then, aiming at a pre-selected deep learning module, three models of a support vector machine regression model (SVR), a classical convolutional neural network model (CNN) and a depth residual network model (res Net-18) which are better in deep semantic processing performance of an image are selected, and respectively aiming at basic characteristics of the image, atmospheric basic element characteristics and air quality basic element characteristics, training is carried out, and a model with good effect is selected.
And selecting the optimal feature subset by utilizing the basic features including the images and the atmospheric environment obtained by the feature extraction according to a Sequence Backward Feature Selection (SBFS) algorithm, and selecting the most effective partial features aiming at the research content from the original overall features to form the optimal feature subset, thereby filtering out the invalid or redundant features, improving the model operation efficiency, reducing the model complexity and realizing feature dimension reduction.
Finally, training the model by using the screened deep learning model and the optimized subset, and performing PM according to the optimized model 2.5 Is a prediction of (2).
The present application is described in further detail below by way of examples, which are not to be construed as limiting the present application in any way.
Firstly, a scheduler of the whole system is realized by using python, and a data processing module such as data quality control, processing, feature extraction and the like of the whole system, a deep learning model training and screening module, a feature selection module and a final prediction module are all executed by a controller. When a new image is acquired, the following operations are taken: 1) extracting position calibration feature points of an image by using a SIFT algorithm, 2) searching a matching relation of two groups of feature points by combining a BFM algorithm and a K adjacent algorithm, and 3) calculating a transformation matrix by using a RANSAS algorithm and using the relation of the matching feature points to perform position registration of the image. 4) On the basis of registration, classifying images according to concentration threshold values under the condition of high and low concentrations, performing image edge detection based on a Sobel operator and an OSTU algorithm on the images according to the high and low concentration classifications, 5) marking connected domains of the images by utilizing a Twopass algorithm, thereby selecting a range of a target region of interest, and 6) extracting basic image features including transmittance, contrast, information entropy and average gradient corresponding to time; 7) And 8) extracting the atmospheric environment observation data for corresponding times, performing quality control selection on the observation data according to a space consistency and extremum detection method, and 8) storing the data.
Then, a deep learning model as well as model screening was implemented using Python. In the training process of each deep learning model, input data of a data extraction module is received. 1) Building a training model based on SVR, minimizing the total deviation of all samples by finding a function f (x) =omega-T ϕ (x) +b of fitting sample points as low as possible, obtaining a penalty coefficient and a kernel parameter representing RBF kernel breadth, and ensuring learning Kyoto and improving generalization capability by adjusting the penalty coefficient and the kernel parameter; 2) Constructing a deep learning training model based on classical CNN, using 5X5X24, 5X5X32 and 5X5X48 as a network low-layer convolution kernel, using 5X5X32, 5X5X48 and 5X5X64 as a network high-layer convolution kernel, and adopting a maximum pooling strategy and a ReLU activation function; 3) And constructing a depth residual error network training model based on res Net-18, using a 2-layer convolution block and 9 residual error blocks, and finally using a full connection layer to complete an image classification task, and introducing the residual error blocks on the basis of the convolution neural network to solve the problem of network degradation. 4) And respectively training by using the three models, and giving out an optimal model according to training results of the three models.
Referring to fig. 4, a feature optimization model based on Sequence Backward Feature Selection (SBFS) is constructed, a heuristic search strategy and a package type evaluation criterion are combined, starting from an input feature corpus, each feature is deleted in turn based on each feature dimension, a model prediction error is calculated, a feature subset with the best prediction effect is selected as a best feature subset of the dimension according to model prediction performance, the steps are repeated in a circulating manner until the feature dimension is reduced to 1, finally, a group with the smallest prediction error is selected as the best feature subset of the SBFS algorithm by comparing the best feature subsets of the dimensions. The basic steps of feature selection include: generating a process, evaluating criteria, stopping criteria and verifying results. It selects the optimal feature subset by performing the first three steps in a loop until a stopping criterion is met and verifies the result. The specific implementation flow is as follows: 1) Setting all the features to form an n-dimensional feature set D_n; 2) Deleting a feature from the feature set D_n so that the model prediction performance of the n-1 dimensional feature subset D_ (n-1) is optimal; 3) Repeating step 2 to obtain an n-2 dimensional feature subset D_ (n-2) and an n-3 dimensional feature subset D_ (n-3) … until a 1 dimensional feature subset D_1 is obtained; 4) Selecting a feature subset D_k with the lowest prediction error from D_n, D_ (n-1), D_ (n-2) and D_ (n-3) … D_1 according to the model prediction performance; 5) D_k is the optimal feature subset.
Finally, the SBFS preferred feature subset and the screened optimal deep learning model are utilized to carry out final training and prediction model construction, 80% of the prepared data set is randomly extracted to be used as a training set, 10% is used as a verification set, 10% is used as a test set, and finally, a prediction system model and parameters are given.
The application has the following advantages:
1) The invention applies the preselection combination based on the image basic feature and the atmospheric environment element feature subset to PM 2.5 In the prediction algorithm of (2), the SBFS feature optimization algorithm is used for removing redundant elements, so that the independence among the optimization features is higher, and the reduction of the element features means that the operation efficiency can be better improved.
2) According to the invention, the model screening of the image-based deep learning network is considered, and the optimal deep learning model screening under specific task conditions (the prediction of the air quality particulate matter concentration is performed based on image characteristics) is provided, so that the screened model can make the prediction more accurately, and the performance of model simulation is improved.
3) According to the invention, the historical characteristic information is considered, and the images at the previous times, the atmospheric environment characteristics and the element characteristics at the current time are brought into the model together, so that the model can sense the correlation of the atmospheric environment on a time scale, and the performance of the model is improved.
The present application also provides a method for PM 2.5 Model training apparatus for prediction, the model training apparatus for PM 2.5 The predicted model training device comprises a data acquisition module, an initial image basic feature set acquisition module, an initial atmosphere basic element feature set acquisition module, an initial air quality basic element feature set acquisition module, a final feature set acquisition module, a deep learning model acquisition module and a training module, wherein,
the data acquisition module is used for acquiring the preprocessed long-time sequence image data and the atmospheric environment observation data of a space corresponding to the long-time sequence which is in space-time matching with the long-time sequence image data;
the initial image basic feature set acquisition module is used for acquiring an initial image basic feature set according to the image data of the long-time sequence;
the initial atmosphere basic element feature set and initial air quality basic element feature set acquisition module is used for acquiring an initial atmosphere basic element feature set and an initial air quality basic element feature set according to the atmospheric environment observation data of the long-time sequence corresponding space;
the final feature set acquisition module is used for filtering the acquired initial image basic feature set, the initial atmosphere basic element feature set and the initial air quality basic element feature set so as to acquire a final feature set;
the deep learning model acquisition module is used for acquiring a deep learning model;
the training module is used for training the deep learning model through the final feature set.
The present application also provides a PM 2.5 Prediction method of the PM 2.5 The prediction method comprises the following steps:
acquiring image data to be predicted;
acquiring atmospheric environment observation data when shooting the image data to be predicted;
acquiring image characteristics to be predicted according to the image data to be predicted;
acquiring atmospheric characteristics and air quality characteristics according to the atmospheric environment observation data;
acquisition by the above-described PM 2.5 A deep learning model trained by a predicted model training method;
and inputting the atmospheric characteristics and the air quality characteristics into the deep learning model so as to obtain a prediction result.
The present application also provides a PM 2.5 Prediction device, the PM 2.5 The prediction device comprises an image data acquisition module to be predicted, an atmospheric environment observation data acquisition module, an image characteristic acquisition module to be predicted, an atmospheric characteristic and air quality characteristic acquisition module, a deep learning model acquisition module and a prediction module; wherein,
the image data acquisition module to be predicted is used for acquiring the image data acquisition module to be predicted;
the atmospheric environment observation data acquisition module is used for acquiring atmospheric environment observation data when the image data to be predicted are shot;
the image characteristic to be predicted acquisition module is used for acquiring image characteristics to be predicted according to the image data to be predicted;
the atmospheric characteristic and air quality characteristic acquisition module is used for acquiring atmospheric characteristic and air quality characteristic according to the atmospheric environment observation data;
a deep learning model acquisition module for acquiring the PM used by any one of claims 1 to 6 2.5 A deep learning model trained by a predicted model training method;
the prediction module is used for inputting the atmospheric characteristics and the air quality characteristics into the deep learning model so as to obtain a prediction result.
Fig. 2 is a block diagram of a client architecture provided by one or more embodiments of the invention.
As shown in fig. 2, the present application further discloses an electronic device, including: the device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus; a computer program is stored in the memory and when executed by the processor causes the processor to perform the method for PM 2.5 A step of a predictive model training method.
The present application also provides a computer-readable storage medium storing a computer program executable by an electronic device, the computer program being capable of being implemented for PM when run on the electronic device 2.5 A step of a predictive model training method.
The communication bus mentioned above for the electronic devices may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The electronic device includes a hardware layer, an operating system layer running on top of the hardware layer, and an application layer running on top of the operating system. The hardware layer includes hardware such as a central processing unit (CPU, central Processing Unit), a memory management unit (MMU, memory Management Unit), and a memory. The operating system may be any one or more computer operating systems that implement electronic device control via processes (processes), such as a Linux operating system, a Unix operating system, an Android operating system, an iOS operating system, or a windows operating system, etc. In addition, in the embodiment of the present invention, the electronic device may be a handheld device such as a smart phone, a tablet computer, or an electronic device such as a desktop computer, a portable computer, which is not particularly limited in the embodiment of the present invention.
The execution body controlled by the electronic device in the embodiment of the invention can be the electronic device or a functional module in the electronic device, which can call a program and execute the program. The electronic device may obtain firmware corresponding to the storage medium, where the firmware corresponding to the storage medium is provided by the vendor, and the firmware corresponding to different storage media may be the same or different, which is not limited herein. After the electronic device obtains the firmware corresponding to the storage medium, the firmware corresponding to the storage medium can be written into the storage medium, specifically, the firmware corresponding to the storage medium is burned into the storage medium. The process of burning the firmware into the storage medium may be implemented by using the prior art, and will not be described in detail in the embodiment of the present invention.
The electronic device may further obtain a reset command corresponding to the storage medium, where the reset command corresponding to the storage medium is provided by the provider, and the reset commands corresponding to different storage media may be the same or different, which is not limited herein.
At this time, the storage medium of the electronic device is a storage medium in which the corresponding firmware is written, and the electronic device may respond to a reset command corresponding to the storage medium in which the corresponding firmware is written, so that the electronic device resets the storage medium in which the corresponding firmware is written according to the reset command corresponding to the storage medium. The process of resetting the storage medium according to the reset command may be implemented in the prior art, and will not be described in detail in the embodiments of the present invention.
For convenience of description, the above devices are described as being functionally divided into various units and modules. Of course, the functions of each unit, module, etc. may be implemented in one or more pieces of software and/or hardware when implementing the present application.
It will be understood by those skilled in the art that all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs unless defined otherwise. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
For the purposes of simplicity of explanation, the methodologies are shown and described as a series of acts, it is to be understood and appreciated by one of ordinary skill in the art that the methodologies are not limited by the order of acts, as some acts may, in accordance with the methodologies, take place in other order or concurrently. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred embodiments, and that the acts are not necessarily required by the embodiments of the invention.
From the above description of embodiments, it will be apparent to those skilled in the art that the present application may be implemented in software plus a necessary general purpose hardware platform. Based on such understanding, the technical solutions of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, a server or a network device, etc.) to perform the methods described in the embodiments or some parts of the embodiments of the present application.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.

Claims (9)

1. Be used for PM 2.5 The model training method for prediction is characterized in that 2.5 The predicted model training method comprises the following steps:
step 1: acquiring preprocessed long-time-sequence image data and atmospheric environment observation data of a space corresponding to the long-time-sequence space which is matched with the long-time-sequence image data in a space-time manner;
step 2: acquiring an initial image basic feature set according to the image data of the long-time sequence;
step 3: acquiring an initial atmospheric basic element feature set and an initial air quality basic element feature set according to the atmospheric environment observation data of the space corresponding to the long-time sequence;
step 4: filtering the acquired initial image basic feature set, initial atmosphere basic element feature set and initial air quality basic element feature set so as to acquire a final feature set;
step 5: obtaining a deep learning model;
step 6: training the deep learning model through the final feature set.
2. The method for PM according to claim 1 2.5 The model training method for prediction is characterized in that before the preprocessed long-time-series image data and the atmospheric environment observation data of the long-time-series corresponding space which is matched with the long-time-series image data are obtained, the model training method for PM 2.5 The predicted model training method further comprises:
preprocessing the long-time-series image data.
3. The method for PM according to claim 2 2.5 A predictive model training method, wherein preprocessing the long-time-series image data includes:
extracting position calibration feature points of the image from each image data by using a SIFT algorithm;
searching a matching relation of two groups of characteristic points by combining a BFM algorithm and a K adjacent algorithm;
the RANSAS algorithm is used for carrying out position registration on the images by using a relation calculation transformation matrix of the matched characteristic points.
4. A method for PM according to claim 3 2.5 A predictive model training method, wherein preprocessing the long-time-series image data further comprises:
after the position registration, classifying the images according to the concentration threshold under the high-low concentration condition, and performing image edge detection based on a Sobel operator and an OSTU algorithm on each image data according to the high-low concentration classification;
marking a connected domain of the image data by using a Twopass algorithm, thereby selecting a range of the target region of interest;
extracting basic image characteristics of each image data corresponding to time, including transmissivity, contrast, information entropy and average gradient;
and extracting the atmospheric environment observation data for corresponding times, and performing quality control selection on the observation data according to a space consistency and extremum detection method, so as to obtain the preprocessed long-time sequence image data.
5. The method for PM according to claim 4 2.5 The predicted model training method is characterized in that the obtaining the deep learning model comprises the following steps:
step 51: constructing a training model based on SVR;
step 52: constructing a deep learning training model based on classical CNN;
step 53: constructing a depth residual error network training model based on res Net-18;
step 54: and training the SVR-based training model, the classical CNN-based deep learning training model and the res Net-18-based deep residual error network training model respectively, and giving an optimal model as a deep learning model through training results of the three models.
6. The method for PM according to claim 5 2.5 The method for training the predicted model is characterized in that the filtering the acquired initial image basic feature set, the initial atmosphere basic element feature set and the initial air quality basic element feature set so as to acquire a final feature set comprises the following steps:
step 41: constructing a feature optimization model selected based on sequence backward features, combining a heuristic search strategy and a package type evaluation criterion, starting from inputting an initial image basic feature set, an initial atmosphere basic element feature set and an initial air quality basic element feature set, deleting each feature in sequence based on each feature dimension, calculating a model prediction error, and selecting a feature subset with the best prediction effect as a best feature subset of the dimension according to model prediction performance;
step 42: repeating step 41, thereby obtaining an optimal feature subset for each dimension;
step 43: and selecting a group with the smallest prediction error as a final feature set of the SBFS algorithm.
7. Be used for PM 2.5 The model training device for prediction is characterized in that 2.5 The predicted model training device comprises:
the data acquisition module is used for acquiring preprocessed long-time sequence image data and long-time sequence corresponding space atmospheric environment observation data which are subjected to space-time matching with the long-time sequence image data;
the initial image basic feature set acquisition module is used for acquiring an initial image basic feature set according to the image data of the long-time sequence;
the system comprises an initial atmosphere basic element feature set and an initial air quality basic element feature set acquisition module, wherein the initial atmosphere basic element feature set and the initial air quality basic element feature set acquisition module are used for acquiring an initial atmosphere basic element feature set and an initial air quality basic element feature set according to the atmospheric environment observation data of the long-time sequence corresponding space;
the final feature set acquisition module is used for filtering the acquired initial image basic feature set, the initial atmosphere basic element feature set and the initial air quality basic element feature set so as to acquire a final feature set;
the deep learning model acquisition module is used for acquiring a deep learning model;
and the training module is used for training the deep learning model through the final feature set.
8. PM (particulate matter) 2.5 The prediction method is characterized in that the PM 2.5 The prediction method comprises the following steps:
acquiring image data to be predicted;
acquiring atmospheric environment observation data when shooting the image data to be predicted;
acquiring image characteristics to be predicted according to the image data to be predicted;
acquiring atmospheric characteristics and air quality characteristics according to the atmospheric environment observation data;
acquiring a PM by the method according to any one of claims 1 to 6 2.5 A deep learning model trained by a predicted model training method;
and inputting the atmospheric characteristics and the air quality characteristics into the deep learning model so as to obtain a prediction result.
9. PM (particulate matter) 2.5 The prediction device is characterized in that the PM 2.5 PredictionThe device comprises:
the image data acquisition module to be predicted is used for acquiring the image data acquisition module to be predicted;
the atmospheric environment observation data acquisition module is used for acquiring atmospheric environment observation data when the image data to be predicted are shot;
the image characteristic obtaining module to be predicted is used for obtaining image characteristics to be predicted according to the image data to be predicted;
the air characteristic and air quality characteristic acquisition module is used for acquiring the air characteristic and the air quality characteristic according to the atmospheric environment observation data;
a deep learning model acquisition module for acquiring the PM for the PM according to any one of claims 1 to 6 2.5 A deep learning model trained by a predicted model training method;
and the prediction module is used for inputting the atmospheric characteristics and the air quality characteristics into the deep learning model so as to obtain a prediction result.
CN202311467778.9A 2023-11-07 2023-11-07 For PM 2.5 Model training method, device and PM for prediction 2.5 Prediction method and device Pending CN117372780A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311467778.9A CN117372780A (en) 2023-11-07 2023-11-07 For PM 2.5 Model training method, device and PM for prediction 2.5 Prediction method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311467778.9A CN117372780A (en) 2023-11-07 2023-11-07 For PM 2.5 Model training method, device and PM for prediction 2.5 Prediction method and device

Publications (1)

Publication Number Publication Date
CN117372780A true CN117372780A (en) 2024-01-09

Family

ID=89403991

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311467778.9A Pending CN117372780A (en) 2023-11-07 2023-11-07 For PM 2.5 Model training method, device and PM for prediction 2.5 Prediction method and device

Country Status (1)

Country Link
CN (1) CN117372780A (en)

Similar Documents

Publication Publication Date Title
CN113469073B (en) SAR image ship detection method and system based on lightweight deep learning
US20180181867A1 (en) Artificial neural network class-based pruning
CN112132093B (en) High-resolution remote sensing image target detection method and device and computer equipment
CN109671020B (en) Image processing method, device, electronic equipment and computer storage medium
CN112287986B (en) Image processing method, device, equipment and readable storage medium
CN113537244B (en) Livestock image target detection method and device based on lightweight YOLOv4
CN111178438A (en) ResNet 101-based weather type identification method
WO2024041480A1 (en) Data processing method and related apparatus
CN112418212A (en) Improved YOLOv3 algorithm based on EIoU
CN114049515A (en) Image classification method, system, electronic device and storage medium
CN112381299A (en) Method and device for determining air quality
CN111340096A (en) Weakly supervised butterfly target detection method based on confrontation complementary learning
Wu et al. Compiler-aware neural architecture search for on-mobile real-time super-resolution
CN111598460A (en) Method, device and equipment for monitoring heavy metal content in soil and storage medium
Jakaria et al. Comparison of classification of birds using lightweight deep convolutional neural networks
CN112017730B (en) Cell screening method and device based on expression quantity prediction model
CN113570007A (en) Method, device and equipment for optimizing construction of part defect identification model and storage medium
CN117173568A (en) Target detection model training method and target detection method
CN116151323A (en) Model generation method, device, electronic equipment and storage medium
CN117372780A (en) For PM 2.5 Model training method, device and PM for prediction 2.5 Prediction method and device
CN112446231A (en) Pedestrian crossing detection method and device, computer equipment and storage medium
Peng et al. Camera captured DIQA with linearity and monotonicity constraints
CN113657453B (en) Detection method based on harmful website generating countermeasure network and deep learning
CN110942179A (en) Automatic driving route planning method and device and vehicle
CN114708467B (en) Bad scene identification method, system and equipment based on knowledge distillation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination