CN117312998A - Road surface covering classification device and method based on image-weather-temperature - Google Patents
Road surface covering classification device and method based on image-weather-temperature Download PDFInfo
- Publication number
- CN117312998A CN117312998A CN202311143753.3A CN202311143753A CN117312998A CN 117312998 A CN117312998 A CN 117312998A CN 202311143753 A CN202311143753 A CN 202311143753A CN 117312998 A CN117312998 A CN 117312998A
- Authority
- CN
- China
- Prior art keywords
- classification
- snow
- image
- ice
- temperature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 133
- 238000005286 illumination Methods 0.000 claims abstract description 19
- 230000005540 biological transmission Effects 0.000 claims abstract description 10
- 230000001419 dependent effect Effects 0.000 claims description 47
- 238000002474 experimental method Methods 0.000 claims description 30
- 238000004458 analytical method Methods 0.000 claims description 29
- 238000004422 calculation algorithm Methods 0.000 claims description 19
- 239000011159 matrix material Substances 0.000 claims description 15
- 238000010238 partial least squares regression Methods 0.000 claims description 15
- 239000013598 vector Substances 0.000 claims description 12
- CURLTUGMZLYLDI-UHFFFAOYSA-N Carbon dioxide Chemical compound O=C=O CURLTUGMZLYLDI-UHFFFAOYSA-N 0.000 claims description 11
- 238000009529 body temperature measurement Methods 0.000 claims description 11
- 235000011089 carbon dioxide Nutrition 0.000 claims description 11
- 230000000694 effects Effects 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 8
- 230000015556 catabolic process Effects 0.000 claims description 8
- 238000007405 data analysis Methods 0.000 claims description 8
- 238000006731 degradation reaction Methods 0.000 claims description 8
- 238000011156 evaluation Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 6
- 238000012216 screening Methods 0.000 claims description 6
- 238000013145 classification model Methods 0.000 claims description 5
- WHXSMMKQMYFTQS-UHFFFAOYSA-N Lithium Chemical compound [Li] WHXSMMKQMYFTQS-UHFFFAOYSA-N 0.000 claims description 4
- 238000013524 data verification Methods 0.000 claims description 4
- 230000007423 decrease Effects 0.000 claims description 4
- 229910052744 lithium Inorganic materials 0.000 claims description 4
- 230000008569 process Effects 0.000 claims description 4
- 238000012549 training Methods 0.000 claims description 4
- 230000017105 transposition Effects 0.000 claims description 3
- 238000012795 verification Methods 0.000 claims description 3
- 230000005611 electricity Effects 0.000 claims description 2
- 238000007792 addition Methods 0.000 claims 1
- 230000000994 depressogenic effect Effects 0.000 claims 1
- 239000000463 material Substances 0.000 claims 1
- 238000001514 detection method Methods 0.000 abstract description 27
- 238000012423 maintenance Methods 0.000 abstract description 14
- 238000010276 construction Methods 0.000 abstract description 5
- 238000012544 monitoring process Methods 0.000 abstract description 5
- 238000007500 overflow downdraw method Methods 0.000 description 17
- 230000004927 fusion Effects 0.000 description 6
- 238000003032 molecular docking Methods 0.000 description 6
- 239000011257 shell material Substances 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000011160 research Methods 0.000 description 5
- 239000010426 asphalt Substances 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 230000008018 melting Effects 0.000 description 3
- 238000002844 melting Methods 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000000750 progressive effect Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000002425 crystallisation Methods 0.000 description 1
- 230000008025 crystallization Effects 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001035 drying Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000008014 freezing Effects 0.000 description 1
- 238000007710 freezing Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 239000005457 ice water Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000010287 polarization Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 239000002352 surface water Substances 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/2431—Multiple classes
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01D—MEASURING NOT SPECIALLY ADAPTED FOR A SPECIFIC VARIABLE; ARRANGEMENTS FOR MEASURING TWO OR MORE VARIABLES NOT COVERED IN A SINGLE OTHER SUBCLASS; TARIFF METERING APPARATUS; MEASURING OR TESTING NOT OTHERWISE PROVIDED FOR
- G01D21/00—Measuring or testing not otherwise provided for
- G01D21/02—Measuring two or more variables by means not covered by a single other subclass
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/251—Fusion techniques of input or preprocessed data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/54—Extraction of image or video features relating to texture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention relates to a road surface covering classification device and method based on image-weather-temperature, and belongs to the technical field of road surface complex ice and snow covering classification. The road surface covering classification method solves the problems that the road surface covering classification in the prior art is only suitable for fixed-point monitoring and cannot meet the detection requirement of a large-area road surface. The intelligent temperature measuring device comprises a USB transmission line, a power module, a battery, an illumination humidity transmitter, a wind speed transmitter, an infrared temperature measuring transmitter, a camera, a processor and an output module, wherein the battery is used for supplying power through the power module, the processor is electrically connected with the illumination humidity transmitter, the wind speed transmitter and the infrared temperature measuring transmitter, the processor is electrically connected with the output module, the output module is electrically connected with a computer through the USB transmission line, and the camera is electrically connected with the computer through the USB. All parameter detection is non-contact detection, so that the method is easier to realize large-scale pavement detection, stable in equipment performance and easy to maintain, and has low construction, use and maintenance cost.
Description
Technical Field
The invention relates to a cover classifying device and method, and belongs to the technical field of road pavement complex ice and snow cover classification.
Background
The road pavement in cold areas can form ice and snow covers under the influence of strong snowfall and low temperature, and four conditions of new snow, granular snow, transparent ice and drying are common. Wherein, the new snow refers to new snowfall, and has the characteristics of small bulk density, small hardness and crystallization; the granular snow is granular snow formed by melting and then freezing new snow under the action of heat or pressure. The road ice and snow cover can seriously influence the running safety of the automobile, and the running safety coefficient of the automobile can be improved by accurately identifying the road ice and snow cover and early warning in advance. In addition, accurate road surface ice and snow cover type identification is significant to road maintenance departments, and accurate control of the amount of ice and snow melting agent used in ice and snow melting is facilitated, so that maintenance cost and influence on environment are reduced.
Aiming at the classification of ice and snow coverings on the road surface, methods based on single type of sensor data are mostly adopted, including a contact method, a weather method, a temperature method, a near infrared light intensity method, an image method and the like; the contact method is to embed capacitive, impedance, piezoelectric, optical fiber sensors into road surface for detection. For example: habib Tabatabai et al propose an embedded sensor to detect surface ice and humid conditions; the contact method has the problems of difficult maintenance and suitability for fixed-point monitoring, and cannot meet the detection requirement of a large-area pavement. The meteorological method is used for collecting data by means of a meteorological station and realizing the prediction of the road surface state by combining an expert system, but the accurate classification of the road surface state is difficult to realize. The temperature method is an effective auxiliary method for detecting the state of road surface covering, for example: riehm uses an infrared thermometer to conduct exothermic research of road ice water conversion; jonsson P studied the temperature differences of different covering states of the pavement; the temperature method has the defect that the performance of independently realizing road condition detection is insufficient. The near infrared light intensity method realizes detection according to different reflection and refractive indexes of infrared light by a pavement covering, for example: l.coloce et al measure diffuse reflection and reflected light under near infrared illumination, extract the polarization contrast after reflection to identify dry asphalt, water layer, wet asphalt, watered asphalt; the near infrared light intensity method has the problems of small detection coverage area, precise and expensive equipment and poor anti-interference performance. The image method realizes road surface state classification by combining image feature extraction with machine learning, deep learning and other technologies, for example: lushan Cheng et al propose a deep learning method for road surface condition classification based on a correction linear unit (ReLu) activation function; yang et al propose 5 road surface state classification methods based on a fused double-attention mechanism (EfficientNet); hojun Lee et al propose a black ice monitoring scheme based on convolutional neural networks; the image method is easy to realize the recognition of the state of the road surface in a large range, but the classification performance of complex coverings is limited.
Therefore, under the influence of the performance of a single detection method, aiming at the classification of complex ice and snow coverings on the road surface, the classification research based on image-weather fusion data is derived, for example: patrik Jonsson researches a modeling problem of performing covering state detection by combining an image method with meteorological data, wherein the adopted image data is a black-and-white image of the road surface; junyong You build a deep learning model of road images and meteorological data by using a regional convolutional neural network; as shown by early experimental researches, the detection performance of the complex ice and snow covering state is still to be improved by only fusing two methods, especially when the covering state is classified based on the traditional covering state type. The patent application discloses a multispectral-based road surface water ice and snow identification and classification method, publication No. CN113390796B, which adopts a method that a light emitting diode is embedded in a detection point, has the problems of difficult maintenance and suitability for fixed-point monitoring, and cannot meet the detection requirement of a large-area road surface.
Therefore, there is a need to provide an image-weather-temperature-based road cover classification device and method for solving the above-mentioned technical problems.
Disclosure of Invention
It is an object of the present invention to provide an image-weather-temperature based road cover classification apparatus and method for solving the problem that the road cover classification of the prior art is only suitable for spot monitoring and cannot meet the detection requirements of a large area road surface, and a brief overview of the present invention is given below to provide a basic understanding of some aspects of the present invention. It should be understood that this summary is not an exhaustive overview of the invention. It is not intended to identify key or critical elements of the invention or to delineate the scope of the invention.
The technical scheme of the invention is as follows:
the utility model provides a road surface covering sorter based on image-weather-temperature, includes USB transmission line, power module, battery, illuminance humidity transducer, wind speed transducer, infrared temperature measurement transducer, camera, treater and output module, the battery is used for supplying power through power module, treater and illuminance humidity transducer, wind speed transducer, infrared temperature measurement transducer electric connection, treater and output module electric connection, output module passes through USB transmission line and computer electric connection, the camera adopts USB and computer electric connection.
Preferably: the output module adopts USB to RS485, the battery adopts 24V lithium electricity, the illumination humidity transducer adopts 485 illumination humidity transducer, the wind speed transducer adopts 485 wind speed transducer, the infrared temperature measurement transducer adopts 485 infrared transducer or, the camera adopts 4800W pixel's CMOS camera.
Preferably: still include the shell, the shell inboard is provided with power module, battery, illuminance humidity transducer, wind speed changer, USB docking station, and illuminance humidity transducer, wind speed changer's detection end is located the shell outside, and the outside of shell is provided with infrared temperature measurement changer, camera, and the shell material is transparent PC board.
An image-weather-temperature based road surface covering classification method comprising the steps of:
step one: collecting data;
step two: extracting image features;
step three: VIP analysis;
step four: partial least squares modeling;
step five: classification experiments based on different classification methods;
step six: the performance of the classification method is compared when the road ice and snow cover types are classified finely.
Preferably: in the first step, the collected data types are temperature data, meteorological data and image data;
the ice and snow covered state object (cover for short) state comprises four types of new snow, granular snow, transparent ice and dry ice, and each data type comprises N samples; the temperature data comprise atmospheric temperature, body sensing temperature and ground surface temperature, wherein the atmospheric temperature and the body sensing temperature are obtained by a weather station, and the ground surface temperature is obtained by an infrared temperature measuring transmitter; the meteorological data comprise illumination, humidity and wind speed, wherein the illumination, the humidity and the wind speed are obtained by an illumination humidity transducer, and the wind speed is obtained by a wind speed transducer; the image data is shot and obtained by a camera;
extracting image features through image data, wherein the image features comprise color features and texture features, the color features comprise 18 first-order and second-order color moment parameters, and the texture features comprise 15 gray level co-occurrence matrix parameters;
converting the acquired RGB image into a YCbCr color space and an HSV color space respectively, extracting the mean value and the second moment of three color components in each color space respectively, converting the RGB image into a gray image, and extracting texture features by using a gray co-occurrence matrix;
in the third step, carrying out VIP analysis on input parameters of different classification methods, and determining an optimal partial least square model by changing a VIP threshold;
the VIP variable screening algorithm is an algorithm based on partial least squares regression analysis, describes the interpretation capability of the independent variable to the dependent variable, and screens the independent variable according to the interpretation capability; let m independent variables x 1 ，x 2 ,…x m And p dependent variables y 1 ,y 2 …y p The VIP calculation formula is as follows:
wherein k is a self-variable number, c h R as the main component of the extracted related argument 2 (y,c h ) Is the correlation coefficient of the dependent variable and the principal component, and represents the interpretation ability of the principal component to y, w hj Representing the weight of the argument on the principal component;
by the main component c h To convey x to y interpretation capability; if c h The interpretation ability of y is strong, and x versus c h The effect of (2) is very large, and the interpretation effect of x on y can be considered to be large; in experimental analysis, determining variables selected in a classification experiment through different VIP values;
in step four, the partial least square method first refers to m independent variables x with correlation 1 ，x 2 ,…x m And p dependent variables y 1 ,y 2 …y p Performing mathematical analysis of the principal component, and then extracting a principal independent component u 1 And a dependent variable principal component v 1 While the main component u 1 And v 1 The correlation degree of the main component reaches the maximum correlation degree of all main components; then build dependent variable y 1 ,y 2 …y p And independent variable x 1 ，x 2 ,…x m Regression equations of the principal components, and performing data verification; if the precision does not meet the requirement of the experimental result, continuing to perform secondary data processing on the data to extract a second pair of main components; by analogy, if n pairs of principal components are extracted in total in the experiment; then y can be obtained 1 ,y 2 …y p Respectively with u1, u 2 …u n Regression equation of u 1 ,u 2 …u n Each main component u of (a) i And x 1 ，x 2 ,…x m All are linear correlation combinations, so that a partial least squares regression equation can be obtained, and y 1 ,y 2 …y p And x 1 ，x 2 ,…x m The regression equation of (2) is a partial least squares regression equation;
let m independent variables x 1 ,x 2 …x m And p dependent variables y 1 ,y 2 …y p The formula after matrix standardization is shown as (1), wherein n represents the observation times;
wherein A, B is the independent variable group and several times of standardized observation data of the dependent variable group, and a and b are the values of the independent variable and the dependent variable;
step four comprises the following steps:
step 4.1: extracting a first pair of principal components u required by data analysis according to the mathematical linear correlation combination of the independent variable and the dependent variable 1 And v 1 Wherein u is 1 ＝α 11 x 1 +…α 1m x m ＝ρ (1)T X,v 1 ＝β 11 y 1 +…β 1p y p ＝γ (1)T Y, can be obtained:
wherein u is 1 、v 1 The first pair of principal components proposed for the two sets of variables, u 1 Is the independent variable set x= [ X ] 1 ，...，x m ] T V 1 Is the dependent variable set y= [ Y ] 1 ，…，y p ] T T represents matrix transposition, X is column vector of independent variable, Y is column vector of dependent variable, alpha and beta represent linear coefficient (constant), ρ (1) 、λ (1) Represents a vector of coefficients and,is a first pair of principal components u 1 、v 1 Is a component of (2);
step 4.2: the formula for calculation A, B is:
wherein sigma (1) 、τ (1) Parameter vectors in the regression model (called model effect load quantity), A 1 、B 1 Is a residual array;
step 4.3: stopping extracting main components required by data analysis if the regression accuracy meets the requirement; if the analysis step does not meet the requirement, repeating the analysis step; when r (the representative number) major components are extracted, it is possible to obtain:
from this, a partial least squares regression equation can be obtained:
y i ＝c j1 x 1 +…+c jm x m ＝1,2,…,p (5)
wherein j=1, 2 … p represents the number of the dependent variable coefficients added;
in the fifth step, N samples of each road surface covering type are divided into a training set and a verification set according to a ratio of 3:2 for the total 3N samples;
establishing a classification model of the road ice and snow covering state by adopting a partial least square algorithm, and respectively assigning values according to 1 to 4 for four covering state types in the modeling process; the VIP algorithm is adopted to input parameters to determine an optimal model, and the selected evaluation indexes are the average classification precision AP (Average Precision) of four ice and snow cover types (new snow, granular snow, transparent ice and dry):
p in the formula i For the classification precision of different ice and snow cover types, n is the number of ice and snow cover types, and the higher the average precision AP value is, the better the performance of the classification method is;
in the sixth step, the performance of different classification methods in complex covering classification is verified, and the comparison experiment flow comprises the following steps:
step 6.1: performing classification experiments of three traditional ice and snow cover types oriented to dry, transparent ice and new snow to obtain average classification accuracy AP of seven classification methods t ；
Step 6.2: increasing the complexity of the covering state by categorizing the traditional covering types; on the basis of new snow, similar types of granular snow are added, classification experiments facing four ice and snow cover types of dry ice, transparent ice, new snow and granular snow are carried out, and average classification precision AP of seven classification methods is obtained f ；
Step 6.3: with the average accuracy degradation rate DRAP (Decline Rate of Average Precision) as an index, the degradation of the average classification accuracy when classifying the different classification methods from three types of coverings to four types of coverings, namely the performance evaluation of the classification method when the complexity of the road ice and snow coverings is increased, is evaluated:
AP in t For average accuracy when classifying three types of covers, AP f For the average precision when classifying four types of covers, the smaller the DRAP value is, the better the performance of the classification method when classifying complex covers is.
Preferably: in step one, each data type contains n=120 samples, and in step five, 120 samples of each road surface cover type, total 480 samples.
The invention has the following beneficial effects:
all parameter detection is non-contact detection, so that the large-scale pavement detection is easier to realize, the equipment performance is stable, the equipment is easy to maintain, and the construction, use and maintenance cost of the invention is low;
the invention can be applied to various use environments such as pavement detection, intelligent traffic, automatic driving, safe driving, road maintenance and the like, and the adaptability of the invention is improved by combining the construction, maintenance and maintenance costs;
according to the invention, the road ice and snow cover is accurately classified, namely, the differential characteristics of the ice and snow cover subdivision types are better extracted, so that the classification precision is improved.
Drawings
FIG. 1 is a schematic diagram of a road surface covering classification device based on image-weather-temperature;
FIG. 2 is an electrical connection diagram of an image-weather-temperature based pavement covering classification device;
FIG. 3 is a top computer interface diagram;
FIG. 4 is a diagram of a method of classifying pavement coverings based on image-weather-temperature;
fig. 5 is a classification diagram based on different classification methods.
In the figure: the intelligent temperature measuring device comprises a 1-USB transmission line, a 2-shell, a 3-power module, a 4-battery, a 5-illuminance humidity transmitter, a 6-wind speed transmitter, a 7-USB docking station, an 8-infrared temperature measuring transmitter and a 9-camera.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the present invention is described below by means of specific embodiments shown in the accompanying drawings. It should be understood that the description is only illustrative and is not intended to limit the scope of the invention. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the present invention.
The first embodiment is as follows: referring to fig. 1-3, a road surface covering classification device based on image-weather-temperature in this embodiment is described, which includes a USB transmission line 1, a power module 3, a battery 4, an illuminance humidity transmitter 5, a wind speed transmitter 6, a USB docking station 7, an infrared temperature measurement transmitter 8, a camera 9, a processor (CPU) and an output module, where the battery 4 is electrically connected with the illuminance humidity transmitter 5, the wind speed transmitter 6, the USB docking station 7, the infrared temperature measurement transmitter 8, the output module and the processor through the power module 3, the processor is electrically connected with the illuminance humidity transmitter 5, the wind speed transmitter 6, the infrared temperature measurement transmitter 8 and the USB docking station 7, the processor is electrically connected with the output module, the output module is electrically connected with a computer through the USB transmission line 1, and the camera 9 is electrically connected with the computer through a USB; all parameter detection is non-contact detection, so that the method is easier to realize large-scale pavement detection, stable in equipment performance and easy to maintain, and has low construction, use and maintenance cost.
The second embodiment is as follows: 1-3, an image-weather-temperature-based road surface covering classification device in the embodiment is described, an output module adopts USB to RS485, a battery 4 adopts 24V lithium battery, the 24V lithium battery is connected with LM2596 four paths of direct current adjustable stabilized voltage power supply, each path of the device can be independently voltage-adjusted according to the voltage requirement of a sensor, an illumination humidity transmitter 5 adopts 485 illumination humidity transmitter, a wind speed transmitter 6 adopts 485 type wind speed transmitter, an infrared temperature measuring transmitter 8 adopts 485 type infrared transmitter or a camera 9 adopts a CMOS camera with 4800W pixels, the illumination humidity transmitter 5, the wind speed transmitter 6, the infrared temperature measuring transmitter 8 and the camera 9 are all data acquisition parts, the sensors adopt an RS485 data transmission mode and a MODBUS communication protocol, and a computer (upper computer) operation interface utilizes AppDesigh in MAT to write, and the device comprises five parts including serial port communication, data acquisition, real-time image display, real-time image processing and detection modules as shown in FIG. 3; the method realizes real-time image display and acquisition, real-time image processing (extracting image textures and color characteristics), real-time temperature, meteorological data acquisition, fusion temperature, meteorological and image characteristic information, achieves the aim of accurately classifying the road ice and snow cover, can be applied to various use environments such as road surface detection, intelligent traffic, automatic driving, safe driving, road maintenance and the like, and combines the construction, maintenance and maintenance costs of the road ice and snow cover to improve the adaptability of the road ice and snow cover; the invention integrates the weather, the temperature sensor and the high-definition camera to collect the data information of the road ice and snow cover, and utilizes the AppDesigner in the matlab to design the upper computer interface for real-time communication, so that the detection and classification are accurate, and the operation is simple.
And a third specific embodiment: referring to fig. 1-3, the road surface covering classification device based on image-weather-temperature in this embodiment is described, and further includes a housing 2, a power module 3, a battery 4, an illuminance humidity transmitter 5, a wind speed transmitter 6, a USB docking station 7 are disposed on the inner side of the housing 2, detection ends of the illuminance humidity transmitter 5 and the wind speed transmitter 6 are located on the outer side of the housing 2, an infrared temperature measuring transmitter 8 and a camera 9 are disposed on the outer side of the housing 2, and the housing 2 is made of a transparent PC board.
The specific embodiment IV is as follows: referring to fig. 1 to 5, a road surface covering classification method based on image-weather-temperature according to the present embodiment is described, and the road surface covering classification device based on image-weather-temperature is adopted, comprising the steps of:
step one: collecting data;
step two: extracting image features;
step three: VIP analysis;
step four: partial least squares modeling;
step five: classification experiments based on different classification methods;
step six: performance comparison of classification methods when the road ice and snow cover types are classified finely;
according to the invention, by reflecting the better combination of the differential characteristics of the complex covering, the classification precision of the road covering is higher compared with that of the conventional method, the gradient rate of the classification precision is lower when the road covering types are finely classified, the accuracy is high, and the problem that the accuracy is lower when the road covering is classified by using the underground sensor or monitor is solved.
Fifth embodiment: 1-5, in the method for classifying road coverings based on image-weather-temperature according to the present embodiment, in the first step, data types including temperature data, weather data and image data are collected, and the data are input into a computer;
the ice and snow covered state object (cover for short) state comprises four types of new snow, granular snow, transparent ice and dry ice, and each data type comprises N samples; the temperature data comprise atmospheric temperature, body sensing temperature and ground surface temperature, the atmospheric temperature and the body sensing temperature are acquired by a weather station and manually input into a computer, and the ground surface temperature is acquired by an infrared temperature measuring transmitter 8; the meteorological data comprise illumination, humidity and wind speed, the illumination, the humidity are obtained by the illumination humidity transducer 5, and the wind speed is obtained by the wind speed transducer 6; the image data is shot and obtained by the camera 9;
extracting image features through image data, wherein the image features comprise color features and texture features, the color features comprise 18 first-order and second-order color moment parameters, and the texture features comprise 15 gray level co-occurrence matrix parameters; the image characteristic parameter types are shown in table one:
table one: image characteristic parameter type
Converting the acquired RGB image into a YCbCr color space and an HSV color space respectively, extracting the mean value and the second moment of three color components in each color space respectively, converting the RGB image into a gray level image for extracting texture features, and extracting the texture features by using a gray level co-occurrence matrix;
in the third step, carrying out VIP analysis on input parameters of different classification methods, and determining an optimal partial least square model by changing a VIP threshold; the classifying method is VIP analysis of four corresponding covering sample parameters under each classifying method, for example, only temperature data is analyzed under a temperature method, temperature and weather data are analyzed under a temperature-weather method, and texture and color data are analyzed under an image method; the classification methods involved in this study are seven: temperature method, weather method, image method, temperature-weather fusion, temperature-image fusion, weather-image fusion, temperature-weather-image fusion;
the VIP variable screening algorithm is an algorithm based on partial least squares regression analysis, describes the interpretation capability of the independent variable to the dependent variable, and screens the independent variable according to the interpretation capability; let m independent variables x 1 ，x 2 ,…x m And p dependent variables y 1 ,y 2 …y p The VIP calculation formula is as follows:
wherein k is a self-variable number, c h R as the main component of the extracted related argument 2 (y,c h ) Is the correlation coefficient of the dependent variable and the principal component, and represents the interpretation ability of the principal component to y, w hj Representing the weight of the argument on the principal component;
by the main component c h To convey x to y interpretation capability; if c h The interpretation ability of y is strong, and x versus c h The effect of (2) is very large, and the interpretation effect of x on y can be considered to be large; in experimental analysis, determining variables selected in a classification experiment through different VIP values;
in step four, the partial least square method first refers to m independent variables x with correlation 1 ，x 2 ,…x m And p dependent variables y 1 ,y 2 …y p Performing mathematical analysis of the principal component, and then extracting a principal independent component u 1 And a dependent variable principal component v 1 While the main component u 1 And v 1 The correlation degree of the main component reaches the maximum correlation degree of all main components; then build dependent variable y 1 ,y 2 …y p And independent variable x 1 ，x 2 ,…x m Regression equations of the principal components, and performing data verification; if the precision does not meet the requirement of the experimental result, continuing to perform secondary data processing on the data to extract a second pair of main components; by analogy, if n pairs of principal components are extracted in total in the experiment; then y can be obtained 1 ,y 2 …y p Respectively with u 1 ,u 2 …u n Regression equation of u 1 ,u 2 …u n Each main component u of (a) i And x 1 ，x 2 ,…x m All are linear correlation combinations, so that a partial least squares regression equation can be obtained, and y 1 ,y 2 …y p And x 1 ，x 2 ,…x m The regression equation of (2) is a partial least squares regression equation;
let m independent variables x 1 ,x 2 …x m And p dependent variables y 1 ,y 2 …y p The formula after matrix standardization is shown as (1), wherein n represents the observation times;
wherein A, B is the independent variable group and several times of standardized observation data of the dependent variable group, and a and b are the values of the independent variable and the dependent variable;
step four comprises the following steps:
step 4.1: extracting a first pair of principal components u required by data analysis according to the mathematical linear correlation combination of the independent variable and the dependent variable 1 And v 1 Wherein u is 1 ＝α 11 x 1 +…α 1m x m ＝ρ (1)T X,v 1 ＝β 11 y 1 +…β 1p y p ＝γ (1)T Y, can be obtained:
wherein u is 1 、v 1 The first pair of principal components proposed for the two sets of variables, u 1 Is the independent variable set x=x 1 ，…，x m ] T V 1 Is the dependent variable set y= [ Y ] 1 ，…，y p ] T T represents matrix transposition, X is column vector of independent variable, Y is column vector of dependent variable, alpha and beta represent linear coefficient (constant), ρ (1) 、γ (1) Represents a vector of coefficients and,is a first pair ofMain component u 1 、v 1 Is a component of (2);
step 4.2: the formula for calculation A, B is:
wherein sigma (1) 、τ (1) Parameter vectors in the regression model (called model effect load quantity), A 1 、B 1 Is a residual array;
step 4.3: stopping extracting main components required by data analysis if the regression accuracy meets the requirement; if the analysis step does not meet the requirement, repeating the analysis step; when r (the representative number) major components are extracted, it is possible to obtain:
from this, a partial least squares regression equation can be obtained:
y i ＝c j1 x 1 +…+c jm x m ＝1,2,…,p (5)
wherein j=1, 2 … p represents the number of the dependent variable coefficients added;wherein (1)>Is u 1 The number of lines (number of observations) in A is represented by k, c jm Regression coefficients that are partial minimum regression equations;
in the fifth step, N samples of each road surface covering type are divided into a training set and a verification set according to a ratio of 3:2 for the total 3N samples; the method adopted by the classification experiment comprises a weather method, a temperature method, an image method, a weather-temperature fusion method, an image-weather fusion method, an image-temperature fusion method and an image-weather-temperature fusion method; the meteorological method is to take collected meteorological data as classified input parameters; the weather-temperature fusion method is to take weather data and temperature data as classified input parameters; all other five methods are so;
sequentially carrying out a VIP screening algorithm and a PLS classification algorithm on meteorological data, temperature data and image data after image feature extraction to obtain classification accuracy; establishing a classification model of the road ice and snow covering state by adopting a partial least square algorithm, and respectively assigning values according to 1 to 4 for four covering state types in the modeling process; the VIP algorithm is adopted to carry out input parameters preferably to determine an optimal model, and the selected evaluation indexes are the average classification precision AP (Average Precision) of four ice and snow cover types (new snow, granular snow, transparent ice and dry):
p in the formula i For the classification precision of different ice and snow cover types, n is the number of ice and snow cover types, and the higher the average precision AP value is, the better the performance of the classification method is;
in the sixth step, as the complexity type of the pavement ice and snow cover is increased, that is, the performance of different classification methods is reduced when the pavement ice and snow cover is classified according to the traditional type, the performance of different classification methods when the pavement ice and snow cover is classified according to the average classification accuracy progressive ratio comparison experiment of the different classification methods is verified when the pavement ice and snow cover is classified according to the traditional type, and the comparison experiment flow comprises the following steps:
step 6.1: performing classification experiments of three traditional ice and snow cover types oriented to dry, transparent ice and new snow to obtain average classification accuracy AP of seven classification methods t ；
Step 6.2: increasing the complexity of the covering state by categorizing the traditional covering types; on the basis of new snow, similar types of granular snow are added, classification experiments facing four ice and snow cover types of dry ice, transparent ice, new snow and granular snow are carried out, and average classification precision AP of seven classification methods is obtained f ；
Step 6.3: with the average accuracy degradation rate DRAP (Decline Rate of Average Precision) as an index, the degradation of the average classification accuracy when classifying the different classification methods from three types of coverings to four types of coverings, namely the performance evaluation of the classification method when the complexity of the road ice and snow coverings is increased, is evaluated:
AP in t For average accuracy when classifying three types of covers, AP f For the average precision when the method is used for classifying four types of covers, the smaller the DRAP value is, the better the performance of the classification method when the method is used for classifying complex covers is; accuracy result comparison index under different road surface covering complexity: average precision taper rate; the novel method is compared with the classification performances of the traditional weather method, temperature method, image method, weather-temperature fusion method, image-weather fusion method and image-temperature fusion method, and the results show that when the novel method is used for classifying ice and snow coverings of four road surfaces, namely, new snow, granular snow, transparent ice and dry ice, the average classification precision of the method provided by the invention can be improved by 8.4% -40.1%; when the complexity of the pavement covering is increased, the average classification accuracy can be reduced by 8.4% -40.1%; the method aims at the fact that the classification precision of the complex ice and snow cover of the pavement in winter is higher.
Specific embodiment six: referring to fig. 1 to 5, a road surface covering classification method based on image-weather-temperature according to the present embodiment is described, wherein each data type in the first step includes n=120 samples, and in the fifth step, 120 samples of each road surface covering type include 480 samples.
Example 1:
according to the invention, the road ice and snow cover is accurately classified, namely, the differential characteristics of the ice and snow cover subdivision types are better extracted, so that the classification precision is improved; firstly, RGB images are collected for four cover types of new snow, grain snow, transparent ice and dry in an actual road surface in winter, texture and color characteristics are extracted, meteorological data such as humidity, wind speed and illuminance and temperature data such as surface temperature and atmospheric temperature are recorded in real time. Then, variable projection importance analysis (VIP) algorithm is adopted to perform variable screening, and classification model is built based on partial least squares algorithm (PLS). Finally, carrying out road ice and snow cover classification experiments based on a weather method, a temperature method, an image method, a weather-temperature fusion method, an image-weather fusion method, an image-temperature fusion method and an image-weather-temperature fusion method under different road complexities; the problems that when the road surface of the cold region is subjected to strong snowfall in winter and long-term low-temperature influence, complex ice and snow covers are easy to form, and the safety driving and road maintenance of the automobile are adversely affected are solved;
step one (data acquisition): the ice and snow covered state object state comprises four types of new snow, granular snow, transparent ice and dry ice, wherein each type comprises 120 samples; synchronizing the image, weather and temperature data in real time;
step two (image feature extraction): the image features are color and texture; wherein the color features are 18 first-order and second-order color moment parameters; the texture features are 15 gray level co-occurrence matrix parameters; the image characteristic parameter types are shown in the table:
image characteristic parameter type
Converting the acquired RGB image into a YCbCr color space and an HSV color space respectively, and extracting the mean value and the second moment of three color components in each color space respectively; in order to extract texture features, converting an RGB image into a gray image, and then utilizing a gray co-occurrence matrix to extract the texture features;
step three (VIP analysis: performing VIP analysis on input parameters of different classification methods, and determining an optimal partial least square model by changing a VIP threshold; VIP variable screening algorithm is based on partial least squares backAn algorithm of analysis describes the interpretation capability of the independent variable to the dependent variable, and screens the independent variable according to the interpretation capability; let m independent variables x 1 ，x 2 ,…x m And p dependent variables y 1 ,y 2 …y p The VIP calculation formula is as follows:
wherein k is a self-variable number, c h R as the main component of the extracted related argument 2 (y,c h ) Is the correlation coefficient of the dependent variable and the principal component, and represents the interpretation ability of the principal component to y, w hj Representing the weight of the argument on the principal component;
by the main component c h To convey x to y interpretation capability; if c h The interpretation ability of y is strong, and x versus c h The effect of (2) is very large, and the interpretation effect of x on y can be considered to be large; in experimental analysis, determining variables selected in a classification experiment through different VIP values;
step four (partial least squares modeling): the partial least squares method first has a correlation to m independent variables x 1 ，x 2 ,…x m And p dependent variables y 1 ,y 2 …y p Performing mathematical analysis of the principal component, and then extracting a principal independent component u 1 And a dependent variable principal component v 1 While the main component u 1 And v 1 The correlation degree of the main component reaches the maximum correlation degree of all main components; then build dependent variable y 1 ,y 2 …y p And independent variable x 1 ，x 2 ,…x m Regression equations of the principal components, and performing data verification; if the precision does not meet the requirement of the experimental result, continuing to perform secondary data processing on the data to extract a second pair of main components; by analogy, if n pairs of principal components are extracted in total in the experiment; then y can be obtained 1 ,y 2 …y p Respectively with u 1 ,u 2 …u n Regression equation of u 1 ,u 2 …u n Each main component u of (a) i And x 1 ，x 2 ,…x m All are linear correlation combinations, so that a partial least squares regression equation can be obtained, and y 1 ,y 2 …y p And x 1 ，x 2 ,…x m The regression equation of (2) is a partial least squares regression equation;
let m independent variables x 1 ,x 2 …x m And p dependent variables y 1 ,y 2 …y p The formula after matrix standardization is shown as (1), wherein n represents the observation times;
the first step: extracting a first pair of principal components u required by data analysis according to the mathematical linear correlation combination of the independent variable and the dependent variable 1 And v 1 Wherein u is 1 ＝a 11 x 1 +…a 1m x m ＝ρ (1)T X,v 1 ＝β 11 y 1 +…β 1p y p ＝γ (1)T Y, can be obtained:
and a second step of: the formula for calculation A, B is:
and a third step of: stopping extracting main components required by data analysis if the regression accuracy meets the requirement; and if the requirement is not met, repeating the analysis steps. When r major components are extracted, it is possible to obtain:
from this, a partial least squares regression equation can be obtained:
y i ＝c j1 x 1 +…+c jm x m ＝1,2,…,p (5)
step five (classification experiments based on different classification methods): 120 samples of each road surface cover type, a total of 480 samples divide the training set and the validation set by 3:2. The method adopted by the classification experiment comprises a weather method, a temperature method, an image method, a weather-temperature fusion method, an image-weather fusion method, an image-temperature fusion method and an image-weather-temperature fusion method; the classification experimental procedure is shown in fig. 5:
establishing a classification model of the road ice and snow covering state by adopting a partial least square algorithm, and respectively assigning values according to 1 to 4 for four covering state types in the modeling process; the VIP algorithm is adopted to carry out input parameters preferably to determine an optimal model, and the selected evaluation indexes are the average classification precision AP (Average Precision) of four ice and snow cover types (new snow, granular snow, transparent ice and dry):
p in the formula i The classification precision of different ice and snow cover types is adopted, and n is the number of the ice and snow cover types; the higher the average accuracy AP value is, the better the classification method performance is;
step six (performance comparison of classification method when the road ice and snow cover types are finely classified): in the research, the performance of different classification methods is reduced along with the increase of the complexity type of the road ice and snow cover, namely, the performance of the different classification methods is reduced when the road ice and snow cover is classified based on the traditional type; therefore, an average classification precision progressive ratio comparison experiment of different classification methods in fine classification of the road ice and snow cover is carried out, and the performance of the different classification methods in complex cover classification is verified; the comparative experimental procedure is shown below:
carrying out classification experiments of three traditional ice and snow cover types oriented to dry, transparent ice and new snow to obtain average classification precision APt of seven classification methods;
increasing the complexity of the covering state by categorizing the traditional covering types; adding similar types of granular snow on the basis of new snow, and carrying out classification experiments facing four ice and snow cover types of dry ice, transparent ice, new snow and granular snow to obtain average classification precision APf of seven classification methods;
with the average accuracy degradation rate DRAP (Decline Rate of Average Precision) as an index, the degradation of the average classification accuracy when classifying the different classification methods from three types of coverings to four types of coverings, namely the performance evaluation of the classification method when the complexity of the road ice and snow coverings is increased, is evaluated:
AP in t For average accuracy when classifying three types of covers, AP f Average accuracy when classifying for four types of covers; the smaller the DRAP value is, the better the performance of the classification method is when the classification method is oriented to the fine classification of complex covers;
experimental results show that the average classification precision of the image-weather-temperature fusion method provided by the method for classifying new snow, granular snow, transparent ice and dry road coverings can reach 93.7%, and the average classification precision can be improved by 8.4% -40.1% compared with other six classification methods; when the road surface covering types are finely classified, namely, 3 types of transparent ice, new snow and dry ice are added into 4 types of new snow, granular snow, transparent ice and dry ice, the reduction rate of the average classification precision of the method disclosed herein can be reduced by 8.4% -40.1% compared with the traditional method; experimental results prove that compared with the traditional method, the method provided by the invention has higher classification precision aiming at the fine classification of the complex ice and snow cover of the winter road surface.
It should be noted that, in the above embodiments, as long as the technical solutions that are not contradictory can be arranged and combined, those skilled in the art can exhaust all the possibilities according to the mathematical knowledge of the arrangement and combination, so the present invention does not describe the technical solutions after the arrangement and combination one by one, but should be understood that the technical solutions after the arrangement and combination have been disclosed by the present invention.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (6)
1. An image-weather-temperature-based road surface covering classification device, characterized in that: including USB transmission line (1), power module (3), battery (4), illuminance humidity transducer (5), wind speed changer (6), infrared temperature measurement changer (8), camera (9), treater and output module, battery (4) are used for supplying power through power module (3), treater and illuminance humidity transducer (5), wind speed changer (6), infrared temperature measurement changer (8) electric connection, treater and output module electric connection, output module pass through USB transmission line (1) and computer electric connection, and camera (9) adopt USB and computer electric connection.
2. The image-weather-temperature based pavement covering classification apparatus of claim 1, wherein: the output module adopts USB to RS485, and battery (4) adopts 24V lithium electricity, and illumination humidity transducer (5) adopts 485 illumination humidity transducer, and wind speed transducer (6) adopts 485 type wind speed transducer, and infrared temperature measurement transducer (8) adopts 485 type infrared transducer or, and camera (9) adopts 4800W pixel's CMOS camera.
3. The image-weather-temperature based pavement covering classification apparatus of claim 1, wherein: still including shell (2), shell (2) inboard is provided with power module (3), battery (4), illuminance humidity transducer (5), wind speed transducer (6), USB expansion depressed place (7), and illuminance humidity transducer (5), wind speed transducer (6) detect the end and be located shell (2) outside, and the outside of shell (2) is provided with infrared temperature measurement transducer (8), camera (9), and shell (2) material is transparent PC board.
4. An image-weather-temperature-based road surface covering classification method is characterized in that: use of an image-weather-temperature based road cover sorting device according to any of claims 1-3, comprising the steps of:
step one: collecting data;
step two: extracting image features;
step three: VIP analysis;
step four: partial least squares modeling;
step five: classification experiments based on different classification methods;
step six: the performance of the classification method is compared when the road ice and snow cover types are classified finely.
5. The image-weather-temperature based pavement covering classification method as set forth in claim 4, wherein: in the first step, the collected data types are temperature data, meteorological data and image data;
the ice and snow covered state object (cover for short) state comprises four types of new snow, granular snow, transparent ice and dry ice, and each data type comprises N samples; the temperature data comprise atmospheric temperature, body sensing temperature and ground surface temperature, the atmospheric temperature and the body sensing temperature are obtained by a weather station, and the ground surface temperature is obtained by an infrared temperature measuring transmitter (8); the meteorological data comprise illumination, humidity and wind speed, wherein the illumination, the humidity and the wind speed are obtained by an illumination humidity transducer (5), and the wind speed is obtained by a wind speed transducer (6); the image data is shot and obtained by a camera (9);
extracting image features through image data, wherein the image features comprise color features and texture features, the color features comprise 18 first-order and second-order color moment parameters, and the texture features comprise 15 gray level co-occurrence matrix parameters;
converting the acquired RGB image into a YCbCr color space and an HSV color space respectively, extracting the mean value and the second moment of three color components in each color space respectively, converting the RGB image into a gray image, and extracting texture features by using a gray co-occurrence matrix;
in the third step, carrying out VIP analysis on input parameters of different classification methods, and determining an optimal partial least square model by changing a VIP threshold;
the VIP variable screening algorithm is an algorithm based on partial least squares regression analysis, describes the interpretation capability of the independent variable to the dependent variable, and screens the independent variable according to the interpretation capability; let m independent variables x 1 ，x 2 ,…x m And p dependent variables y 1 ,y 2 …y p The VIP calculation formula is as follows:
wherein k is a self-variable number, c h R as the main component of the extracted related argument 2 (y,c h ) Is the correlation coefficient of the dependent variable and the principal component, and represents the interpretation ability of the principal component to y, w hj Representing the weight of the argument on the principal component;
by the main component c h To convey x to y interpretation capability; if c h The interpretation ability of y is strong, and x versus c h The effect of (2) is very large, and the interpretation effect of x on y can be considered to be large; in experimental analysis, determining variables selected in a classification experiment through different VIP values;
in step four, the partial least square method first refers to m independent variables x with correlation 1 ，x 2 ,…x m And p dependent variables y 1 ,y 2 …y p Performing mathematical analysis of the principal component, and then extracting a principal independent component u 1 And a dependent variable principal component v 1 While the main component u 1 And v 1 The correlation degree of the main component reaches the maximum correlation degree of all main components; then build dependent variable y 1 ,y 2 …y p And independent variable x 1 ，x 2 ,…x m Regression equations of the principal components, and performing data verification; if the precision does not meet the requirement of the experimental result, continuing to perform secondary data processing on the data to extract a second pair of main components; by analogy, if n pairs of principal components are extracted in total in the experiment; then y can be obtained 1 ,y 2 …y p Respectively with u 1 ,u 2 …u n Regression equation of u 1 ,u 2 …u n Each main component u of (a) i And x 1 ，x 2 ,…x m All are linear correlation combinations, so that a partial least squares regression equation can be obtained, and y 1 ,y 2 …y p And x 1 ，x 2 ,…x m The regression equation of (2) is a partial least squares regression equation;
let m independent variables x 1 ,x 2 …x m And p dependent variables y 1 ,y 2 …y p The formula after matrix standardization is shown as (1), wherein n represents the observation times;
wherein A, B is the independent variable group and several times of standardized observation data of the dependent variable group, and a and b are the values of the independent variable and the dependent variable;
step four comprises the following steps:
step 4.1: extracting a first pair of principal components u required by data analysis according to the mathematical linear correlation combination of the independent variable and the dependent variable 1 And v 1 Wherein u is 1 ＝α 11 x 1 +…α 1m x m ＝ρ (1)T X,v 1 ＝β 11 y 1 +…β 1p y p ＝γ (1)T Y, can be obtained:
wherein u is 1 、v 1 The first pair of principal components proposed for the two sets of variables, u 1 Is the independent variable set x= [ X ] 1 ，...x m ] T V 1 Is the dependent variable set y= [ Y ] 1 ，...，y p ] T T represents matrix transposition, X is column vector of independent variable, Y is column vector of dependent variable, alpha and beta represent linear coefficient (constant), ρ (1) 、γ (1) Represents a vector of coefficients and,is a first pair of principal components u 1 、v 1 Is a component of (2);
step 4.2: the formula for calculation A, B is:
wherein sigma (1) 、τ (1) Parameter vectors in the regression model (called model effect load quantity), A 1 、B 1 Is a residual array;
step 4.3: stopping extracting main components required by data analysis if the regression accuracy meets the requirement; if the analysis step does not meet the requirement, repeating the analysis step; when r (the representative number) major components are extracted, it is possible to obtain:
from this, a partial least squares regression equation can be obtained:
y i ＝c j1 x 1 +…+c jm x m ＝1,2,…,p (5)
wherein j=1, 2 … p, represents a dependent variableThe number of coefficient additions;
in the fifth step, N samples of each road surface covering type are divided into a training set and a verification set according to a ratio of 3:2 for the total 3N samples;
establishing a classification model of the road ice and snow covering state by adopting a partial least square algorithm, and respectively assigning values according to 1 to 4 for four covering state types in the modeling process; the VIP algorithm is adopted to input parameters to determine an optimal model, and the selected evaluation indexes are the average classification precision AP (Average Precision) of four ice and snow cover types (new snow, granular snow, transparent ice and dry):
p in the formula i For the classification precision of different ice and snow cover types, n is the number of ice and snow cover types, and the higher the average precision AP value is, the better the performance of the classification method is;
in the sixth step, the performance of different classification methods in complex covering classification is verified, and the comparison experiment flow comprises the following steps:
step 6.1: performing classification experiments of three traditional ice and snow cover types oriented to dry, transparent ice and new snow to obtain average classification accuracy AP of seven classification methods t ；
Step 6.2: increasing the complexity of the covering state by categorizing the traditional covering types; on the basis of new snow, similar types of granular snow are added, classification experiments facing four ice and snow cover types of dry ice, transparent ice, new snow and granular snow are carried out, and average classification precision AP of seven classification methods is obtained f ；
Step 6.3: with the average accuracy degradation rate DRAP (Decline Rate ofAverage Precision) as an index, the degradation of the average classification accuracy when classifying the different classification methods from three types of coverings to four types of coverings, namely the performance evaluation of the classification method when the complexity of the road ice and snow coverings is increased, is evaluated:
AP in t For average accuracy when classifying three types of covers, AP f For the average precision when classifying four types of covers, the smaller the DRAP value is, the better the performance of the classification method when classifying complex covers is.
6. The image-weather-temperature based pavement covering classification method as set forth in claim 5, wherein: in step one, each data type contains n=120 samples, and in step five, 120 samples of each road surface cover type, total 480 samples.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311143753.3A CN117312998B (en) | 2023-09-06 | 2023-09-06 | Road surface covering classification device and method based on image-weather-temperature |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311143753.3A CN117312998B (en) | 2023-09-06 | 2023-09-06 | Road surface covering classification device and method based on image-weather-temperature |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117312998A true CN117312998A (en) | 2023-12-29 |
| CN117312998B CN117312998B (en) | 2024-04-16 |
Family
ID=89285702
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311143753.3A Active CN117312998B (en) | 2023-09-06 | 2023-09-06 | Road surface covering classification device and method based on image-weather-temperature |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117312998B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108830741A (en) * | 2018-06-20 | 2018-11-16 | 汤怀志 | A kind of farm environment intelligent monitor system |
| CN110879992A (en) * | 2019-11-27 | 2020-03-13 | 内蒙古工业大学 | Grassland surface covering object classification method and system based on transfer learning |
| CN112668613A (en) * | 2020-12-07 | 2021-04-16 | 中国西安卫星测控中心 | Satellite infrared imaging effect prediction method based on weather forecast and machine learning |
| US20210241026A1 (en) * | 2020-02-04 | 2021-08-05 | Nio Usa, Inc. | Single frame 4d detection using deep fusion of camera image, imaging radar and lidar point cloud |
| CN116086547A (en) * | 2023-03-07 | 2023-05-09 | 南京气象科技创新研究院 | Contact net icing detection method based on infrared imaging and meteorological monitoring |
-
2023
- 2023-09-06 CN CN202311143753.3A patent/CN117312998B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108830741A (en) * | 2018-06-20 | 2018-11-16 | 汤怀志 | A kind of farm environment intelligent monitor system |
| CN110879992A (en) * | 2019-11-27 | 2020-03-13 | 内蒙古工业大学 | Grassland surface covering object classification method and system based on transfer learning |
| US20210241026A1 (en) * | 2020-02-04 | 2021-08-05 | Nio Usa, Inc. | Single frame 4d detection using deep fusion of camera image, imaging radar and lidar point cloud |
| CN112668613A (en) * | 2020-12-07 | 2021-04-16 | 中国西安卫星测控中心 | Satellite infrared imaging effect prediction method based on weather forecast and machine learning |
| CN116086547A (en) * | 2023-03-07 | 2023-05-09 | 南京气象科技创新研究院 | Contact net icing detection method based on infrared imaging and meteorological monitoring |
Non-Patent Citations (1)
| Title |
|---|
| MARIA ANGELA MUSCI 等: "Ice Detection on Aircraft Surface Using Machine Learning Approaches Based on Hyperspectral and Multispectral Images", DRONES, vol. 4, no. 3, 18 August 2020 (2020-08-18), pages 1 - 26 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117312998B (en) | 2024-04-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103413441B (en) | Road weather condition monitoring method | |
| WO2020199538A1 (en) | Bridge key component disease early-warning system and method based on image monitoring data | |
| CN114549914B (en) | Airplane large-area icing detection method and detection system | |
| CN109886396A (en) | A kind of transmission line galloping on-line prediction system and method | |
| CN106408005B (en) | Eutrophic lake water quality risk evaluation method based on MODIS pigment concentration estimation | |
| CN108680733A (en) | A kind of intelligent perception system and method for bituminous paving military service performance | |
| CN107202761A (en) | The portable detection equipment and detection method of a kind of quick detection fruit internal quality | |
| CN114358401A (en) | Road icing identification method fusing image and meteorological environment data | |
| CN106483143A (en) | A kind of solar energy photovoltaic panel dust stratification on-Line Monitor Device and its detection method | |
| CN107490769B (en) | A kind of fuel cell water distribution visualization on-line detecting system | |
| Lu et al. | Application research of convolution neural network in image classification of icing monitoring in power grid | |
| CN106709903A (en) | PM2.5 concentration prediction method based on image quality | |
| CN117312998B (en) | Road surface covering classification device and method based on image-weather-temperature | |
| Chen et al. | An internet-of-things-enabled system for road icing detection and prediction | |
| CN114596315A (en) | Aircraft ground detection icing method, device and system and computer equipment | |
| CN117470867B (en) | Method and device for distinguishing contamination of insulator of power transformation equipment and electronic equipment | |
| CN113917564A (en) | Multi-parameter analysis remote sensing type road surface meteorological condition detector and detection method | |
| CN204989497U (en) | Contact ice sheet detects sensor | |
| CN106991526A (en) | A kind of road surface drainage infiltration neatly risk supervision device and its implementation | |
| CN116086547B (en) | Contact net icing detection method based on infrared imaging and meteorological monitoring | |
| Xiao et al. | Asphalt pavement water film thickness detection and prediction model: A review | |
| Yang et al. | Research on the classification method of complex snow and ice cover on highway pavement based on image-meteorology-temperature fusion | |
| CN112241691B (en) | Channel ice condition intelligent identification method based on unmanned aerial vehicle inspection and image characteristics | |
| CN110187074B (en) | Cross-country skiing track snow quality prediction method | |
| CN114383341A (en) | Active ice and snow melting system for road surface and control method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |