CN115497058A - Non-contact vehicle weighing method based on multispectral imaging technology - Google Patents
Non-contact vehicle weighing method based on multispectral imaging technology Download PDFInfo
- Publication number
- CN115497058A CN115497058A CN202211076683.XA CN202211076683A CN115497058A CN 115497058 A CN115497058 A CN 115497058A CN 202211076683 A CN202211076683 A CN 202211076683A CN 115497058 A CN115497058 A CN 115497058A
- Authority
- CN
- China
- Prior art keywords
- tire
- pixel
- image
- true
- temperature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01G—WEIGHING
- G01G19/00—Weighing apparatus or methods adapted for special purposes not provided for in the preceding groups
- G01G19/02—Weighing apparatus or methods adapted for special purposes not provided for in the preceding groups for weighing wheeled or rolling bodies, e.g. vehicles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/10—Geometric CAD
- G06F30/17—Mechanical parametric or variational design
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/24—Character recognition characterised by the processing or recognition method
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2119/00—Details relating to the type or aim of the analysis or the optimisation
- G06F2119/14—Force analysis or force optimisation, e.g. static or dynamic forces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T90/00—Enabling technologies or technologies with a potential or indirect contribution to GHG emissions mitigation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Geometry (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Computer Hardware Design (AREA)
- Medical Informatics (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Optimization (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Analysis (AREA)
- Computational Mathematics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a non-contact vehicle weighing method based on a multispectral imaging technology, which comprises the following steps: processing the pixel temperature matrix data based on an OpenCV image processing algorithm, and detecting through a geometric fitting and region growing algorithm of image iteration to obtain mechanical deformation parameters of a target sample tire; detecting and obtaining the embossed characters of the tire side wall of the target sample tire, and obtaining the tire size and air pressure information of the target sample tire; and inputting the mechanical deformation parameters and the character information of the tire to be detected into the machine learning model, and calculating the load of the tire to be detected. The vehicle weighing system can realize rapid weighing of vehicles in all-weather multi-environment multi-application scenes, and is used for solving the problems of weak generalization capability, limited applicable environment and the like of the conventional vehicle weighing method based on the computer vision, and the technical problems of short service life and high cost of the conventional weighing system.
Description
Technical Field
The invention belongs to the technical field of image recognition and weighing, and particularly relates to a non-contact vehicle weighing method based on a multispectral imaging technology.
Background
The rapid development of the transportation industry puts higher requirements on the intelligent operation and maintenance of traffic. At present, infrastructures such as roads and bridges are in overload and overrun working states for a long time, and irreversible damage can be caused to the infrastructures of the roads and the bridges, so that the life safety and property safety of people are greatly threatened.
Computer vision has evolved into an effective means for solving traffic intelligence operations and maintenance. The traffic intelligence operation and maintenance is a novel traffic operation and maintenance concept integrating multiple high and new technologies into a whole, and integrating automation, intellectualization and informatization. Various intelligent traffic operation and maintenance means are the development directions of future traffic management systems, and are beneficial to long-term service of traffic infrastructure. The non-contact vehicle load identification mode based on computer vision is a novel technology for starting at once and is an important part of intelligent traffic.
Advanced non-contact measurement technology based on computer vision has been developed to a certain extent, and has continuously attracted the attention of researchers as a vehicle load detection method with low cost, high efficiency and easy operation. The traditional vehicle weighing means is mainly a contact type weighing method realized by adopting vehicle dynamic weighing equipment and an internal sensor. The weighing equipment adopted by the method is in an overload and overrun working state for a long time, and an internal sensor of the weighing equipment is easily damaged, so that the equipment is short in service life and high in maintenance cost.
At present, a non-contact vehicle weighing method based on computer vision also has certain defects, the vehicle weighing method based on optical images is weak in generalization capability, the images are easily affected by light sources and use environments, and image processing means cannot be widely applied to various applications. Moreover, the load measurement algorithm corresponding to the non-contact vehicle weighing method based on computer vision is weak in generalization capability, and the load estimation method based on fitting can only be applied to vehicle load estimation in a fixed application scene.
Therefore, how to further improve the non-contact measurement accuracy and application scene based on computer vision through a multiband invisible light source, overcome the defect that the traditional optical camera cannot measure at night, and overcome the weak generalization capability of the existing load estimation algorithm through an artificial intelligence technology becomes a problem to be solved by technicians in the field.
Disclosure of Invention
The technical problem to be solved is as follows: the invention discloses a non-contact vehicle weighing method based on a multispectral imaging technology, which can realize the rapid weighing of vehicles in all-weather multi-environment multi-application scenes and is used for solving the problems of weak generalization capability, limited applicable environment and the like of the conventional vehicle weighing method based on computational vision and the technical problems of short service life and high cost of the conventional weighing system.
The technical scheme is as follows:
a non-contact vehicle weighing method based on a multispectral imaging technology comprises the following steps:
s1, obtaining a side thermal imaging image of a target sample tire in a normal working state; extracting pixel temperature matrix data from the side thermographic image to obtain surface temperature information of the target sample tire; processing the pixel temperature matrix data based on an OpenCV image processing algorithm, and detecting through a geometric fitting and region growing algorithm of image iteration to obtain mechanical deformation parameters of a target sample tire;
s2, obtaining an optical image of the tire sidewall of the target sample; detecting embossed characters of the target sample tire sidewall from the optical image of the target sample tire sidewall; obtaining tire size and air pressure information of a target sample tire by impressing characters;
s3, taking the obtained mechanical deformation parameters of the target sample tire and the corresponding tire size and air pressure information as training samples of a machine learning model, and training to obtain the machine learning model for predicting the tire load;
and S4, inputting the mechanical deformation parameters and the character information of the tire to be detected into the machine learning model, and calculating the load of the tire to be detected.
Further, in step S1, the process of obtaining the side thermal imaging image of the target sample tire in the normal working state includes the following steps:
centering the thermal imaging acquisition assembly on the center of the tire hub by utilizing infrared laser calibration; capturing lateral thermal imaging information of the tire in a normal working state by adopting a thermal imaging acquisition component; the side thermographic information captured by the thermographic acquisition assembly is saved in the format of a CSV.
Further, in step S1, the mechanical deformation parameters include: tire maximum pixel radius R, tire maximum pixel area S 1 The wheel hub pixel radius r, the contact pixel length of the tire and the ground 1, the pixel distance h from the center of the tire to the ground, and the equivalent pixel area S after the tire is deformed 2 The image pixel area difference delta S before and after the tire deformation and the tire-ground contact dividing line pixel length L.
Further, in step S1, the process of processing the pixel temperature matrix data based on the OpenCV image processing algorithm and obtaining the mechanical deformation parameter of the target sample tire through the geometric fitting of image iteration and the region growing algorithm detection includes the following steps:
s11, generating a temperature image linearly related to the temperature according to the pixel temperature matrix data; calculating a pixel gradient amplitude of the temperature image by using a sobel edge detection operator, and performing image segmentation around a pixel point of which the temperature difference in the temperature image is greater than a preset temperature difference threshold value;
s12, reserving the gradient amplitude of the first 5 percent of the segmented image, taking out the point with the maximum gradient amplitude, and marking the color; sequentially selecting pixel points marked by colors from the lower direction of the image, and taking the selected pixel points as seed points for fitting the outer contour of the tire for the first time, wherein the selected seed points are all the pixel points on the interface between the tire and the air;
s13, adopting the seed points selected in the step S12, carrying out region growth on the seed points by using a region growth algorithm to find points adjacent to the seed points as new seed points, and carrying out second fitting on the outer contour of the tire;
s14, reserving the seed points selected in the step S12 and the step S13, taking the outer contour of the tire subjected to the second fitting as a reference, finding the pixel point with the maximum gradient amplitude value in the upper half part of the tire from top to bottom, and carrying out the outer contour fitting of the tire for the third time;
s15, with the tire outer contour fitted in the step S14 as a reference, searching a pixel point with the maximum gradient amplitude value from the center of the tire, taking the pixel point as a seed point for fitting the wheel hub outer contour, and fitting to obtain the wheel hub outer contour;
s16, repeating iteration to fit to obtain a tire outer contour and a hub outer contour which accord with a preset error standard;
s17, finding a pixel gradient amplitude point at the junction of the tire and the ground within an included angle range of 45 degrees at the left lower part and 45 degrees at the right lower part of the circle center of the tire, drawing a tire and ground parting line by taking the found pixel gradient amplitude point as a reference, and obtaining an interface between the tire and the ground after the tire is deformed;
and S18, performing end point processing on image pixels of the interface between the tire and the ground from bottom to top, calculating Y coordinate difference values of two adjacent pixel points, and judging that any pixel point is an end point of the contact between the tire and the ground when the Y coordinate difference value of the pixel point and the adjacent pixel point is larger than a preset coordinate difference value threshold value to obtain the real contact pixel length between the tire and the ground.
Further, the obtained mechanical deformation parameters of the tire are corrected by adopting a scale factor alpha:
wherein Rim is the hub radius; r is the radius of the hub pixel obtained by fitting;
the corrected mechanical deformation parameters are as follows:
r true ＝α×r
R true ＝α×R
S 1true ＝α 2 ×S 1
l true ＝α×k
h true ＝α×h
S 2true ＝α 2 ×S 2
ΔS true ＝α 2 ×ΔS
L true ＝α×L
in the formula, r true 、R true 、s 1true 、l true 、h true 、s 2true 、ΔS true And L true Respectively obtaining corrected wheel hub pixel radius, tire maximum pixel area, tire and ground contact pixel length, tire circle center to ground pixel distance, tire deformed equivalent pixel area, tire before and after deformation image pixel area difference and tire and ground contact parting line pixel length; r, S 1 、l、h、S 2 And delta S and L are respectively the maximum pixel radius of the tire, the maximum pixel area of the tire, the contact pixel length of the tire and the ground, the pixel distance from the center of the tire to the ground, the equivalent pixel area after the tire is deformed, the image pixel area difference before and after the tire is deformed and the contact dividing line pixel length of the tire and the ground which are obtained by fitting.
Further, in step S2, performing OCR character recognition on the optical image of the tire sidewall of the target sample based on a deep learning end-to-end OCR character recognition algorithm, and acquiring size information and air pressure information of the tire according to a recognition result; the dimension information comprises tire section height H, tire section width b, hub radius Rim and air pressure information atm; the tire true air pressure is 1.1 to 1.2 times the maximum air pressure indicated by the tire identifier.
Further, in step S3, each training sample includes 12 mechanical features and 1 label; the mechanical characteristics are respectively as follows: maximum radius R of tire true Maximum area S of tire 1true Radius r of the hub true Length of contact between tire and ground true Distance h from the center of the tire to the ground true Equivalent area S after tire deformation 2true Area difference Δ S before and after tire deformation true The length L of the contact dividing line of the tire and the ground true Tire section height H, tire section width b, and tire air pressure atm: the label is as follows: a tire-ground contact force F;
the machine learning model is an integrated decision tree model reflecting a mapping relation between 12 mechanical features and 1 mechanical response.
Further, the non-contact vehicle weighing method further comprises the following steps:
s5, temperature correction is carried out on the tire real load predicted by the machine learning model through the regression coefficient beta:
in the formula (I), the compound is shown in the specification,
the tire true load after temperature correction, F is the tire true load predicted by the machine learning model,
is the tire surface average temperature; the value range of the regression coefficient beta is 0.9 to 1.5, when the surface temperature of the tire reaches 90% of the cold tire pressure temperature corresponding to the standard tire pressure, the value is 0.9, a linear interpolation value is adopted in the middle, when the surface temperature of the tire reaches 150% of the cold tire pressure temperature corresponding to the standard tire pressure, the value is 1.5, and the value is obtained by the linear interpolation in the middle;
s6, predicting the load W of the whole vehicle according to the number of the axles:
further, after extracting the contour image after the tire segmentation, mapping pixel points of the contour image back to pixel temperature matrix data, and calculating the average temperature of the tire surface by using the temperature data in the tire contour
Wherein n represents the number of temperature pixels of the tire surface captured by thermal imaging, T i The temperature of the ith temperature data of the tire surface.
The invention also discloses an automatic load prediction system, which comprises a thermal imaging acquisition component, a data processing device, an optical image acquisition component, an OCR recognition device, a machine learning prediction model and a model updating device;
the data processing device comprises a storage unit, a thermal imaging data processing program and an OCR character recognition program;
the storage unit is used for storing the thermal imaging image of the tire to be detected, which is shot by the thermal imaging acquisition assembly, and the optical image of the sidewall of the tire to be detected, which is shot by the optical image acquisition assembly; the OCR character recognition program is used for recognizing embossed characters contained in the optical image of the tire sidewall; the data processing device acquires the tire size and air pressure information of the tire to be detected through the embossed characters and transmits the tire size and air pressure information to the machine learning prediction model; the thermal imaging data processing program processes the thermal imaging image of the tire to be detected according to the method, calculates the mechanical deformation parameter of the tire to be detected, and transmits the mechanical deformation parameter to the machine learning prediction model;
the machine learning prediction model processes the imported mechanical deformation parameters, tire sizes and air pressure information of the tire to be detected, and calculates to obtain the load of the tire to be detected;
and the model updating device is used for importing new sample data into the machine learning model and updating the machine learning model.
Has the advantages that:
first, compared with the existing optical camera detection method, the non-contact vehicle weighing method based on the multispectral imaging technology provided by the invention can detect the deformation information of the tire with extremely high precision according to different radiances and surface temperatures of objects, is not limited by a light source and an application scene, and overcomes the defect that the optical camera can only work in specific light source environments such as daytime and the like and cannot work at night.
Secondly, the non-contact vehicle weighing method based on the multispectral imaging technology considers 12 mechanical characteristics based on a machine learning load prediction algorithm, comprehensively considers variables related to tire loads, can continuously improve the prediction precision of the model through real-time updating of the model, improves the generalization performance of the model, and is suitable for various types of tires by means of an expansion data set.
Thirdly, the non-contact vehicle weighing method based on the multispectral imaging technology only needs to install the cameras outside the lane to shoot the vehicle outline, the equipment is convenient to erect, the number of the needed cameras is small, the cameras can be replaced according to the precision requirement of the method, and the maximization of the required benefits is reasonably realized. The newly-erected camera can quickly finish the vehicle weight only by simple distortion correction, and has strong mobility and reproduction capability.
Drawings
FIG. 1 is a flowchart of a non-contact vehicle weighing method based on multispectral imaging technology according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of an outdoor field deployment of an automated load forecasting system of the present invention;
FIG. 3 is a schematic view of a thermal image of the outer contour of the tire captured by the thermal image acquisition assembly;
FIG. 4 is a schematic view of a tire deformation profile detected from a thermal imaging profile;
FIG. 5 is a schematic view of the average temperature of the outer surface of the tire captured by the thermal image acquisition assembly;
FIG. 6 is a schematic thermal imaging view at night;
FIG. 7 is a diagram illustrating the effects of character recognition on a tire;
FIG. 8 is a graph of the predicted effect of the automated load prediction system of the present invention.
Detailed Description
The following examples are presented to enable one of ordinary skill in the art to more fully understand the present invention and are not intended to limit the invention in any way.
Referring to fig. 1, the present embodiment discloses a non-contact vehicle weighing method based on multispectral imaging technology, which includes the following steps:
s1, obtaining a side thermal imaging image of a target sample tire in a normal working state; extracting pixel temperature matrix data from the side thermographic image to obtain surface temperature information of the target sample tire; and processing the pixel temperature matrix data based on an OpenCV image processing algorithm, and detecting through a geometric fitting and region growing algorithm of image iteration to obtain mechanical deformation parameters of the target sample tire.
S2, obtaining an optical image of the tire sidewall of the target sample; detecting embossed characters of the target sample tire sidewall from the optical image of the target sample tire sidewall; tire size and air pressure information of a target sample tire is obtained by embossing characters.
And S3, taking the obtained mechanical deformation parameters of the target sample tire and the corresponding tire size and air pressure information as training samples of the machine learning model, and training to obtain the machine learning model for predicting the tire load.
And S4, inputting the mechanical deformation parameters and the character information of the tire to be detected into the machine learning model, and calculating the load of the tire to be detected.
The embodiment also discloses an automatic load prediction system based on infrared thermal imaging and machine vision, which comprises a thermal imaging acquisition component, a data processing device, an optical image acquisition component, an OCR recognition device, a machine learning prediction model and a model updating device. The thermal imaging acquisition assembly is used for acquiring thermal imaging images of all tires of the target vehicle to obtain CSV temperature data; the optical image acquisition assembly is used for acquiring an optical image of the sidewall of the tire. The data processing device comprises a storage unit, a thermal imaging data processing program and an OCR character recognition program. The storage unit is used for storing tire deformation parameters and OCR character recognition information obtained by detecting the thermal imaging image; the machine learning prediction model is used for receiving 12 mechanical characteristics sent by the data processing device and predicting and correcting the tire load; and the model updating component is used for retraining the machine learning prediction model after the system predicts the load each time, replacing the original model, bringing newly measured data into the database and enhancing the generalization capability of the model.
In this embodiment, the thermal imaging acquisition assembly may be a high resolution high frame rate thermal imaging device with a resolution greater than or equal to 640 × 480, and in this embodiment, a K26HE25 high speed high frame rate infrared thermal imaging acquisition assembly is adopted. The optical image acquisition component adopts a Nikon D5600 single-lens reflex camera (hereinafter referred to as a camera), and the rest components adopt a notebook computer, as shown in FIG. 2, the optical image acquisition component is an outdoor deployment schematic diagram of an automatic load prediction system.
In this embodiment, the thermal imaging acquisition component performs thermal imaging capture on the tire of the target vehicle to obtain an original CSV temperature data file, extracts temperature information corresponding to each pixel point of the CSV temperature data file, and redraws a temperature image according to a linear relationship, as shown in fig. 3. And then, edge detection is carried out on the temperature image by adopting a sobel operator, the temperature gradient amplitude in the image is calculated, the point with the maximum gradient amplitude is found, and pixel points 5% before the gradient amplitude are reserved for color marking.
According to the priori knowledge, the point with the maximum gradient amplitude at the contact surface of the lower half part of the tire and the air can be closest to the edge contour of the tire, and edge detection points at the interface of the tire and the air are sequentially selected from the lower part of the image to the upper part as seed points for fitting the outer contour circle of the tire. By first selectionAnd fitting the outer contour of the tire by the seed points, performing region growing operation on the seed points selected for the first time, selecting the seed points for the second time, and continuously fitting the outer contour of the tire. And selecting a seed point for the third time from the upper half part of the tire from top to bottom, and further fitting the outer contour of the tire. And sequentially searching temperature gradient amplitude points towards the circle center by taking the tire outline as a boundary, and fitting the hub outline by taking the points as edge pixel points of the hub. Finding out a pixel gradient amplitude point at the boundary between the tire and the ground within 45 degrees from the left lower part and the right lower part of the circle center of the tire, taking the point as a reference tire and ground parting line to obtain an interface between the tire and the ground after the tire is deformed, carrying out end point processing on image pixels of the interface between the tire and the ground from bottom to top, calculating a Y coordinate difference value of two adjacent pixel points, judging the point as an end point of the contact between the tire and the ground when the difference value is greater than a certain threshold value, obtaining the real contact pixel length between the tire and the ground, finishing the outer contour edge parting of the tire and a wheel hub based on thermal imaging data, and obtaining the following parameters, namely the maximum pixel radius R of the tire, and the maximum pixel area S of the tire 1 Wheel hub pixel radius r, tire-ground contact pixel length l, tire center-to-ground pixel distance h, and equivalent pixel area S after tire deformation 2 The difference Δ S between the image pixel areas before and after the tire deformation, and the tire-ground contact dividing line pixel length L are shown in fig. 4.
Calculating the average temperature of the outer surface of the tire from the thermal information captured by the thermal imaging,
as shown in fig. 5.
The optical image of the tire is acquired by using the optical acquisition assembly, and then an OCR recognition algorithm is called, wherein the OCR recognition algorithm adopts a character recognition technology based on deep learning, a PSENET character positioning network and a CRNN character recognition network are trained by a transfer learning method to form a two-stage character recognition method, the two-stage character recognition method is identifier information of the sidewall of the tire, and tire size information (tire section height H, tire section width b and hub radius Rim) and air pressure information atm are obtained, as shown in FIG. 7.
According to the size information of the wheel hub, carrying out size correction on various deformation parameters measured in thermal imaging to obtain the maximum radius R of the tire true Maximum area of tire S 1true Radius of hub r true Length of contact between tyre and ground true Distance h from the center of the tire to the ground true Equivalent area S after tire deformation 2true Area difference Δ S before and after tire deformation true Length L of tire-ground contact dividing line true 。
According to a data set acquired by an indoor test (acquiring a large number of images of tires with different sizes, different tire pressures, different loads and different temperatures, identifying the height of a tire section, the width of the section and the tire pressure of the tire obtained by tire manufacturers and models through an OCR (optical character recognition) technology to form a certain amount of sample data), the XGboost model is trained to obtain a load prediction model.
12 mechanical characteristics measured by the thermal imaging module and the character recognition module are input into the XGboost load prediction model to obtain the predicted load of a single tire, and the error of the method is verified to be less than 5% outdoors, so that the prediction effect is good, and is shown in figure 8.
According to the characteristic that the tire pressure is subjected to temperature change, temperature correction is carried out on the tire load to obtain the corrected load of a single tire
Further calculating the load of the whole vehicle according to the number of the axles to obtain the load of the whole vehicle
If the embodiment has the condition of measuring the real load of the tire, the tire load measured by a third-party device is combined with the 12 mechanical characteristics provided by the invention to construct a data set for updating the model, so that the automatic updating and self-learning mechanism of the model is realized, and the universal capability of the load prediction model is improved.
The multispectral imaging technology-based non-contact vehicle weighing method provided by the embodiment can accurately predict the tire-ground contact force of the vehicle under the non-contact condition. The method is characterized in that the deformation outer contour of the tire represented by multispectral thermodynamic information in a normal working state of the tire is captured through thermal imaging equipment, and an image segmentation algorithm is designed aiming at the multispectral imaging captured by thermal imaging acquisition component equipment, so that the deformation information of the tire in normal working is acquired; meanwhile, detecting the identifier of the tire side wall by an optical camera to obtain the basic size information of the tire; and inputting the tire deformation information and the tire identifier information into the XGboost machine learning prediction model to further realize the prediction of the tire load. Compared with the prior art, the measuring module of the embodiment can more accurately measure the deformation information and the thermodynamic information of the tire than an optical camera; compared with the existing calculation model, the load prediction module of the embodiment is more accurate, the error of the whole vehicle weighing is less than 5%, various mechanical characteristics are more comprehensively considered, and the generalization capability of the model can be continuously enhanced through the model, so that the application range of the model is expanded, and the robustness is stronger. In addition, the embodiment can realize the real-time weighing of all-weather vehicles at night or in the daytime, and has wider applicability. Fig. 6 is a schematic diagram of thermal imaging during night operation.
The above description is only of the preferred embodiments of the present invention, and it should be noted that: it will be apparent to those skilled in the art that various modifications and adaptations can be made without departing from the principles of the invention and these are intended to be within the scope of the invention.
Claims (10)
1. A non-contact vehicle weighing method based on a multispectral imaging technology is characterized by comprising the following steps:
s1, obtaining a side thermal imaging image of a target sample tire in a normal working state; extracting pixel temperature matrix data from the side thermographic image to obtain surface temperature information of the target sample tire; processing the pixel temperature matrix data based on an OpenCV image processing algorithm, and detecting through a geometric fitting and region growing algorithm of image iteration to obtain mechanical deformation parameters of a target sample tire;
s2, obtaining an optical image of the tire sidewall of the target sample; detecting embossed characters of the target sample tire sidewall from the optical image of the target sample tire sidewall; obtaining tire size and air pressure information of a target sample tire by impressing characters;
s3, taking the obtained mechanical deformation parameters of the target sample tire and the corresponding tire size and air pressure information as training samples of a machine learning model, and training to obtain the machine learning model for predicting the tire load;
and S4, inputting the mechanical deformation parameters and the character information of the tire to be detected into the machine learning model, and calculating the load of the tire to be detected.
2. The method for non-contact vehicle weighing based on multispectral imaging technology as claimed in claim 1, wherein the step S1 of obtaining the thermal imaging image of the side surface of the target sample tire under normal working condition comprises the following steps:
centering the thermal imaging acquisition assembly on the center of the tire hub by utilizing infrared laser calibration; capturing lateral thermal imaging information of the tire in a normal working state by adopting a thermal imaging acquisition assembly; the side thermographic information captured by the thermographic acquisition assembly is saved in the format of a CSV.
3. The method for non-contact vehicle weighing based on multispectral imaging technology as claimed in claim 1, wherein in step S1, the mechanical deformation parameters comprise: tire maximum pixel radius R, tire maximum pixel area S 1 Wheel hub pixel radius r, tire-ground contact pixel length l, tire center-to-ground pixel distance h, and equivalent pixel area S after tire deformation 2 The image pixel area difference delta S before and after the tire deformation and the tire-ground contact dividing line pixel length L.
4. The non-contact vehicle weighing method based on the multispectral imaging technology as claimed in claim 1, wherein in step S1, the OpenCV image processing algorithm is used for processing pixel temperature matrix data, and the process of detecting and obtaining the mechanical deformation parameters of the target sample tire through the geometry fitting of image iteration and the region growing algorithm comprises the following steps:
s11, generating a temperature image linearly related to the temperature according to the pixel temperature matrix data; calculating the pixel gradient amplitude of the temperature image by using a sobel edge detection operator, and performing image segmentation around pixel points with temperature difference larger than a preset temperature difference threshold value in the temperature image;
s12, reserving the gradient amplitude of the first 5 percent of the segmented image, taking out the point with the maximum gradient amplitude, and marking the color; sequentially selecting pixel points marked by colors from the lower direction of the image, and taking the selected pixel points as seed points for fitting the outer contour of the tire for the first time, wherein the selected seed points are all the pixel points on the interface between the tire and the air;
s13, adopting the seed points selected in the step S12, carrying out region growth on the seed points by using a region growth algorithm to find points adjacent to the seed points as new seed points, and carrying out second fitting on the outer contour of the tire;
s14, reserving the seed points selected in the step S12 and the step S13, taking the outer contour of the tire subjected to the second fitting as a reference, finding the pixel point with the maximum gradient amplitude value in the upper half part of the tire from top to bottom, and carrying out the outer contour fitting of the tire for the third time;
s15, with the tire outer contour fitted in the step S14 as a reference, searching a pixel point with the maximum gradient amplitude value from the center of the tire, taking the pixel point as a seed point for fitting the wheel hub outer contour, and fitting to obtain the wheel hub outer contour;
s16, repeating iteration to fit to obtain a tire outer contour and a hub outer contour which accord with a preset error standard;
s17, finding a pixel gradient amplitude point at the junction of the tire and the ground within the included angle range of 45 degrees at the left lower part and 45 degrees at the right lower part of the center of the circle of the tire, and drawing a tire and ground parting line by taking the found pixel gradient amplitude point as a reference to obtain an interface of the tire and the ground after the tire is deformed;
and S18, performing end point processing on image pixels of the interface between the tire and the ground from bottom to top, calculating Y coordinate difference values of two adjacent pixel points, and judging that any pixel point is an end point of the contact between the tire and the ground when the Y coordinate difference value of the pixel point and the adjacent pixel point is larger than a preset coordinate difference value threshold value to obtain the real contact pixel length between the tire and the ground.
5. The method for non-contact vehicle weighing based on multispectral imaging technology according to claim 1 or 4, wherein the obtained mechanical deformation parameters of the tire are corrected by a scaling factor α:
where Rim is the hub radius; r is the radius of the hub pixel obtained by fitting;
the corrected mechanical deformation parameters are as follows:
r true ＝α×r
R true ＝α×R
s 1true ＝α 2 ×s 1
l true ＝α×k
h true ＝α×h
s 2true ＝α 2 ×S 2
ΔS true ＝α 2 ×ΔS
L true ＝α×L
in the formula, r true 、R true 、s 1true 、l true 、h true 、s 2true 、ΔS true And L true Respectively obtaining corrected wheel hub pixel radius, tire maximum pixel area, tire and ground contact pixel length, tire circle center to ground pixel distance, tire deformed equivalent pixel area, tire before and after deformation image pixel area difference and tire and ground contact parting line pixel length; r, R,s 1 、l、h、s 2 And delta S and L are respectively the maximum pixel radius of the tire, the maximum pixel area of the tire, the contact pixel length of the tire and the ground, the pixel distance from the center of the tire to the ground, the equivalent pixel area after the tire is deformed, the image pixel area difference before and after the tire is deformed and the contact dividing line pixel length of the tire and the ground which are obtained by fitting.
6. The non-contact vehicle weighing method based on the multispectral imaging technology as claimed in claim 1, wherein in step S2, OCR character recognition is performed on the optical image of the tire sidewall of the target sample based on a deep learning end-to-end OCR character recognition algorithm, and size information and air pressure information of the tire are obtained according to the recognition result; the dimension information comprises tire section height H, tire section width b, hub radius Rim and air pressure information atm; the tire true air pressure is 1.1 to 1.2 times the maximum atmospheric pressure indicated by the tire identifier.
7. The method for non-contact vehicle weighing based on multispectral imaging technology as claimed in claim 1, wherein in step S3, each training sample comprises 12 mechanical features and 1 tag; the mechanical characteristics are respectively as follows: maximum radius R of tire true Maximum area s of tire 1true Radius r of the hub true Length of contact between tire and ground true Distance h from center of circle of tire to ground true Equivalent area S after tire deformation 2true Area difference Δ S before and after tire deformation true The length L of the contact dividing line of the tire and the ground true Tire section height H, tire section width b and tire pressure atm; the label is as follows: a tire-ground contact force F;
the machine learning model is an integrated decision tree model reflecting the mapping relation between 12 mechanical features and 1 mechanical response.
8. The method of claim 1 further comprising the steps of:
s5, temperature correction is carried out on the tire real load predicted by the machine learning model through the regression coefficient beta:
in the formula (I), the compound is shown in the specification,
f is the tire real load predicted by the machine learning model, and F is the tire real load after temperature correction,
is the tire surface average temperature; the value range of the regression coefficient beta is 0.9 to 1.5, when the surface temperature of the tire reaches 90% of the cold tire pressure temperature corresponding to the standard tire pressure, the value is 0.9, a linear interpolation value is adopted in the middle, when the surface temperature of the tire reaches 150% of the cold tire pressure temperature corresponding to the standard tire pressure, the value is 1.5, and the value is obtained by the linear interpolation in the middle;
s6, predicting the load W of the whole vehicle according to the number of the axles:
9. the method according to claim 1 or 8, wherein the segmented tire contour image is extracted, the pixels of the contour image are mapped back to the pixel temperature matrix data, and the average tire surface temperature is calculated from the temperature data within the tire contour
Where n represents the number of temperature pixels captured by thermal imaging of the tire surface, T i The temperature of the ith temperature data of the tire surface.
10. An automatic load prediction system is characterized by comprising a thermal imaging acquisition component, a data processing device, an optical image acquisition component, an OCR recognition device, a machine learning prediction model and a model updating device;
the data processing device comprises a storage unit, a thermal imaging data processing program and an OCR character recognition program;
the storage unit is used for storing the thermal imaging image of the tire to be detected shot by the thermal imaging acquisition assembly and the optical image of the side wall of the tire to be detected shot by the optical image acquisition assembly; the OCR character recognition program is used for recognizing embossed characters contained in the optical image of the tire sidewall; the data processing device acquires the tire size and the air pressure information of the tire to be detected through the embossed characters and transmits the tire size and the air pressure information to the machine learning prediction model; the thermal imaging data processing program processes the thermal imaging image of the tire to be detected according to the method of any one of claims 1 to 9, calculates the mechanical deformation parameters of the tire to be detected, and transmits the parameters to the machine learning prediction model;
the machine learning prediction model processes the imported mechanical deformation parameters, tire sizes and air pressure information of the tire to be detected, and calculates to obtain the load of the tire to be detected;
and the model updating device is used for importing new sample data into the machine learning model and updating the machine learning model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211076683.XA CN115497058B (en) | 2022-09-02 | 2022-09-02 | Non-contact vehicle weighing method based on multispectral imaging technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211076683.XA CN115497058B (en) | 2022-09-02 | 2022-09-02 | Non-contact vehicle weighing method based on multispectral imaging technology |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115497058A true CN115497058A (en) | 2022-12-20 |
| CN115497058B CN115497058B (en) | 2023-09-01 |
Family
ID=84468846
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211076683.XA Active CN115497058B (en) | 2022-09-02 | 2022-09-02 | Non-contact vehicle weighing method based on multispectral imaging technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115497058B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190120723A1 (en) * | 2017-10-24 | 2019-04-25 | Konica Minolta, Inc. | Tire deformation calculator, overload detecting system, and method for calculating tire deformation |
| CN110659649A (en) * | 2019-09-29 | 2020-01-07 | 哈尔滨工业大学 | Image processing and character recognition algorithm based on near infrared light imaging |
| CN111735524A (en) * | 2020-08-27 | 2020-10-02 | 湖南大学 | Tire load obtaining method based on image recognition, vehicle weighing method and system |
| CN114092683A (en) * | 2021-11-09 | 2022-02-25 | 湖南中登科技有限公司 | Tire deformation identification method and device based on visual feedback and depth network |
-
2022
- 2022-09-02 CN CN202211076683.XA patent/CN115497058B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190120723A1 (en) * | 2017-10-24 | 2019-04-25 | Konica Minolta, Inc. | Tire deformation calculator, overload detecting system, and method for calculating tire deformation |
| CN110659649A (en) * | 2019-09-29 | 2020-01-07 | 哈尔滨工业大学 | Image processing and character recognition algorithm based on near infrared light imaging |
| CN111735524A (en) * | 2020-08-27 | 2020-10-02 | 湖南大学 | Tire load obtaining method based on image recognition, vehicle weighing method and system |
| CN114092683A (en) * | 2021-11-09 | 2022-02-25 | 湖南中登科技有限公司 | Tire deformation identification method and device based on visual feedback and depth network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115497058B (en) | 2023-09-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112380952B (en) | Power equipment infrared image real-time detection and identification method based on artificial intelligence | |
| CN109685858B (en) | Monocular camera online calibration method | |
| CN109099929B (en) | Intelligent vehicle positioning device and method based on scene fingerprints | |
| CN110889398B (en) | Multi-modal image visibility detection method based on similarity network | |
| CN112434586B (en) | Multi-complex scene target detection method based on domain self-adaptive learning | |
| CN113436157B (en) | Vehicle-mounted image identification method for pantograph fault | |
| CN108875803B (en) | Hazardous chemical substance transport vehicle detection and identification method based on video image | |
| CN113592828B (en) | Nondestructive testing method and system based on industrial endoscope | |
| CN112215125A (en) | Water level identification method based on YOLOv3 | |
| CN103729631A (en) | Vision-based connector surface feature automatically-identifying method | |
| CN112598066A (en) | Lightweight road pavement detection method and system based on machine vision | |
| CN111832760B (en) | Automatic inspection method for well lid based on visual algorithm | |
| CN112258490A (en) | Low-emissivity coating intelligent damage detection method based on optical and infrared image fusion | |
| CN111461036A (en) | Real-time pedestrian detection method using background modeling enhanced data | |
| CN111582126A (en) | Pedestrian re-identification method based on multi-scale pedestrian contour segmentation fusion | |
| KR101874968B1 (en) | Visibility measuring system base on image information and method for using the same | |
| CN115937518A (en) | Pavement disease identification method and system based on multi-source image fusion | |
| CN115497058B (en) | Non-contact vehicle weighing method based on multispectral imaging technology | |
| CN113326846A (en) | Rapid bridge apparent disease detection method based on machine vision | |
| CN115493679A (en) | Toll station vehicle dynamic weighing system based on multi-view-field thermal imaging technology | |
| CN111476314A (en) | Fuzzy video detection method integrating optical flow algorithm and deep learning | |
| CN114581440B (en) | Method for rapidly positioning leakage point based on image recognition | |
| CN116310902A (en) | Unmanned aerial vehicle target detection method and system based on lightweight neural network | |
| CN113313008B (en) | Target and identification tracking method based on YOLOv3 network and mean shift | |
| CN108734158B (en) | Real-time train number identification method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |