CN117953348A - Training method, device, equipment and medium of road scene prediction model - Google Patents
Training method, device, equipment and medium of road scene prediction model Download PDFInfo
- Publication number
- CN117953348A CN117953348A CN202410166174.9A CN202410166174A CN117953348A CN 117953348 A CN117953348 A CN 117953348A CN 202410166174 A CN202410166174 A CN 202410166174A CN 117953348 A CN117953348 A CN 117953348A
- Authority
- CN
- China
- Prior art keywords
- road
- type
- scene
- loss
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012549 training Methods 0.000 title claims abstract description 71
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000005286 illumination Methods 0.000 claims description 94
- 238000004590 computer program Methods 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000006870 function Effects 0.000 description 13
- 102100034112 Alkyldihydroxyacetonephosphate synthase, peroxisomal Human genes 0.000 description 10
- 101000799143 Homo sapiens Alkyldihydroxyacetonephosphate synthase, peroxisomal Proteins 0.000 description 10
- 238000000848 angular dependent Auger electron spectroscopy Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000007613 environmental effect Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000010355 oscillation Effects 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008033 biological extinction Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004313 glare Effects 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Landscapes
- Traffic Control Systems (AREA)
Abstract
The embodiment of the invention discloses a training method, a training device, training equipment and training media for a road scene prediction model. The method comprises the following steps: acquiring a sample road image and a truth road scene type marked by the sample road image; inputting the sample road image into a road scene prediction model to obtain a predicted road scene type of the sample road image; respectively determining a scene basic loss value and uncertainty loss data of a road scene prediction model according to the true road scene type and the predicted road scene type of the sample road image; determining a model loss value of the road scene prediction model according to the scene basic loss value and the uncertainty loss data; and training the road scene prediction model by adopting the model loss value. By the aid of the scheme, accuracy of training of the road scene prediction model is improved.
Description
Technical Field
The embodiment of the invention relates to the technical field of image recognition in the intelligent driving field, in particular to a training method, device, equipment and medium of a road scene prediction model.
Background
In modern traffic systems, automatic driving assistance systems (ADVANCED DRIVER ASSISTANCE SYSTEMS, abbreviated as ADAS) play an increasingly important role. The ADAS system provides functions such as collision early warning, lane keeping, adaptive cruise, etc. through various sensors and algorithms to improve driving safety and comfort. However, in practical applications, the ADAS system faces various challenges, one of which is how to accurately identify environmental factors under different illumination intensities, weather types and road conditions, and adjust collision warning strategies accordingly.
First, the intensity of illumination has a significant impact on the performance of the ADAS system. In strong light environments, such as direct sunlight in noon, a camera of a vehicle may be affected by glare, resulting in degradation of image quality, thereby affecting the accuracy of collision early warning. In low light environments, such as dusk or dawn, the light-sensing capability of the camera may be affected, as well as the performance of the system. Therefore, accurately identifying the illumination intensity and adjusting the early warning strategy according to the illumination condition is a key for improving the performance of the ADAS system.
Second, weather type also has an important impact on the performance of ADAS systems. In severe weather conditions such as rainy, snowy or foggy days, visibility may be reduced and road conditions may become complicated, all of which may increase the risk of collisions. Therefore, the weather type is accurately identified, and the early warning strategy is adjusted according to the weather condition, so that the method is also an important link for improving the performance of the ADAS system.
Again, the road type has an important impact on the performance of the ADAS system. On highways, the speed of the vehicle is typically high, and the consequences of a collision may be more severe. On non-highways, however, the risk of collisions may also increase due to the higher frequency of occurrence of pedestrians, bicycles, and other non-motor vehicles, although the speed of the vehicle is slower. Therefore, the road type is accurately identified, and the early warning strategy is adjusted according to the road condition, which is also an important link for improving the ADAS system performance.
To sum up, in order to improve the safety and comfort of automatic driving, how to improve the accuracy of the training of the road scene prediction model for road scene type recognition in the automatic driving process is important.
Disclosure of Invention
The invention provides a training method, device, equipment and medium for a road scene prediction model, so as to improve the accuracy of training the road scene prediction model.
According to an aspect of the present invention, there is provided a training method of a road scene prediction model, including:
acquiring a sample road image and a truth value road scene type marked by the sample road image; the truth road scene type comprises a truth illumination type, a truth weather type and a truth road type;
Inputting the sample road image into a road scene prediction model to obtain a predicted road scene type of the sample road image; the predicted road scene type comprises a predicted illumination type, a predicted weather type and a predicted road type;
Respectively determining a scene basic loss value and uncertainty loss data of the road scene prediction model according to the true road scene type and the predicted road scene type of the sample road image; the uncertainty loss data is used for adjusting the magnitude of the scene basic loss value in a model loss value and representing the stability of the sample road image for prediction on the road scene type;
Determining a model loss value of the road scene prediction model according to the scene basic loss value and the uncertainty loss data;
and training the road scene prediction model by adopting the model loss value.
According to another aspect of the present invention, there is provided a training apparatus of a road scene prediction model, including:
The image acquisition module is used for acquiring a sample road image and a truth value road scene type marked by the sample road image; the truth road scene type comprises a truth illumination type, a truth weather type and a truth road type;
The model prediction module is used for inputting the sample road image into a road scene prediction model to obtain a predicted road scene type of the sample road image; the predicted road scene type comprises a predicted illumination type, a predicted weather type and a predicted road type;
The data determining module is used for respectively determining a scene basic loss value and uncertainty loss data of the road scene prediction model according to the true road scene type and the predicted road scene type of the sample road image; the uncertainty loss data is used for adjusting the magnitude of the scene basic loss value in a model loss value and representing the stability of the sample road image for prediction on the road scene type;
the model loss determining module is used for determining a model loss value of the road scene prediction model according to the scene basic loss value and the uncertainty loss data;
and the model training module is used for training the road scene prediction model by adopting the model loss value.
According to another aspect of the present invention, there is provided an electronic apparatus including:
one or more processors;
a memory for storing one or more programs;
when the one or more programs are executed by the one or more processors, the one or more processors are enabled to perform any one of the training methods for the road scene prediction model provided by the embodiments of the present invention.
According to another aspect of the present invention, there is provided a computer readable storage medium storing computer instructions for causing a processor to implement, when executed, a training method for any one of the road scene prediction models provided by the embodiments of the present invention.
The embodiment of the invention improves the training scheme of the road scene prediction model by acquiring the sample road image and the true value road scene type marked by the sample road image; inputting the sample road image into a road scene prediction model to obtain a predicted road scene type of the sample road image; respectively determining a scene basic loss value and uncertainty loss data of a road scene prediction model according to the true road scene type and the predicted road scene type of the sample road image; determining a model loss value of the road scene prediction model according to the scene basic loss value and the uncertainty loss data; and training the road scene prediction model by adopting the model loss value. According to the scheme, the uncertainty loss data is introduced to dynamically adjust the scene basic loss value, so that the road scene prediction model can be ensured to be used for carrying out balanced training on different road scene types, and finally, the road scene prediction model is converged to a stable state, and the accuracy of training the road scene prediction model is improved.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a training method of a road scene prediction model according to an embodiment of the present invention;
fig. 2 is a flowchart of a training method of a road scene prediction model according to a second embodiment of the present invention;
fig. 3 is a schematic structural diagram of a training device for a road scene prediction model according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device for implementing a training method of a road scene prediction model according to a fourth embodiment of the present invention.
Detailed Description
The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present invention are shown in the drawings.
Example 1
Fig. 1 is a flowchart of a method for training a road scene prediction model according to an embodiment of the present invention, where the method may be performed by a road scene prediction model when the road scene prediction model for performing road scene type prediction in automatic driving is trained, and the apparatus may be implemented in a software and/or hardware manner and may be configured in an electronic device that carries a training function of the road scene prediction model.
Referring to the training method of the road scene prediction model shown in fig. 1, the method includes:
s110, acquiring a sample road image and marking a true value road scene type by the sample road image.
The sample road image is a road image obtained in the past by a vehicle. The method for acquiring the sample road image is not limited in the embodiment of the invention, and can be set by a technician according to experience. For example, a sample road image may be acquired by a forward facing camera mounted on the vehicle.
The true value road scene type refers to the road scene type marked on the sample road image. Illustratively, the truth road scene types include a truth illumination type, a truth weather type, and a truth road type. The true value illumination type refers to the road illumination type corresponding to the sample road image. For example, the truth illumination type may include truth day and truth night. The true weather type refers to the road weather type corresponding to the sample road image. For example, the truth weather types may include truth sunny days, truth overcast days, truth rainy days, truth foggy days, and truth others. The true value road type refers to the road type corresponding to the sample road image. For example, the truth road types may include true high speed and true non-high speed.
It should be noted that, the method for labeling the true value road scene type in the embodiment of the invention is not limited, and the method can be manually labeled by a technician according to experience.
S120, inputting the sample road image into a road scene prediction model to obtain a predicted road scene type of the sample road image.
The road scene prediction model refers to a model for determining the type of the road scene. The predicted road scene type refers to a road scene type obtained based on a road scene prediction model. The predicted road scene types include a predicted illumination type, a predicted weather type, and a predicted road type. The predicted illumination type refers to a road illumination type corresponding to the obtained sample road image based on the road scene prediction model. For example, predicting the illumination type may include predicting a day and predicting a night. The predicted weather type is the road weather type corresponding to the obtained sample road image based on the road scene prediction model. For example, the predicted weather types may include predicting a sunny day, predicting a cloudy day, predicting a rainy day, predicting a foggy day, and predicting others. The predicted road type is the road type corresponding to the obtained sample road image based on the road scene prediction model. For example, the predicted road types may include predicted high speed and predicted non-high speed.
The road scene prediction model in the embodiment of the invention can be constructed based on a multi-task classification network; multiple tasks in a multi-task classification network share the same set of convolutional layers. For example, the road scene prediction model can adopt ResNet-18 as an infrastructure, so that the road scene prediction model meets the edge deployment requirement. Of these, resNet is a popular Convolutional Neural Network (CNN) structure that solves the problem of gradient extinction by introducing residual connections, enabling more advanced models to be trained.
It can be understood that the road scene prediction model in the embodiment of the invention is constructed based on the multi-task classification network, and can simultaneously predict the illumination type, the weather type and the road type, and the network adopts the design of a shared backbone network, namely a plurality of tasks share the same group of convolution layers, so that the parameter number of the road scene prediction model is reduced, and the demand on board-end computing resources is reduced.
S130, respectively determining a scene basic loss value and uncertainty loss data of a road scene prediction model according to the true road scene type and the predicted road scene type of the sample road image.
The scene basic loss value refers to a loss function value between a true road scene type and a predicted road scene type. The method for determining the scene basic loss value is not limited in any way, and can be set by a technician according to experience. For example, the scene basis loss value may be determined in a cross entropy function manner.
The uncertainty loss data can be used for adjusting the magnitude of a scene basic loss value in a model loss value and characterizing the stability of a sample road image predicted on a road scene type.
It should be noted that, in the road scene prediction model in the embodiment of the present invention, due to the possible difference in difficulty between different tasks (i.e. different road scene types) and the interaction between tasks, the conventional multi-task learning method may cause that some tasks cannot converge to an optimal solution or oscillations of a loss function occur, so uncertainty loss data is introduced in the embodiment of the present invention to balance training progress between different tasks and improve training accuracy.
And S140, determining a model loss value of the road scene prediction model according to the scene basic loss value and the uncertainty loss data.
The model loss value refers to an overall loss function value of the road scene prediction model.
And S150, training a road scene prediction model by adopting the model loss value.
Specifically, parameters in the road scene prediction model are adjusted based on the model loss value, so that training of the road scene prediction model is achieved.
The embodiment of the invention improves the training scheme of the road scene prediction model by acquiring the sample road image and the true value road scene type marked by the sample road image; inputting the sample road image into a road scene prediction model to obtain a predicted road scene type of the sample road image; respectively determining a scene basic loss value and uncertainty loss data of a road scene prediction model according to the true road scene type and the predicted road scene type of the sample road image; determining a model loss value of the road scene prediction model according to the scene basic loss value and the uncertainty loss data; and training the road scene prediction model by adopting the model loss value. According to the scheme, the uncertainty loss data is introduced to dynamically adjust the scene basic loss value, so that the road scene prediction model can be ensured to be used for carrying out balanced training on different road scene types, and finally, the road scene prediction model is converged to a stable state, and the accuracy of training the road scene prediction model is improved.
Example two
Fig. 2 is a flowchart of a training method for a road scene prediction model according to a second embodiment of the present invention, where the operation of determining a scene basis loss value and uncertainty loss data of the road scene prediction model according to a true road scene type and a predicted road scene type of a sample road image is further subdivided into "determining a scene basis loss value of the road scene prediction model according to a true road scene type and a predicted road scene type of the sample road image, respectively", based on a preset basis loss determination method, based on the above embodiments; the scene basic loss value comprises an illumination basic loss value, a weather basic loss value and a road basic loss value; respectively determining light type difference data between a true value light type and a predicted light type of a sample road image, weather type difference data between a true value weather type and a predicted weather type, and road type difference data between a true value road type and a predicted road type; respectively determining an illumination type variance, a weather type variance and a road type variance according to the illumination type difference data, the weather type difference data and the road type difference data; and determining uncertainty loss data of the road scene prediction model according to the illumination type variance, the weather type variance and the road type variance so as to perfect a data determination mechanism. In the portions of the embodiments of the present invention that are not described in detail, reference may be made to the descriptions of other embodiments.
Referring to fig. 2, the training method of the road scene prediction model includes:
and S210, acquiring a sample road image and marking a true value road scene type by the sample road image.
The truth road scene type comprises a truth illumination type, a truth weather type and a truth road type.
S220, inputting the sample road image into a road scene prediction model to obtain a predicted road scene type of the sample road image.
The predicted road scene type comprises a predicted illumination type, a predicted weather type and a predicted road type.
S230, determining scene foundation loss values of a road scene prediction model according to the true road scene type and the predicted road scene type of the sample road image based on a preset foundation loss determination method.
The method for determining the prediction basis loss is not limited in the embodiment of the invention, and can be set by a technician according to experience. For example, the prediction basis loss determination method may be a cross entropy loss function determination method.
The scene basic loss values comprise an illumination basic loss value, a weather basic loss value and a road basic loss value. The illumination base loss value refers to a loss function value between the true illumination type and the predicted illumination type. The weather base loss value refers to the loss function value between the true weather type and the predicted weather type. The road base loss value refers to a loss function value between the true road type and the predicted road type.
S240, respectively determining the light type difference data between the true-value light type and the predicted light type of the sample road image, the weather type difference data between the true-value weather type and the predicted weather type, and the road type difference data between the true-value road type and the predicted road type.
Wherein the care type difference data may be used to quantify a degree of difference between the true illumination type and the predicted illumination type. The weather type difference data may be used to quantify the degree of difference between the true weather type and the predicted weather type. The road type difference data may be used to quantify the degree of difference between the true road type and the predicted road type.
The truth road scene type and the predicted road scene type in the embodiment of the invention obey Gaussian distribution, so that the illumination type difference data, the weather type difference data and the road type difference data can be determined based on the truth road scene type and the predicted road scene type.
S250, respectively determining the illumination type variance, the weather type variance and the road type variance according to the illumination type difference data, the weather type difference data and the road type difference data.
Wherein the illumination type variance may be used to quantify the degree of discretization between the true illumination type and the predicted illumination type. The weather type variance may be used to quantify the degree of discretization between the true weather type and the predicted weather type. The road type variance may be used to quantify the degree of discretization between the true road type and the predicted road type.
And S260, determining uncertainty loss data of the road scene prediction model according to the illumination type variance, the weather type variance and the road type variance.
In the prior art, in the multi-task joint learning, some problems may occur due to different dependency relationships and difficulties of various tasks. For example, a task with higher relative difficulty may not converge to the vicinity of the optimal solution, or a task with lower relative difficulty may have serious loss oscillation in the training process, and the embodiment of the present invention introduces scene loss adjustment data and scene loss fluctuation data, so that the losses of a plurality of tasks can converge to the vicinity of the optimal solution and cannot excessively affect each other.
In an alternative embodiment, determining uncertainty loss data for a road scene prediction model based on a light type variance, a weather type variance, and a road type variance includes: obtaining scene loss adjustment data and scene loss fluctuation data of a road scene prediction model according to the illumination type variance, the weather type variance and the road type variance respectively; wherein the scene loss adjustment data includes illumination loss adjustment data, weather loss adjustment data, and road loss adjustment data; the scene loss fluctuation data comprise illumination loss fluctuation data, weather loss fluctuation data and road loss fluctuation data; uncertainty penalty data is generated that includes scene penalty adjustment data and scene penalty fluctuation data.
Wherein the scene loss adjustment data may be used to adjust the size of the scene base loss value. The illumination loss adjustment data may be used to adjust the magnitude of the illumination base loss value. The weather loss adjustment data may be used to adjust the magnitude of the weather base loss value. The road loss adjustment data may be used to adjust the magnitude of the road base loss value.
The scene loss fluctuation data can represent the stability of a prediction result of the road scene prediction model. The illumination loss fluctuation data can characterize the stability of the predicted illumination type output by the road scene prediction model. The weather loss fluctuation data may characterize the stability of the predicted weather type output by the road scene prediction model. The road loss fluctuation data may characterize the stability of the predicted road type output by the road scene prediction model.
It can be understood that by introducing scene loss adjustment data, adjustment of scene basic loss values is realized, so that different road scene types can be more uniformly trained in the road scene prediction model, and the different road scene types in the road scene prediction model can be converged to the vicinity of the optimal solution, and the accuracy of training the road scene prediction model is improved; meanwhile, the scene loss fluctuation data is introduced, so that the stability of the output result of the road scene prediction model is described, and the road scene prediction model can perform key training on a certain road scene type with larger fluctuation according to the scene loss fluctuation data, and the accuracy of training the road scene prediction model is improved.
S270, determining a model loss value of the road scene prediction model according to the scene basic loss value and the uncertainty loss data.
In an alternative embodiment, determining a model penalty value for the road scene prediction model based on the scene base penalty value and the uncertainty penalty data comprises: determining a scene loss adjustment value of the road scene prediction model according to the scene loss adjustment data and the scene basis loss value; the scene loss adjustment values comprise an illumination loss adjustment value, a weather loss adjustment value and a road loss adjustment value; and determining a model loss value of the road scene prediction model according to the scene loss adjustment value and the scene loss fluctuation data.
The scene loss adjustment value refers to an adjusted scene basic loss value. The illumination loss adjustment value refers to data obtained by adjusting the illumination base loss value based on the illumination loss adjustment data. The weather loss adjustment value refers to data obtained by adjusting the weather base loss value based on the weather loss adjustment data. The road loss adjustment value refers to data obtained by adjusting the road base loss value based on the road loss adjustment data.
It can be appreciated that by introducing the scene loss adjustment value, determining the model loss value based on the scene loss adjustment value and scene loss fluctuation data, the accuracy of the determined model loss value is improved, and further, the accuracy of the subsequent training of the road scene prediction model based on the model loss value is improved.
In an alternative embodiment, determining a model penalty value for the road scene prediction model based on the scene base penalty value and the uncertainty penalty data comprises: based on the following model loss calculation formula, processing the scene basic loss value and the uncertainty loss data, and calculating to obtain a model loss value:
Wherein L total represents a model loss value; i represents a road scene type; σ i represents the type variance corresponding to the road scene type; l i (W) represents a scene basis loss value corresponding to the road scene type; Scene loss adjustment data corresponding to the road scene type; /(I) Scene loss fluctuation data corresponding to the road scene type is represented. The road scene type comprises an illumination type, a weather type and a road type.
Illustratively, the model loss value of the road scene prediction model is determined based on the model loss calculation formula described above:
Wherein L total represents a model loss value; 1 represents an illumination type; σ 1 represents the illumination type variance; l 1 (W) represents an illumination base loss value; Representing illumination loss fluctuation data; /(I) Representing illumination loss adjustment data; /(I)Representing an illumination loss adjustment value; 2 represents a weather type; σ 2 represents the weather type variance; l 2 (W) represents a weather base loss value; /(I)Weather loss fluctuation data; /(I)Weather loss adjustment data; /(I)Representing a weather loss adjustment value; 3 represents a road type; σ 3 represents the road type variance; l 3 (W) represents a road base loss value; /(I)Road loss fluctuation data is represented; Means for representing road loss adjustment data; /(I) Representing the road loss adjustment value.
It can be appreciated that by introducing a model loss calculation formula, the model loss value is determined, and the accuracy of the determined model loss value is improved.
S280, training a road scene prediction model by adopting a model loss value.
The embodiment of the invention provides a training scheme of a road scene prediction model, which is characterized in that scene basic loss values and uncertainty loss data operations of the road scene prediction model are respectively determined according to true road scene types and predicted road scene types of a sample road image, and are refined into scene basic loss values of the road scene prediction model based on a preset basic loss determination method according to the true road scene types and the predicted road scene types of the sample road image; the scene basic loss value comprises an illumination basic loss value, a weather basic loss value and a road basic loss value; respectively determining light type difference data between a true value light type and a predicted light type of a sample road image, weather type difference data between a true value weather type and a predicted weather type, and road type difference data between a true value road type and a predicted road type; respectively determining an illumination type variance, a weather type variance and a road type variance according to the illumination type difference data, the weather type difference data and the road type difference data; and determining uncertainty loss data of the road scene prediction model according to the illumination type variance, the weather type variance and the road type variance, so that a data determination mechanism is perfected. According to the scheme, the uncertainty loss data of the road scene prediction model are determined by introducing the illumination type variance, the weather type variance and the road type variance, so that the accuracy of the determined uncertainty loss data is improved.
On the basis of the technical scheme, the embodiment of the invention provides a use scheme of a road scene prediction model. In the prior art, the performance of collision warning systems in Automatic Driving Assistance Systems (ADASs) is often affected by a variety of factors, such as visibility, road friction, vehicle speed, and the like. Variations in these factors can lead to reduced accuracy and reliability of the collision warning system. In the embodiment of the invention, the trained road scene prediction model is deployed on the vehicle and can be used for analyzing the road image captured by the camera in real time, the road scene prediction model can periodically classify the road image, identify the type of the road scene where the vehicle is currently located and provide the information to the collision early warning system. Thus, the early warning system can adjust parameters according to the current environmental conditions, so that more accurate and reliable early warning service is provided.
According to the embodiment of the invention, the performance of the collision early warning system in the ADAS system is improved by combining the deep learning technology and the image processing algorithm, the limitation faced by the traditional method under the changeable environmental conditions is overcome, and safer and more comfortable driving experience is brought to drivers and passengers; in addition, the road scene prediction model in the embodiment of the invention adopts a multi-task classification network sharing a backbone network, predicts three tasks of illumination intensity, weather type and road type, introduces uncertainty loss data to enable multi-task collaborative training to be more stable, has high prediction accuracy and strong universality, and can be widely applied to various road scenes.
Example III
Fig. 3 is a schematic structural diagram of a training device for a road scene prediction model according to a third embodiment of the present invention. The embodiment is applicable to the condition that a road scene prediction model for road scene type prediction in automatic driving is trained, the method can be executed by the road scene prediction model, and the device can be realized in a software and/or hardware mode and can be configured in an electronic device for bearing the training function of the road scene prediction model.
As shown in fig. 3, the apparatus includes: an image acquisition module 310, a model prediction module 320, a data determination module 330, a model loss determination module 340, and a model training module 350. Wherein,
An image obtaining module 310, configured to obtain a sample road image and a truth road scene type marked by the sample road image; the truth road scene type comprises a truth illumination type, a truth weather type and a truth road type;
The model prediction module 320 is configured to input the sample road image into a road scene prediction model to obtain a predicted road scene type of the sample road image; the predicted road scene type comprises a predicted illumination type, a predicted weather type and a predicted road type;
A data determining module 330, configured to determine a scene basic loss value and uncertainty loss data of the road scene prediction model according to the true road scene type and the predicted road scene type of the sample road image, respectively; the uncertainty loss data is used for adjusting the magnitude of the scene basic loss value in a model loss value and representing the stability of the sample road image for prediction on the road scene type;
A model loss determination module 340, configured to determine a model loss value of the road scene prediction model according to the scene basic loss value and the uncertainty loss data;
And the model training module 350 is configured to train the road scene prediction model by using the model loss value.
The embodiment of the invention improves the training scheme of the road scene prediction model by acquiring the sample road image and the true value road scene type marked by the sample road image; inputting the sample road image into a road scene prediction model to obtain a predicted road scene type of the sample road image; respectively determining a scene basic loss value and uncertainty loss data of a road scene prediction model according to the true road scene type and the predicted road scene type of the sample road image; determining a model loss value of the road scene prediction model according to the scene basic loss value and the uncertainty loss data; and training the road scene prediction model by adopting the model loss value. According to the scheme, the uncertainty loss data is introduced to dynamically adjust the scene basic loss value, so that the road scene prediction model can be ensured to be used for carrying out balanced training on different road scene types, and finally, the road scene prediction model is converged to a stable state, and the accuracy of training the road scene prediction model is improved.
Optionally, the data determining module 330 includes:
a base loss value determining unit, configured to determine, based on a preset base loss determining method, the scene base loss value of the road scene prediction model according to the true road scene type and the predicted road scene type of the sample road image, respectively; the scene basic loss value comprises an illumination basic loss value, a weather basic loss value and a road basic loss value;
A difference data determining unit configured to determine illumination type difference data between the true-value illumination type and the predicted illumination type of the sample road image, weather type difference data between the true-value weather type and the predicted weather type, and road type difference data between the true-value road type and the predicted road type, respectively;
The variance determining unit is used for determining illumination type variances, weather type variances and road type variances according to the illumination type difference data, the weather type difference data and the road type difference data;
And the loss data determining unit is used for determining uncertainty loss data of the road scene prediction model according to the illumination type variance, the weather type variance and the road type variance.
Optionally, the loss data determining unit is specifically configured to:
Obtaining scene loss adjustment data and scene loss fluctuation data of the road scene prediction model according to the illumination type variance, the weather type variance and the road type variance respectively; wherein the scene loss adjustment data includes lighting loss adjustment data, weather loss adjustment data, and road loss adjustment data; the scene loss fluctuation data comprise illumination loss fluctuation data, weather loss fluctuation data and road loss fluctuation data;
the uncertainty penalty data including the scene penalty adjustment data and the scene penalty fluctuation data is generated.
Optionally, the model loss determination module is specifically configured to:
Determining a scene loss adjustment value of the road scene prediction model according to the scene loss adjustment data and the scene base loss value; the scene loss adjusting values comprise an illumination loss adjusting value, a weather loss adjusting value and a road loss adjusting value;
And determining a model loss value of the road scene prediction model according to the scene loss adjustment value and the scene loss fluctuation data.
Optionally, the model loss determination module 340 is specifically configured to:
processing the scene basic loss value and the uncertainty loss data based on the following model loss calculation formula, and calculating to obtain the model loss value:
Wherein L total represents a model loss value; i represents a road scene type; σ i represents the type variance corresponding to the road scene type; l i (W) represents a scene basis loss value corresponding to the road scene type; Scene loss adjustment data corresponding to the road scene type; /(I) Scene loss fluctuation data corresponding to the road scene type is represented.
Optionally, the road scene prediction model is constructed based on a multi-task classification network; multiple tasks in the multi-task classification network share the same set of convolutional layers.
The training device for the road scene prediction model provided by the embodiment of the invention can execute the training method for the road scene prediction model provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of executing the training method for each road scene prediction model.
In the technical scheme of the invention, the related processes of collecting, storing, using, processing, transmitting, providing, disclosing and the like of the sample road image and the like all accord with the regulations of related laws and regulations, and the public order is not violated.
Example IV
Fig. 4 is a schematic structural diagram of an electronic device for implementing a training method of a road scene prediction model according to a fourth embodiment of the present invention. The electronic device 410 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 4, the electronic device 410 includes at least one processor 411, and a memory, such as a Read Only Memory (ROM) 412, a Random Access Memory (RAM) 413, etc., communicatively connected to the at least one processor 411, wherein the memory stores computer programs executable by the at least one processor, and the processor 411 may perform various suitable actions and processes according to the computer programs stored in the Read Only Memory (ROM) 412 or the computer programs loaded from the storage unit 418 into the Random Access Memory (RAM) 413. In the RAM 413, various programs and data required for the operation of the electronic device 410 may also be stored. The processor 411, the ROM 412, and the RAM 413 are connected to each other through a bus 414. An input/output (I/O) interface 415 is also connected to bus 414.
Various components in the electronic device 410 are connected to the I/O interface 415, including: an input unit 416 such as a keyboard, a mouse, etc.; an output unit 417 such as various types of displays, speakers, and the like; a storage unit 418, such as a magnetic disk, optical disk, or the like; and a communication unit 419 such as a network card, modem, wireless communication transceiver, etc. The communication unit 419 allows the electronic device 410 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.
The processor 411 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 411 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 411 performs the various methods and processes described above, such as the training method of the road scene prediction model.
In some embodiments, the method of training the road scene prediction model may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as storage unit 418. In some embodiments, some or all of the computer program may be loaded and/or installed onto the electronic device 410 via the ROM 412 and/or the communication unit 419. When the computer program is loaded into RAM 413 and executed by processor 411, one or more steps of the above-described training method of the road scene prediction model may be performed. Alternatively, in other embodiments, the processor 411 may be configured to perform the training method of the road scene prediction model in any other suitable way (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (10)
1. A method of training a road scene prediction model, comprising:
acquiring a sample road image and a truth value road scene type marked by the sample road image; the truth road scene type comprises a truth illumination type, a truth weather type and a truth road type;
Inputting the sample road image into a road scene prediction model to obtain a predicted road scene type of the sample road image; the predicted road scene type comprises a predicted illumination type, a predicted weather type and a predicted road type;
respectively determining a scene basic loss value and uncertainty loss data of the road scene prediction model according to the true road scene type and the predicted road scene type of the sample road image; the uncertainty loss data is used for adjusting the magnitude of the scene basic loss value in a model loss value and representing the stability of the sample road image for prediction on the road scene type;
Determining a model loss value of the road scene prediction model according to the scene basic loss value and the uncertainty loss data;
and training the road scene prediction model by adopting the model loss value.
2. The method of claim 1, wherein the determining scene basis loss values and uncertainty loss data of the road scene prediction model from the true road scene type and the predicted road scene type of the sample road image, respectively, comprises:
Based on a preset basic loss determination method, determining the scene basic loss value of the road scene prediction model according to the true road scene type and the predicted road scene type of the sample road image respectively; the scene basic loss value comprises an illumination basic loss value, a weather basic loss value and a road basic loss value;
Respectively determining illumination type difference data between the true-value illumination type and the predicted illumination type of the sample road image, weather type difference data between the true-value weather type and the predicted weather type, and road type difference data between the true-value road type and the predicted road type;
respectively determining an illumination type variance, a weather type variance and a road type variance according to the illumination type difference data, the weather type difference data and the road type difference data;
and determining uncertainty loss data of the road scene prediction model according to the illumination type variance, the weather type variance and the road type variance.
3. The method of claim 2, wherein the determining uncertainty loss data for the road scene prediction model from the illumination type variance, the weather type variance, and the road type variance comprises:
Obtaining scene loss adjustment data and scene loss fluctuation data of the road scene prediction model according to the illumination type variance, the weather type variance and the road type variance respectively; wherein the scene loss adjustment data includes lighting loss adjustment data, weather loss adjustment data, and road loss adjustment data; the scene loss fluctuation data comprise illumination loss fluctuation data, weather loss fluctuation data and road loss fluctuation data;
the uncertainty penalty data including the scene penalty adjustment data and the scene penalty fluctuation data is generated.
4. A method according to claim 3, wherein said determining a model loss value of the road scene prediction model from the scene basis loss value and the uncertainty loss data comprises:
Determining a scene loss adjustment value of the road scene prediction model according to the scene loss adjustment data and the scene base loss value; the scene loss adjusting values comprise an illumination loss adjusting value, a weather loss adjusting value and a road loss adjusting value;
And determining a model loss value of the road scene prediction model according to the scene loss adjustment value and the scene loss fluctuation data.
5. The method according to claim 3 or 4, wherein said determining a model loss value of the road scene prediction model from the scene basis loss value and the uncertainty loss data comprises:
processing the scene basic loss value and the uncertainty loss data based on the following model loss calculation formula, and calculating to obtain the model loss value:
Wherein L total represents a model loss value; i represents a road scene type; σ i represents the type variance corresponding to the road scene type; l i (W) represents a scene basis loss value corresponding to the road scene type; Scene loss adjustment data corresponding to the road scene type; σ i 2 represents scene loss fluctuation data corresponding to the road scene type.
6. The method of any one of claims 1-4, wherein the road scene prediction model is constructed based on a multi-tasking classification network; multiple tasks in the multi-task classification network share the same set of convolutional layers.
7. A training device for a road scene prediction model, comprising:
The image acquisition module is used for acquiring a sample road image and a truth value road scene type marked by the sample road image; the truth road scene type comprises a truth illumination type, a truth weather type and a truth road type;
The model prediction module is used for inputting the sample road image into a road scene prediction model to obtain a predicted road scene type of the sample road image; the predicted road scene type comprises a predicted illumination type, a predicted weather type and a predicted road type;
The data determining module is used for respectively determining a scene basic loss value and uncertainty loss data of the road scene prediction model according to the true road scene type and the predicted road scene type of the sample road image; the uncertainty loss data is used for adjusting the magnitude of the scene basic loss value in a model loss value and representing the stability of the sample road image for prediction on the road scene type;
the model loss determining module is used for determining a model loss value of the road scene prediction model according to the scene basic loss value and the uncertainty loss data;
and the model training module is used for training the road scene prediction model by adopting the model loss value.
8. The apparatus of claim 7, wherein the data determination module comprises:
a base loss value determining unit, configured to determine, based on a preset base loss determining method, the scene base loss value of the road scene prediction model according to the true road scene type and the predicted road scene type of the sample road image, respectively; the scene basic loss value comprises an illumination basic loss value, a weather basic loss value and a road basic loss value;
A difference data determining unit configured to determine illumination type difference data between the true-value illumination type and the predicted illumination type of the sample road image, weather type difference data between the true-value weather type and the predicted weather type, and road type difference data between the true-value road type and the predicted road type, respectively;
The variance determining unit is used for determining illumination type variances, weather type variances and road type variances according to the illumination type difference data, the weather type difference data and the road type difference data;
And the loss data determining unit is used for determining uncertainty loss data of the road scene prediction model according to the illumination type variance, the weather type variance and the road type variance.
9. An electronic device, comprising:
one or more processors;
a memory for storing one or more programs;
When executed by the one or more processors, causes the one or more processors to implement a method of training a road scene prediction model as claimed in any one of claims 1 to 6.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements a method for training a road scene prediction model according to any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410166174.9A CN117953348A (en) | 2024-02-05 | 2024-02-05 | Training method, device, equipment and medium of road scene prediction model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410166174.9A CN117953348A (en) | 2024-02-05 | 2024-02-05 | Training method, device, equipment and medium of road scene prediction model |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117953348A true CN117953348A (en) | 2024-04-30 |
Family
ID=90800075
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410166174.9A Pending CN117953348A (en) | 2024-02-05 | 2024-02-05 | Training method, device, equipment and medium of road scene prediction model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117953348A (en) |
-
2024
- 2024-02-05 CN CN202410166174.9A patent/CN117953348A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220092874A1 (en) | Method and apparatus of determining vehicle queuing information, roadside device and cloud control platform | |
| CN115861380B (en) | Method and device for tracking visual target of end-to-end unmanned aerial vehicle under foggy low-illumination scene | |
| CN113135193A (en) | Method, apparatus, storage medium, and program product for outputting warning information | |
| CN113780578B (en) | Model training method, device, electronic equipment and readable storage medium | |
| CN112699773B (en) | Traffic light identification method and device and electronic equipment | |
| CN117953348A (en) | Training method, device, equipment and medium of road scene prediction model | |
| CN114970737A (en) | Accident reason determining method, device, equipment and storage medium | |
| CN113793373A (en) | Visibility detection method, device, equipment and medium | |
| CN117647852B (en) | Weather state detection method and device, electronic equipment and storage medium | |
| CN117962905A (en) | Speed limit information determining method and device, electronic equipment and storage medium | |
| CN115857176B (en) | Head-up display, height adjusting method and device thereof and storage medium | |
| CN116691673A (en) | Vehicle control method, device, equipment and storage medium | |
| CN116168132B (en) | Street view reconstruction model acquisition method, device, equipment and medium | |
| CN112784789B (en) | Method, device, electronic equipment and medium for identifying traffic flow of road | |
| CN116358902B (en) | Vehicle function testing method and device, electronic equipment and storage medium | |
| CN117962774A (en) | Weather state prediction method and device, electronic equipment and storage medium | |
| CN116401554A (en) | Data classification method, device, equipment and medium | |
| CN115512188A (en) | Multi-target detection method, device, equipment and medium | |
| CN114898323A (en) | Scene matching method and device, electronic equipment and storage medium | |
| CN116630916A (en) | Object detection method, device, equipment and storage medium | |
| CN118062010A (en) | Channel change decision method and device, electronic equipment and storage medium | |
| CN116767262A (en) | Driving auxiliary line display method, device, equipment and medium | |
| CN117799614A (en) | Intelligent lane changing method applied to vehicle | |
| CN117302268A (en) | Driving strategy determination method, model training method and automatic driving vehicle | |
| CN117853971A (en) | Method, device, equipment and storage medium for detecting sprinkled object |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |