CN113989785A - Driving scene classification method, device, equipment and storage medium - Google Patents
Driving scene classification method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN113989785A CN113989785A CN202111458258.2A CN202111458258A CN113989785A CN 113989785 A CN113989785 A CN 113989785A CN 202111458258 A CN202111458258 A CN 202111458258A CN 113989785 A CN113989785 A CN 113989785A
- Authority
- CN
- China
- Prior art keywords
- driving
- classification
- classification result
- image
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Probability & Statistics with Applications (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application discloses a driving scene classification method and a related device, wherein the method comprises the following steps: acquiring a target driving image, wherein the target driving image is a driving environment image acquired in the current driving process; then, a multi-classification result of the current driving environment corresponding to the target driving image is determined according to the target driving image through a driving scene multi-classification model, wherein the multi-classification result comprises classification results of the current driving environment under multiple reference class branches, and the driving scene multi-classification model is used for identifying the classification results of the driving image under the multiple reference class branches according to the driving image corresponding to the driving environment. Therefore, the multi-classification detection of the driving environment is realized, and the classification result of the driving environment under various reference type branches is identified, so that the actual requirement of an intelligent driving scene can be met, and the effect of assisting intelligent driving is effectively achieved.
Description
Technical Field
The application relates to the technical field of computer vision, in particular to a driving scene classification method, a device, equipment and a storage medium.
Background
In recent years, various technologies related to smart driving have been rapidly developed. The road scene perception technology is particularly important in the field of intelligent driving, and is particularly used for identifying the category of a current driving scene according to a road scene image captured by a vehicle-mounted camera.
The road scene perception technology is used as an important component of an intelligent vehicle-mounted system, and can be helpful for improving the robustness of the intelligent driving system, and different reactions in different driving scenes are realized, so that the safety of intelligent driving is ensured. For example, when the current driving scene is judged to belong to fog, rain, snow and the like, the intelligent driving system can appropriately adjust the driving state of the vehicle, such as control the vehicle to decelerate, so that the vehicle adapts to the current driving scene, dangers are prevented, and the accident rate is reduced.
At present, the related art can only perform single-label identification on a driving scene image, namely, whether a current driving scene belongs to a certain specific driving scene is identified according to the driving scene image, and the identification method is difficult to meet the actual requirements of an intelligent driving scene and effectively play a role in assisting intelligent driving.
Disclosure of Invention
The embodiment of the application provides a driving scene classification method, a driving scene classification device and a driving scene classification storage medium, which can realize multi-label identification aiming at a driving scene image and meet the actual requirements of an intelligent driving scene.
In view of the above, a first aspect of the present application provides a driving scene classification method, including:
acquiring a target driving image; the target driving image is a driving environment image acquired in the current driving process;
determining a multi-classification result of the current driving environment corresponding to the target driving image according to the target driving image through a driving scene multi-classification model; the multi-classification result comprises a classification result of the current driving environment under a plurality of reference category branches; the driving scene multi-classification model is used for identifying the classification result of the driving image under the multiple reference category branches according to the driving image corresponding to the driving environment.
Optionally, the determining, by the driving scene multi-classification model and according to the target driving image, a multi-classification result of the current driving environment corresponding to the target driving image includes:
determining a classification result of the current driving environment under an illumination branch, a classification result of the current driving environment under a weather branch and a classification result of the current driving environment under a camera state branch according to the target driving image through the driving scene multi-classification model;
candidate classification results under the illumination branch comprise day, dusk and night; candidate classification results under the weather branch comprise sunny days, rainy days, foggy days and snowy days; the candidate classification results under the camera state branch include a normal camera state and an abnormal camera state.
Optionally, the determining, by the driving scene multi-classification model and according to the target driving image, a classification result of the current driving environment under an illumination branch, a classification result of the current driving environment under a weather branch, and a classification result of the current driving environment under a camera state branch includes:
determining a multi-classification result vector according to the target driving image through the driving scene multi-classification model; the multi-classification result vector comprises 9 predicted values, a first predicted value to a third predicted value in the multi-classification result vector are used for bearing classification probabilities corresponding to three candidate classification results under the illumination branch, a fourth predicted value to a seventh predicted value in the multi-classification result vector are used for bearing classification probabilities corresponding to four candidate classification results under the weather branch, and an eighth predicted value to a ninth predicted value in the multi-classification result vector are used for bearing classification probabilities corresponding to two candidate classification results under the camera state branch;
determining a classification result of the current driving environment under an illumination branch according to a first prediction value to a third prediction value in the multi-classification result vector; determining the classification result of the current driving environment under the weather branch according to the fourth predicted value to the seventh predicted value in the multi-classification result vector; and determining the classification result of the current driving environment under the camera state branch according to the eighth predicted value to the ninth predicted value in the multi-classification result vector.
Optionally, the driving scene multi-classification model includes a plurality of feature extraction units, each feature extraction unit includes a depth separable convolution structure and a residual structure, and each depth separable convolution structure includes a depth convolution structure for extracting image space features and a point convolution structure for extracting image channel features.
Optionally, the driving scenario multi-classification model is trained by:
acquiring a training sample set; the training sample set comprises a plurality of training samples, each training sample comprises a training driving image and a corresponding labeled multi-classification result, and the labeled multi-classification result comprises labeled classification results of the training driving images under the multiple reference category branches;
determining a prediction multi-classification result corresponding to the training driving image according to the training driving image in the training sample through a driving scene multi-classification model to be trained; the predicted multi-classification result comprises a predicted classification result of the training driving image under the multiple reference class branches;
constructing a loss function according to the labeling classification result and the prediction classification result in the training sample;
and adjusting model parameters of the driving scene multi-classification model based on the loss function.
Optionally, before determining, by the driving scene multi-classification model to be trained, a prediction multi-classification result corresponding to the training driving image according to the training driving image in the training sample, the method further includes:
performing data preprocessing and/or data enhancement operation on the training driving image in the training sample to obtain a target training driving image;
determining a prediction multi-classification result corresponding to the training driving image according to the training driving image in the training sample through the driving scene multi-classification model to be trained, wherein the prediction multi-classification result comprises the following steps:
and determining a prediction multi-classification result corresponding to the target training driving image according to the target training driving image through a driving scene multi-classification model to be trained.
A second aspect of the present application provides a driving scenario classification apparatus, the apparatus comprising:
the image acquisition module is used for acquiring a target driving image; the target driving image is a driving environment image acquired in the current driving process;
the image multi-classification module is used for determining a multi-classification result of the current driving environment corresponding to the target driving image according to the target driving image through a driving scene multi-classification model; the multi-classification result comprises a classification result of the current driving environment under a plurality of reference category branches; the driving scene multi-classification model is used for identifying the classification result of the driving image under the multiple reference category branches according to the driving image corresponding to the driving environment.
Optionally, the image multi-classification module is specifically configured to:
determining a classification result of the current driving environment under an illumination branch, a classification result of the current driving environment under a weather branch and a classification result of the current driving environment under a camera state branch according to the target driving image through the driving scene multi-classification model;
candidate classification results under the illumination branch comprise day, dusk and night; candidate classification results under the weather branch comprise sunny days, rainy days, foggy days and snowy days; the candidate classification results under the camera state branch include a normal camera state and an abnormal camera state.
A third aspect of the present application provides an electronic device, the device comprising: a processor and a memory;
the memory for storing a computer program;
the processor is configured to invoke the computer program to execute the driving scenario classification method according to the first aspect.
A fourth aspect of the present application provides a computer-readable storage medium for storing a computer program for executing the driving scenario classification method of the first aspect.
According to the technical scheme, the embodiment of the application has the following advantages:
the embodiment of the application provides a driving scene classification method, wherein a target driving image is acquired firstly, and the target driving image is a driving environment image acquired in the current driving process; then, a multi-classification result of the current driving environment corresponding to the target driving image is determined according to the target driving image through a driving scene multi-classification model, wherein the multi-classification result comprises classification results of the current driving environment under multiple reference class branches, and the driving scene multi-classification model is used for identifying the classification results of the driving image under the multiple reference class branches according to the driving image corresponding to the driving environment. Therefore, the multi-classification detection of the driving environment is realized, and the classification result of the driving environment under various reference type branches is identified, so that the actual requirement of an intelligent driving scene can be met, and the effect of assisting intelligent driving is effectively achieved.
Drawings
Fig. 1 is a schematic flowchart of a driving scene classification method according to an embodiment of the present disclosure;
fig. 2 is a schematic diagram of a working architecture of a driving scene multi-classification model according to an embodiment of the present application;
FIG. 3 is a schematic structural diagram of a depth separable convolution structure provided in an embodiment of the present application;
FIG. 4 is a schematic flowchart of a model training method for a driving scene multi-classification model according to an embodiment of the present disclosure;
FIG. 5 is a schematic diagram of a training and application architecture of a driving scenario multi-classification model provided in an embodiment of the present application;
fig. 6 is a schematic structural diagram of a driving scene classification device according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions of the present application better understood, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the drawings described above, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Referring to fig. 1, fig. 1 is a schematic flow chart of a driving scene classification method provided in an embodiment of the present application. It should be understood that the execution subject of the driving scene classification method may be any electronic device with image processing capability, such as a terminal device or a server, wherein the terminal device may be, for example, an in-vehicle terminal, and the server may be, for example, a cloud server of an intelligent driving system. As shown in fig. 1, the driving scene classification method includes the steps of:
step 101: acquiring a target driving image; the target driving image is a driving environment image acquired during the current driving.
In practical applications, the vehicle-mounted camera mounted on the vehicle can acquire a driving environment image in real time during the driving process of the vehicle, and the driving environment image can be a picture or a video image (i.e. a video frame in a video) including a scene around a road. After the vehicle-mounted terminal acquires the driving environment image acquired by the vehicle-mounted camera, the driving environment image can be regarded as a target driving image, and the type of the current driving environment is identified according to the target driving image; or, the vehicle-mounted terminal may also upload a target driving environment image acquired by the vehicle-mounted camera to the server, and the server identifies the type of the current driving environment according to the target driving image.
Step 102: determining a multi-classification result of the current driving environment corresponding to the target driving image according to the target driving image through a driving scene multi-classification model; the multi-classification result comprises a classification result of the current driving environment under a plurality of reference category branches; the driving scene multi-classification model is used for identifying the classification result of the driving image under the multiple reference category branches according to the driving image corresponding to the driving environment.
After the target driving image is acquired, the target driving image can be input into a pre-trained driving scene multi-classification model, and the driving scene multi-classification model can correspondingly output a multi-classification result of the current driving environment corresponding to the target driving image by analyzing and processing the target driving image. The multi-classification result comprises a classification result of the current driving environment corresponding to the target driving image under a plurality of reference class branches.
In a possible implementation manner, according to the target driving image, a classification result of a current driving environment corresponding to the target driving image under an illumination branch, a classification result under a weather branch, and a classification result under a camera state branch may be determined through the driving scene multi-classification model; the candidate classification results under the illumination branch comprise day, dusk and night; candidate classification results under the weather branch include sunny days, rainy days, foggy days and snowy days; the candidate classification results under the camera state branch include a normal camera state and an abnormal camera state.
Specifically, after the target driving image is input into the driving scene multi-classification model, the driving scene multi-classification model can correspondingly output the classification result of the current driving environment corresponding to the target driving image under the illumination branch by analyzing and processing the target driving image, and the output classification result under the illumination branch can be any one of three candidate classification results under the illumination branch, namely any one of the daytime, the dusk and the nighttime; the driving scene multi-classification model can also output a classification result of the current driving environment corresponding to the target driving image under a weather branch, wherein the output classification result under the weather branch can be any one of four candidate classification results under the weather branch, namely any one of sunny days, rainy days, foggy days and snowy days; the driving scene multi-classification model can also output the classification result of the current driving environment corresponding to the target driving image under the camera state branch, the output classification result under the camera state branch can be any one of two candidate classification results under the camera state branch, namely, the classification result under the camera state branch is any one of the normal camera state and the abnormal camera state, and the abnormal camera state can be caused by the lens contamination stain or the large water drop, the lens blur, the night light overexposure and the like.
As an example, a multi-classification result vector may be determined from the target driving image through a driving scene multi-classification model; the multi-classification result vector comprises 9 predicted values, a first predicted value to a third predicted value in the multi-classification result vector are used for bearing classification probabilities corresponding to three candidate classification results under an illumination branch, a fourth predicted value to a seventh predicted value in the multi-classification result vector are used for bearing classification probabilities corresponding to four candidate classification results under a weather branch, and an eighth predicted value to a ninth predicted value in the multi-classification result vector are used for bearing classification probabilities corresponding to two candidate classification results under a camera state branch. Further, according to the first predicted value to the third predicted value in the multi-classification result vector, the classification result of the current driving environment under the illumination branch is determined; determining the classification result of the current driving environment under the weather branch according to the fourth predicted value to the seventh predicted value in the multi-classification result vector; and determining the classification result of the current driving environment under the camera state branch according to the eighth predicted value to the ninth predicted value in the multi-classification result vector.
Specifically, the driving scene multi-classification model can correspondingly output a one-dimensional multi-classification result vector by analyzing and processing the target driving image, the multi-classification result vector includes nine predicted values, the sequence numbers of the nine predicted values are 0 to 8 respectively, and the nine predicted values are segmented and grouped, wherein the predicted values with the sequence numbers of 0 to 2 are used for representing the classification result under the illumination branch, for example, the predicted value with the sequence number of 0 is used for representing the probability that the current driving environment corresponding to the target driving image is daytime, the predicted value with the sequence number of 1 is used for representing the probability that the current driving environment corresponding to the target driving image is dusk, and the predicted value with the sequence number of 2 is used for representing the probability that the current driving environment corresponding to the target driving image is nighttime; the predicted values with the sequence numbers of 3 to 6 are used for representing the classification results under the weather branch, for example, the predicted value with the sequence number of 3 is used for representing the probability that the current driving environment corresponding to the target driving image is sunny, the predicted value with the sequence number of 4 is used for representing the probability that the current driving environment corresponding to the target driving image is rainy, the predicted value with the sequence number of 5 is used for representing the probability that the current driving environment corresponding to the target driving image is foggy, and the predicted value with the sequence number of 6 is used for representing the probability that the current driving environment corresponding to the target driving image is snowy; the predicted values with the sequence numbers of 7 to 8 are used for representing the classification results under the camera state branch, for example, the predicted value with the sequence number of 7 is used for representing the probability that the current driving environment corresponding to the target driving image is in a normal camera state, and the predicted value with the sequence number of 8 is used for representing the probability that the current driving environment corresponding to the target driving image is in an abnormal camera state.
Furthermore, the classification result under the illumination branch can be determined according to the predicted value with the sequence number of 0-2 in the multi-classification result vector; for example, the largest predicted value in the predicted values of the sequences 0 to 2 is determined, and the candidate classification result corresponding to the largest predicted value is the classification result of the current driving environment corresponding to the target driving image under the illumination branch. The classification result under the illumination branch can be determined according to the predicted values with the sequence numbers of 3 to 6 in the multi-classification result vector; for example, the largest predicted value in the predicted values of the sequences 3 to 6 is determined, and the candidate classification result corresponding to the largest predicted value is the classification result of the current driving environment corresponding to the target driving image under the weather branch. The classification result under the camera state branch can be determined according to the predicted values with the sequence numbers of 7-8 in the multi-classification result vector; for example, the largest predicted value in the predicted values of the sequences 7 to 8 is determined, and the candidate classification result corresponding to the largest predicted value is the classification result of the current driving environment corresponding to the target driving image under the camera state branch.
Fig. 2 is a schematic view of an exemplary operating architecture of a driving scenario multi-classification model according to an embodiment of the present application. As shown in fig. 2, after the target driving image is input into the driving scene multi-classification model, a BackBone feature extraction process may be performed first to obtain a feature map corresponding to the target driving image. Determining the classification result of the target driving image under the illumination branch, the weather branch and the camera state branch according to the characteristic diagram corresponding to the target driving image through a classifier; the classifier herein may specifically include three Softmax layers, which are respectively used to determine the classification results of the driving image under the illumination branch, the weather branch, and the camera state branch. And finally, jointly outputting the classification results of the target driving image under the illumination branch, the weather branch and the camera state branch to obtain a multi-classification result vector.
In consideration of the requirements and characteristics of actual deployment, the embodiment of the application can adopt a lightweight network based on deep separable convolution as the driving scene multi-classification model, and the lightweight driving scene multi-classification model is adopted to execute the driving scene multi-classification task, so that the driving scene multi-classification task can be guaranteed to be completed at a higher reasoning speed, the actual requirements of an intelligent driving system are met, namely the intelligent driving system can be guaranteed to rapidly know the specific category of the current driving scene, and then a decision instruction is rapidly given. That is, the driving scene multi-classification model may include a plurality of feature extraction units, each of which includes a depth-separable convolution structure including a depth convolution structure for extracting an image space feature and a point convolution structure for extracting an image channel feature, and a residual structure.
Specifically, the structure of the depth-separable Convolution structure is composed of a depth Convolution (Depthwise Convolution) structure and a 1 × 1 point Convolution (Pointwise Convolution) structure, as shown in fig. 3, the depth-separable Convolution structure separately learns the spatial features and the channel features of the image, learns the spatial features of the image from the depth Convolution structure, and learns the channel features of the image from the point Convolution structure, so that the computation amount can be reduced while the model parameters are reduced. The basic network structure of the driving scene multi-classification model is built by referring to Xconcept integrally, the core of the driving scene multi-classification model is a feature extraction unit consisting of a depth separable convolution structure and a residual error structure, the extraction of image basic features is realized through a plurality of stacked feature extraction units, and finally, a classifier is built through a full connection layer to finish multi-classification of driving scenes.
After the classification results of the current driving environment under the multiple reference category branches are determined through the driving scene multi-classification model, the determined classification results of the current driving environment under the multiple reference category branches can be fed back to the intelligent driving system, so that the intelligent driving system can give out corresponding driving decision instructions according to the classification results to control the driving of the vehicle.
In the driving scene classification method, a target driving image is obtained first, wherein the target driving image is a driving environment image acquired in the current driving process; then, a multi-classification result of the current driving environment corresponding to the target driving image is determined according to the target driving image through a driving scene multi-classification model, wherein the multi-classification result comprises classification results of the current driving environment under multiple reference class branches, and the driving scene multi-classification model is used for identifying the classification results of the driving image under the multiple reference class branches according to the driving image corresponding to the driving environment. Therefore, the multi-classification detection of the driving environment is realized, and the classification result of the driving environment under various reference type branches is identified, so that the actual requirement of an intelligent driving scene can be met, and the effect of assisting intelligent driving is effectively achieved.
For the driving scene multi-classification model used in the embodiment shown in fig. 2, the embodiment of the present application further provides a corresponding model training method. Referring to fig. 4 and 5, fig. 4 is a schematic flow chart of a model training method of a driving scenario multi-classification model provided in an embodiment of the present application, and fig. 5 is a schematic diagram of a training and application architecture of the driving scenario multi-classification model provided in the embodiment of the present application, as shown in fig. 4 and 5, the model training method includes the following steps:
step 401: acquiring a training sample set; the training sample set comprises a plurality of training samples, each training sample comprises a training driving image and a corresponding labeled multi-classification result, and the labeled multi-classification result comprises labeled classification results of the training driving images under the multiple reference type branches.
In practical application, driving images acquired by the vehicle-mounted camera under different scenes such as urban roads, rural roads, expressways and the like can be acquired and used as training driving images. For each acquired training driving image, labeling classification results of the training driving image under multiple reference category branches are labeled, for example, for each training driving image, labeling classification results of the training driving image under an illumination branch, a weather branch and a camera state branch may be labeled, that is, three labeling labels are configured for each training driving image, and the three labeling labels are respectively used for representing illumination conditions (such as day, dusk or night), weather conditions (such as sunny days, rainy days, foggy days or snowy days), and camera state conditions (such as normal camera states or abnormal camera states, where the normal camera states include conditions that a lens is stained or water drops, a lens is blurred, and night light overexposure affects actual imaging quality). And then, forming a training sample by using the training driving image and the labeling classification result thereof under the multiple reference class branches, and forming a training sample set by using a plurality of training samples of the same type.
It should be understood that, when all the training driving images are labeled under multiple reference category branches, it is required to ensure that the number distribution of the training driving images belonging to various classification results under each reference category branch is uniform, so as to ensure that the trained driving scene multi-classification model can accurately identify various classification results under each reference category branch.
Step 402: determining a prediction multi-classification result corresponding to the training driving image according to the training driving image in the training sample through a driving scene multi-classification model to be trained; the predicted multi-classification result comprises a predicted classification result of the training driving image under the multiple reference class branches.
And then, processing the training driving image in the training sample by using a driving scene multi-classification model to be trained to obtain a prediction multi-classification result corresponding to the training driving image, wherein the prediction multi-classification result comprises prediction classification results of the training driving image under a plurality of reference class branches.
It should be understood that the driving scenario multi-classification model to be trained herein is a training basis of the driving scenario multi-classification model introduced in the embodiment shown in fig. 2, and has the same model structure as the driving scenario multi-classification model introduced in the embodiment shown in fig. 2, but the model parameters are obtained by initial assignment. The working principle and the model structure of the driving scene multi-classification model to be trained can refer to the description contents of the working principle and the model structure of the driving scene multi-classification model in the embodiment shown in fig. 2, and are not repeated here.
In order to enable the trained driving scene multi-classification model to have better generalization, before the driving scene multi-classification model to be trained passes through, data preprocessing and/or data enhancement operation can be performed on the training driving image in the training sample, so that the target training driving image is obtained. Here, the data preprocessing includes, but is not limited to, scaling, cropping, flipping and the like of the training driving image, and the data enhancement operation includes, but is not limited to, brightness adjustment, exposure adjustment and the like of the training driving image. Correspondingly, after data preprocessing and/or data enhancement operation is completed to obtain a target training driving image, the target training driving image can be processed by using a driving scene multi-classification model to be trained to obtain a prediction multi-classification result corresponding to the target training driving image; it should be understood that the labeled multi-classification result corresponding to the target training driving image is substantially the labeled multi-classification result corresponding to the training driving image in the training sample.
Step 403: and constructing a loss function according to the labeling classification result and the prediction classification result in the training sample.
After a prediction classification result corresponding to a training driving image is generated through a driving scene multi-classification model to be trained, a loss function can be constructed according to the prediction classification result and a label classification result in a training sample, namely, the loss is calculated according to a prediction value and a label output by the driving scene multi-classification model to be trained, and a formula for specifically constructing the loss function is as follows:
wherein C is the constructed loss function; y iskTo label the classification result, akTo predict the classification result.
Step 404: and adjusting model parameters of the driving scene multi-classification model based on the loss function.
And then, with the constructed loss function convergence as a target, performing back propagation and gradient updating on the driving scene multi-classification model to be trained so as to adjust model parameters of the driving scene multi-classification model to be trained, and continuously and repeatedly executing the processes until a group of optimal model parameters are obtained, so that the driving scene multi-classification model can achieve a high accurate judgment rate on each candidate classification result under each reference class branch.
The embodiment of the present application further provides a driving scene classification device, refer to fig. 6, and fig. 6 is a schematic structural diagram of the driving scene classification device provided in the embodiment of the present application. As shown in fig. 6, the driving scene classification apparatus includes:
an image acquisition module 601, configured to acquire a target driving image; the target driving image is a driving environment image acquired in the current driving process;
an image multi-classification module 602, configured to determine, according to the target driving image, a multi-classification result of a current driving environment corresponding to the target driving image through a driving scene multi-classification model; the multi-classification result comprises a classification result of the current driving environment under a plurality of reference category branches; the driving scene multi-classification model is used for identifying the classification result of the driving image under the multiple reference category branches according to the driving image corresponding to the driving environment.
Optionally, the image multi-classification module 602 is specifically configured to:
determining a classification result of the current driving environment under an illumination branch, a classification result of the current driving environment under a weather branch and a classification result of the current driving environment under a camera state branch according to the target driving image through the driving scene multi-classification model;
candidate classification results under the illumination branch comprise day, dusk and night; candidate classification results under the weather branch comprise sunny days, rainy days, foggy days and snowy days; the candidate classification results under the camera state branch include a normal camera state and an abnormal camera state.
Optionally, the image multi-classification module 602 is specifically configured to:
determining a multi-classification result vector according to the target driving image through the driving scene multi-classification model; the multi-classification result vector comprises 9 predicted values, a first predicted value to a third predicted value in the multi-classification result vector are used for bearing classification probabilities corresponding to three candidate classification results under the illumination branch, a fourth predicted value to a seventh predicted value in the multi-classification result vector are used for bearing classification probabilities corresponding to four candidate classification results under the weather branch, and an eighth predicted value to a ninth predicted value in the multi-classification result vector are used for bearing classification probabilities corresponding to two candidate classification results under the camera state branch;
determining a classification result of the current driving environment under an illumination branch according to a first prediction value to a third prediction value in the multi-classification result vector; determining the classification result of the current driving environment under the weather branch according to the fourth predicted value to the seventh predicted value in the multi-classification result vector; and determining the classification result of the current driving environment under the camera state branch according to the eighth predicted value to the ninth predicted value in the multi-classification result vector.
Optionally, the driving scene multi-classification model includes a plurality of feature extraction units, each feature extraction unit includes a depth separable convolution structure and a residual structure, and each depth separable convolution structure includes a depth convolution structure for extracting image space features and a point convolution structure for extracting image channel features.
Optionally, the apparatus further comprises a model training module; the model training module is configured to:
acquiring a training sample set; the training sample set comprises a plurality of training samples, each training sample comprises a training driving image and a corresponding labeled multi-classification result, and the labeled multi-classification result comprises labeled classification results of the training driving images under the multiple reference category branches;
determining a prediction multi-classification result corresponding to the training driving image according to the training driving image in the training sample through a driving scene multi-classification model to be trained; the predicted multi-classification result comprises a predicted classification result of the training driving image under the multiple reference class branches;
constructing a loss function according to the labeling classification result and the prediction classification result in the training sample;
and adjusting model parameters of the driving scene multi-classification model based on the loss function.
Optionally, the model training module is further configured to:
performing data preprocessing and/or data enhancement operation on the training driving image in the training sample to obtain a target training driving image;
and determining a prediction multi-classification result corresponding to the target training driving image according to the target training driving image through a driving scene multi-classification model to be trained.
According to the driving scene classification device, multi-classification detection aiming at the driving environment is realized, and the classification result of the driving environment under various reference type branches is identified, so that the actual requirement of an intelligent driving scene can be met, and the effect of assisting intelligent driving is effectively achieved.
The embodiment of the application also provides an electronic device, which comprises a processor and a memory; wherein the memory is used for storing a computer program; the processor is used for calling the computer program stored in the memory to execute any one implementation of the driving scene classification method in the foregoing embodiments.
The embodiment of the present application further provides a computer-readable storage medium for storing a program code, where the program code is configured to execute any one implementation of the driving scenario classification method described in the foregoing embodiments.
Embodiments of the present application further provide a computer program product including instructions, which when run on a computer, cause the computer to perform any one of the implementation manners of the driving scenario classification method described in the foregoing embodiments.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing computer programs.
It should be understood that in the present application, "at least one" means one or more, "a plurality" means two or more. "and/or" for describing an association relationship of associated objects, indicating that there may be three relationships, e.g., "a and/or B" may indicate: only A, only B and both A and B are present, wherein A and B may be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of single item(s) or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.
The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.
Claims (10)
1. A method of classifying a driving scenario, the method comprising:
acquiring a target driving image; the target driving image is a driving environment image acquired in the current driving process;
determining a multi-classification result of the current driving environment corresponding to the target driving image according to the target driving image through a driving scene multi-classification model; the multi-classification result comprises a classification result of the current driving environment under a plurality of reference category branches; the driving scene multi-classification model is used for identifying the classification result of the driving image under the multiple reference category branches according to the driving image corresponding to the driving environment.
2. The method according to claim 1, wherein the determining a multi-classification result of the current driving environment corresponding to the target driving image according to the target driving image through the driving scene multi-classification model comprises:
determining a classification result of the current driving environment under an illumination branch, a classification result of the current driving environment under a weather branch and a classification result of the current driving environment under a camera state branch according to the target driving image through the driving scene multi-classification model;
candidate classification results under the illumination branch comprise day, dusk and night; candidate classification results under the weather branch comprise sunny days, rainy days, foggy days and snowy days; the candidate classification results under the camera state branch include a normal camera state and an abnormal camera state.
3. The method of claim 2, wherein the determining, according to the target driving image, the classification result of the current driving environment under the illumination branch, the classification result of the current driving environment under the weather branch, and the classification result of the current driving environment under the camera state branch by the driving scene multi-classification model comprises:
determining a multi-classification result vector according to the target driving image through the driving scene multi-classification model; the multi-classification result vector comprises 9 predicted values, a first predicted value to a third predicted value in the multi-classification result vector are used for bearing classification probabilities corresponding to three candidate classification results under the illumination branch, a fourth predicted value to a seventh predicted value in the multi-classification result vector are used for bearing classification probabilities corresponding to four candidate classification results under the weather branch, and an eighth predicted value to a ninth predicted value in the multi-classification result vector are used for bearing classification probabilities corresponding to two candidate classification results under the camera state branch;
determining a classification result of the current driving environment under an illumination branch according to a first prediction value to a third prediction value in the multi-classification result vector; determining the classification result of the current driving environment under the weather branch according to the fourth predicted value to the seventh predicted value in the multi-classification result vector; and determining the classification result of the current driving environment under the camera state branch according to the eighth predicted value to the ninth predicted value in the multi-classification result vector.
4. The method according to any one of claims 1 to 3, wherein the driving scene multi-classification model comprises a plurality of feature extraction units, the feature extraction units comprising depth separable convolution structures and residual structures, the depth separable convolution structures comprising depth convolution structures for extracting image space features and point convolution structures for extracting image channel features.
5. The method of claim 1, wherein the driving scenario multi-classification model is trained by:
acquiring a training sample set; the training sample set comprises a plurality of training samples, each training sample comprises a training driving image and a corresponding labeled multi-classification result, and the labeled multi-classification result comprises labeled classification results of the training driving images under the multiple reference category branches;
determining a prediction multi-classification result corresponding to the training driving image according to the training driving image in the training sample through a driving scene multi-classification model to be trained; the predicted multi-classification result comprises a predicted classification result of the training driving image under the multiple reference class branches;
constructing a loss function according to the labeling classification result and the prediction classification result in the training sample;
and adjusting model parameters of the driving scene multi-classification model based on the loss function.
6. The method according to claim 5, wherein before the determining the predicted multi-classification result corresponding to the training driving image according to the training driving image in the training sample by the driving scene multi-classification model to be trained, the method further comprises:
performing data preprocessing and/or data enhancement operation on the training driving image in the training sample to obtain a target training driving image;
determining a prediction multi-classification result corresponding to the training driving image according to the training driving image in the training sample through the driving scene multi-classification model to be trained, wherein the prediction multi-classification result comprises the following steps:
and determining a prediction multi-classification result corresponding to the target training driving image according to the target training driving image through a driving scene multi-classification model to be trained.
7. A driving scenario classification apparatus, characterized in that the apparatus comprises:
the image acquisition module is used for acquiring a target driving image; the target driving image is a driving environment image acquired in the current driving process;
the image multi-classification module is used for determining a multi-classification result of the current driving environment corresponding to the target driving image according to the target driving image through a driving scene multi-classification model; the multi-classification result comprises a classification result of the current driving environment under a plurality of reference category branches; the driving scene multi-classification model is used for identifying the classification result of the driving image under the multiple reference category branches according to the driving image corresponding to the driving environment.
8. The apparatus of claim 7, wherein the image multi-classification module is specifically configured to:
determining a classification result of the current driving environment under an illumination branch, a classification result of the current driving environment under a weather branch and a classification result of the current driving environment under a camera state branch according to the target driving image through the driving scene multi-classification model;
candidate classification results under the illumination branch comprise day, dusk and night; candidate classification results under the weather branch comprise sunny days, rainy days, foggy days and snowy days; the candidate classification results under the camera state branch include a normal camera state and an abnormal camera state.
9. An electronic device, characterized in that the device comprises: a processor and a memory;
the memory for storing a computer program;
the processor, configured to invoke the computer program to perform the driving scenario classification method of any of claims 1 to 6.
10. A computer-readable storage medium for storing a computer program for executing the driving scenario classification method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111458258.2A CN113989785A (en) | 2021-12-01 | 2021-12-01 | Driving scene classification method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111458258.2A CN113989785A (en) | 2021-12-01 | 2021-12-01 | Driving scene classification method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113989785A true CN113989785A (en) | 2022-01-28 |
Family
ID=79732997
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111458258.2A Pending CN113989785A (en) | 2021-12-01 | 2021-12-01 | Driving scene classification method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113989785A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115203457A (en) * | 2022-07-15 | 2022-10-18 | 小米汽车科技有限公司 | Image retrieval method, image retrieval device, vehicle, storage medium and chip |
| CN116030439A (en) * | 2023-03-30 | 2023-04-28 | 深圳海星智驾科技有限公司 | Target identification method and device, electronic equipment and storage medium |
-
2021
- 2021-12-01 CN CN202111458258.2A patent/CN113989785A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115203457A (en) * | 2022-07-15 | 2022-10-18 | 小米汽车科技有限公司 | Image retrieval method, image retrieval device, vehicle, storage medium and chip |
| CN115203457B (en) * | 2022-07-15 | 2023-11-14 | 小米汽车科技有限公司 | Image retrieval method, device, vehicle, storage medium and chip |
| CN116030439A (en) * | 2023-03-30 | 2023-04-28 | 深圳海星智驾科技有限公司 | Target identification method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111209810B (en) | Boundary frame segmentation supervision deep neural network architecture for accurately detecting pedestrians in real time through visible light and infrared images | |
| CN106599773B (en) | Deep learning image identification method and system for intelligent driving and terminal equipment | |
| CN112183788A (en) | Domain adaptive equipment operation detection system and method | |
| CN102682301B (en) | Adaptation for clear path detection with additional classifiers | |
| CN106295645B (en) | A kind of license plate character recognition method and device | |
| CN109800682B (en) | Driver attribute identification method and related product | |
| CN111767878B (en) | Deep learning-based traffic sign detection method and system in embedded device | |
| CN113989785A (en) | Driving scene classification method, device, equipment and storage medium | |
| Shashidhar et al. | Vehicle Number Plate Detection and Recognition using YOLO-V3 and OCR Method | |
| CN110599453A (en) | Panel defect detection method and device based on image fusion and equipment terminal | |
| CN111046973A (en) | Multitask detection method and device and storage medium | |
| CN115861380A (en) | End-to-end unmanned aerial vehicle visual target tracking method and device in foggy low-light scene | |
| Chen et al. | Visual depth guided image rain streaks removal via sparse coding | |
| CN111274964A (en) | Detection method for analyzing water surface pollutants based on visual saliency of unmanned aerial vehicle | |
| CN114359669A (en) | Picture analysis model adjusting method and device and computer readable storage medium | |
| CN112949453A (en) | Training method of smoke and fire detection model, smoke and fire detection method and smoke and fire detection equipment | |
| CN110334703B (en) | Ship detection and identification method in day and night image | |
| CN114742996A (en) | Image semantic segmentation method and device, electronic equipment and storage medium | |
| EP4332910A1 (en) | Behavior detection method, electronic device, and computer readable storage medium | |
| CN115984133A (en) | Image enhancement method, vehicle snapshot method, device and medium | |
| CN115187954A (en) | Image processing-based traffic sign identification method in special scene | |
| CN114419018A (en) | Image sampling method, system, device and medium | |
| CN112597825A (en) | Driving scene segmentation method and device, electronic equipment and storage medium | |
| CN113111730B (en) | Quick high-precision image blurring detection method and device | |
| CN113628206B (en) | License plate detection method, device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |