CN108921013B - Visual scene recognition system and method based on deep neural network - Google Patents
Visual scene recognition system and method based on deep neural network Download PDFInfo
- Publication number
- CN108921013B CN108921013B CN201810472012.2A CN201810472012A CN108921013B CN 108921013 B CN108921013 B CN 108921013B CN 201810472012 A CN201810472012 A CN 201810472012A CN 108921013 B CN108921013 B CN 108921013B
- Authority
- CN
- China
- Prior art keywords
- scene
- network
- time
- neural network
- weather
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
- Traffic Control Systems (AREA)
Abstract
A deep neural network based visual scene recognition system, comprising: the vehicle-mounted vision system is used for acquiring a vehicle forward-looking vision image; the off-line training module is used for carrying out sample acquisition on a vehicle forward-looking visual image acquired from a vehicle-mounted visual system by utilizing a deep convolutional neural network for visual input, marking the acquired sample to generate a sample label and carrying out step-by-step training on neural network parameters; the deep convolutional neural network is composed of three-branch classification networks sharing two layers of shallow convolutional characteristics, and network parameters are trained through samples and training tasks; and the online analysis module is used for carrying out real-time scene analysis on the sample trained by the offline training module by adopting a network compression and time-sharing parallel analysis strategy and outputting the time of day, the weather and the scene abnormal state of the road scene where the vehicle-mounted vision system is located.
Description
Technical Field
The invention relates to the field of vehicle safety, in particular to a visual scene recognition system and method based on a deep neural network.
Background
Intellectualization now becomes an important development direction in the automotive industry, and sensing technologies based on visual sensors are becoming mature and increasingly widely applied in the field of active safety of vehicles. For a visual system, aiming at different road environments, corresponding adjustment can be performed from parameter and strategy levels from image acquisition to application layer algorithms, so that the method has strong application value and significance for accurately identifying scene information in visual input, and the robustness and fault tolerance of the system can be further enhanced by abnormal diagnosis of visual scenes.
Most of the existing visual systems do not have perfect basic algorithms, the existing time-of-day judgment is mostly based on a system clock, the weather judgment is based on auxiliary sensors such as rainfall and the like, and the methods and the visual system input do not have strong relevance, so that the effect is poor in some boundary conditions. The abnormal diagnosis of the visual scene is based on the block brightness statistics of the visual input, the threshold value is difficult to set, and the robustness is poor.
Disclosure of Invention
The invention aims to solve the problems that in the prior art, most of the existing time-of-day judgment is based on a system clock, weather judgment is based on auxiliary sensors such as rainfall and the like, and the method and the visual system input do not have strong correlation, so that the effect is poor in some boundary conditions, and provides a visual scene recognition system and method based on a deep neural network.
The technical scheme adopted by the invention for solving the technical problems is as follows: a deep neural network based visual scene recognition system, comprising: the vehicle-mounted vision system is used for acquiring a vehicle forward-looking vision image; the off-line training module is used for carrying out sample acquisition on a vehicle forward-looking visual image acquired from a vehicle-mounted visual system by utilizing a deep convolutional neural network for visual input, marking the acquired sample to generate a sample label and carrying out step-by-step training on neural network parameters; the deep convolutional neural network is composed of three-branch classification networks sharing two layers of shallow convolutional characteristics, and network parameters are trained through samples and training tasks; and the online analysis module is used for performing real-time scene analysis by adopting a network compression and time-sharing parallel analysis strategy and utilizing the deep convolutional neural network trained by the offline training module, and outputting the time of day, the weather and the scene abnormal state of the road scene where the vehicle-mounted vision system is located.
The invention utilizes the multitask depth convolution neural network to identify the time of day, weather and scene abnormal conditions of a visual scene, and the input has stronger correlation with the visual system, so that the configuration parameters required by bottom acquisition and upper application of the visual system are more accurately set and have stronger robustness, thereby further improving the perception capability and robustness of the visual system, and the output can provide relevant prior information such as exposure parameters, detection classifier selection, image processing threshold values and the like for image acquisition and application algorithms on one hand, and can provide common diagnosis of image input abnormity such as image blurring, lens shielding and the like for the visual system on the other hand.
Further, the deep convolutional neural network is composed of a three-branch classification network sharing two layers of shallow convolutional features, and comprises: the time-of-day identification network: the time identification network inputs shared shallow features and outputs three time classifications of day, dusk and night; weather identification network: the weather identification network inputs shared shallow features and outputs five weather classifications of sunny days, cloudy days, rainy days, snowy days and foggy days; and, a scene anomaly identification network: the scene abnormity identification network inputs shared shallow layer characteristics, and outputs five abnormal scene classifications of normal scene, scene shading, fuzzy scene, too dark scene and too exposed scene.
Further, the off-line training module comprises a sample collecting and labeling unit and a neural network parameter step-by-step training unit; the sample acquisition and labeling unit is used for acquiring a vehicle forward-looking visual image in an off-line manner, extracting a discrete time sequence training sample, and expanding the sample by utilizing space transformation; sample distribution of each category of each task is balanced, labeling is carried out, and a sample label is generated; the tasks include: identifying time of day, weather and scene abnormity; the categories include: a time of day category, a weather category, and a scene anomaly category; sample labels for the time of day category: 0-day, 1-dusk, 2-night; sample labels for weather categories: 0-sunny, 1-cloudy, 2-rainy, 3-snowy, 4-foggy; and, for the sample label of the scene anomaly category: 0-normal, 1-occlusion, 2-fuzzy, 3-too bright, 4-too dark;
the neural network parameter step-by-step training unit is used for classified task training, the cross entropy is used as a loss function, firstly, shared feature layer parameters are trained, and the contribution coefficients of all tasks to weight updating are the same, namely:
loss-1/3 × L _ time +1/3 × L _ weather +1/3 × L _ absolute; and then, solidifying the convolution parameters of the shared characteristic layer, and updating the weight coefficient of each branch network by each task according to the respective loss function.
Further, the online analysis module comprises a network compression unit and a time-sharing parallel analysis unit; the network compression unit is used for quantizing and thinning the neural network parameters obtained by the offline training, evaluating the output precision loss of the compressed network by using the test set, and determining whether to retrain the quantized model according to the precision of the quantized data; the time-sharing parallel analysis unit is used for preferentially executing each frame of scene anomaly detection, and the weather and weather detection adopts a frame separation or frame skipping execution mode; in the front-end application, a double-thread or multi-thread parallel mode is adopted, one thread executes scene abnormity identification, the other thread alternately performs time-of-day and weather identification in a time-sharing mode, and the convolutional neural network hardware acceleration unit applies priority to the scene abnormity identification network to take precedence.
The invention also provides a visual scene recognition method based on the deep neural network, which comprises the steps of collecting a forward-looking visual image of a vehicle through a vehicle-mounted visual system, carrying out off-line network training on the visual image by using the deep convolutional neural network, carrying out on-line scene analysis, and outputting the time of day, the weather and the scene abnormal state of a road scene where the vehicle-mounted visual system is located; the deep convolutional neural network is composed of three-branch classification networks sharing two layers of shallow convolutional characteristics, and network parameters are trained through samples and training tasks.
Further, the deep convolutional neural network is composed of a three-branch classification network sharing two layers of shallow convolutional features, and comprises: the time-of-day identification network: the time identification network inputs shared shallow features and outputs three time classifications of day, dusk and night; weather identification network: the weather identification network inputs shared shallow features and outputs five weather classifications of sunny days, cloudy days, rainy days, snowy days and foggy days; and, a scene anomaly identification network: the scene abnormity identification network inputs shared shallow layer characteristics, and outputs five abnormal scene classifications of normal scene, scene shading, fuzzy scene, too dark scene and too exposed scene.
Further, the offline network training comprises: collecting and labeling samples, and training neural network parameters step by step; the sample collection and labeling means: collecting a vehicle forward-looking visual image in an off-line manner, extracting a discrete time sequence training sample, and expanding the sample by utilizing spatial transformation; sample distribution of each category of each task is balanced, labeling is carried out, and a sample label is generated; the tasks include: identifying time of day, weather and scene abnormity; the categories include: a time of day category, a weather category, and a scene anomaly category; sample labels for the time of day category: 0-day, 1-dusk, 2-night; sample labels for weather categories: 0-sunny, 1-cloudy, 2-rainy, 3-snowy, 4-foggy; and, for the sample label of the scene anomaly category: 0-normal, 1-occlusion, 2-fuzzy, 3-too bright, 4-too dark; the step-by-step training of the neural network parameters comprises the following steps: the classification task training is adopted, the loss function adopts cross entropy, firstly, the shared characteristic layer parameters are trained, and the contribution coefficients of all tasks to weight updating are the same, namely:
Loss＝1/3*L_time+1/3*L_weather+1/3*L_abnormal；
and then, solidifying the convolution parameters of the shared characteristic layer, and updating the weight coefficient of each branch network by each task according to the respective loss function.
Further, the online scene analysis comprises network compression and time-sharing parallel analysis strategies; the network compression is as follows: quantizing and thinning the neural network parameters obtained by the offline training, wherein the length of quantized data bits and the thinning degree are configuration parameters; evaluating the loss of the compressed network output precision by using the test set, and determining whether to retrain the quantized model according to the quantized data precision; the time-sharing parallel analysis strategy is as follows: each frame of scene anomaly detection is preferentially executed, and the weather and weather detection adopts a frame separation or frame skipping execution mode; in the front-end application, a double-thread or multi-thread parallel mode is adopted, one thread executes scene abnormity identification, the other thread alternately performs time-of-day and weather identification in a time-sharing mode, and the convolutional neural network hardware acceleration unit applies priority to the scene abnormity identification network to take precedence.
The substantial effects of the invention are as follows: the method utilizes the deep convolutional neural network to identify the scene information of the input image, can effectively identify the environmental information of the current vehicle, including the time of day, the weather and the like, and can effectively improve the image acquisition quality of a visual system and the operation efficiency and effect of an application algorithm by optimizing the configuration of related parameters. Meanwhile, the method provided by the invention can also identify lens shading, blurring and brightness abnormity in the visual input scene, and can effectively improve the fault diagnosis capability and fault tolerance of the visual system. The network structure of the invention has extensibility, and can enrich the scene classification result by adding other classification task branches.
Drawings
FIG. 1 is a general diagram of a system architecture of the present invention;
FIG. 2 is a diagram of a multitasking scene recognition deep neural network architecture according to the present invention.
Detailed Description
The technical solution of the present invention is further specifically described below by way of specific examples in conjunction with the accompanying drawings.
The invention provides a visual scene recognition system based on a deep neural network, which is based on vehicle-mounted camera visual input, judges the driving environment of a vehicle and the scene state of a visual system, provides basic configuration information for a related algorithm, inputs the system as foresight vehicle-mounted camera input, and outputs weather conditions, time-of-day conditions, camera shielding conditions and the like, and as shown in figure 1, the visual scene recognition system comprises: the vehicle-mounted vision system is used for acquiring a vehicle forward-looking vision image; the off-line training module is used for collecting samples from a vehicle forward-looking visual image collected by the vehicle-mounted visual system, marking the samples, generating sample labels, and performing step-by-step training on neural network parameters to obtain a deep convolutional neural network; the on-line analysis module is used for carrying out real-time scene analysis by utilizing the deep convolutional neural network obtained by the training of the off-line training module based on the input of the vehicle-mounted vision system; and the output module is used for outputting the time of day, the weather and the scene abnormal state of the road scene where the vehicle-mounted vision system is located.
The off-line training module comprises a sample collecting and labeling unit and a neural network parameter step-by-step training unit. The sample acquisition and labeling unit is used for acquiring a vehicle forward-looking visual image in an off-line manner, extracting a discrete time sequence training sample, and expanding the sample by utilizing space transformation; sample distribution of each category of each task is balanced, labeling is carried out, and a sample label is generated; the tasks include: identifying time of day, weather and scene abnormity; the categories include: a time of day category, a weather category, and a scene anomaly category; sample labels for the time of day category: 0-day, 1-dusk, 2-night; sample labels for weather categories: 0-sunny, 1-cloudy, 2-rainy, 3-snowy, 4-foggy; and, for the sample label of the scene anomaly category: 0-normal, 1-occlusion, 2-fuzzy, 3-too bright, 4-too dark; the neural network parameter step-by-step training unit is used for classified task training, the cross entropy is used as a loss function, firstly, shared feature layer parameters are trained, and the contribution coefficients of all tasks to weight updating are the same, namely:
loss-1/3 × L _ time +1/3 × L _ weather +1/3 × L _ absolute; and then, solidifying the convolution parameters of the shared characteristic layer, and updating the weight coefficient of each branch network by each task according to the respective loss function.
The online analysis module comprises a network compression unit and a time-sharing parallel analysis unit; the network compression unit is used for quantizing and thinning the neural network parameters obtained by the offline training, evaluating the output precision loss of the compressed network by using the test set, and determining whether to retrain the quantized model according to the precision of the quantized data; the time-sharing parallel analysis unit is used for preferentially executing each frame of scene anomaly detection, and the weather and weather detection adopts a frame separation or frame skipping execution mode; in the front-end application, a double-thread or multi-thread parallel mode is adopted, one thread executes scene abnormity identification, the other thread alternately performs time-of-day and weather identification in a time-sharing mode, and the convolutional neural network hardware acceleration unit applies priority to the scene abnormity identification network to take precedence.
A visual scene recognition method based on a deep neural network comprises the steps that a vehicle forward-looking visual image is collected through a vehicle-mounted visual system, and offline network training is carried out on the collected vehicle forward-looking visual image to obtain a deep convolutional neural network; based on the input of the vehicle-mounted vision system, the deep convolutional neural network obtained by the off-line training module is used for carrying out on-line scene analysis, and the time of day, the weather and the scene abnormal state of the road scene where the vehicle-mounted vision system is located are output.
1. The multitask deep neural network architecture comprises the following steps: the deep convolutional neural network adopted by the invention is composed of three-branch classification networks sharing two-layer shallow convolutional characteristics, the network architecture is shown in figure 2, the branch network structures are multiplexed, different network parameters are trained through different samples and training tasks, and the specific branch network structure is as follows:
1.1 day time identification network: the time-of-day identification network inputs shared shallow features and outputs three time-of-day and time-of-day similar classifications of day, dusk and night (including tunnels).
1.2 weather recognition network: the weather identification network inputs shared shallow features and outputs five weather classifications of sunny days, cloudy days, rainy days, snowy days and foggy days.
1.3 scene abnormity identification network: the scene abnormity identification network inputs shared shallow layer characteristics, and outputs five abnormal scene classifications of normal scene, scene shading, fuzzy scene, too dark scene and too exposed scene.
2, off-line network training: collecting forward-looking driving scene data samples, marking correspondingly, and training each branch network parameter respectively by three types of recognition tasks:
2.1 sample collection and labeling: and (3) acquiring vehicle-mounted forward-looking driving scene data in an off-line manner, extracting 100000 discrete time sequence training samples, balancing the distribution of each class of sample of each task, and manually marking to generate a sample label. The label content comprises: the time of day category (0-day, 1-dusk, 2-night), the weather category (0-sunny, 1-cloudy, 2-rainy, 3-snowy, 4-foggy), and the scene anomaly category (0-normal, 1-occlusion, 2-fuzzy, 3-overly bright, 4-overly dark). Sample expansion is performed by using spatial transformation such as image color gamut, geometry and the like (if the collected samples are further expanded, the step can be omitted).
2.2 step-by-step training of network parameters: as the training is classified task, the loss function adopts cross entropy.
Firstly, training shared feature layer parameters, wherein contribution coefficients of all tasks to weight updating are the same, namely:
Loss＝1/3*L_time+1/3*L_weather+1/3*L_abnormal
and then, solidifying the convolution parameters of the shared characteristic layer, and updating the weight coefficient of each branch network by each task according to the respective loss function.
3. And (3) online scene analysis: and (3) based on vehicle-mounted visual input, performing real-time scene analysis by using the deep convolutional neural network obtained by training in the step (2), wherein the real-time scene analysis comprises network compression and time-sharing parallel analysis strategies.
3.1 network compression: and (3) quantizing (8/16 bits) and thinning (20-50%) the neural network parameters obtained by training in the step (2), wherein the length of quantized data bits and the thinning degree are configurable parameters, evaluating the loss of the network output precision after compression by using a test set, and determining whether to retrain the quantized model according to the precision of the quantized data.
3.2 time-sharing parallel analysis strategy: the scene anomaly detection priority is high, each frame is required to be executed preferentially, the updating frequency of the relative result of weather and weather detection is low, a frame separation or frame skipping execution mode can be adopted, a double-thread (or multi-thread) parallel mode can be adopted in front-end application, one thread executes scene anomaly identification, the other thread alternately performs day-time and weather identification in a time-sharing mode, and the convolutional neural network hardware acceleration unit applies for priority to identify network priority by the scene anomaly. The above-described embodiment is only a preferred embodiment of the present invention, and is not intended to limit the present invention in any way, and other variations and modifications may be made without departing from the scope of the invention as set forth in the claims.
Claims (6)
1. A visual scene recognition system based on a deep neural network, comprising:
the vehicle-mounted vision system is used for acquiring a vehicle forward-looking vision image;
the off-line training module is used for collecting samples from a vehicle forward-looking visual image collected from a vehicle-mounted visual system by using a deep convolutional neural network, marking the samples to generate sample labels, and training neural network parameters step by step; the deep convolutional neural network is composed of three-branch classification networks sharing two layers of shallow convolutional characteristics, and network parameters are trained through samples and training tasks; and the number of the first and second groups,
the online analysis module is used for performing real-time scene analysis by adopting a network compression and time-sharing parallel analysis strategy and utilizing the deep convolutional neural network trained by the offline training module, and outputting the time of day, the weather and the scene abnormal state of a road scene where the vehicle-mounted vision system is located;
the off-line training module comprises a sample acquisition and labeling unit and a neural network parameter step-by-step training unit;
the sample acquisition and labeling unit is used for acquiring a vehicle forward-looking visual image in an off-line manner, extracting a discrete time sequence training sample, and expanding the sample by utilizing space transformation; sample distribution of each category of each task is balanced, labeling is carried out, and a sample label is generated;
the tasks include: identifying time of day, weather and scene abnormity;
the categories include: a time of day category, a weather category, and a scene anomaly category;
sample labels for the time of day category: 0-day, 1-dusk, 2-night;
sample labels for weather categories: 0-sunny, 1-cloudy, 2-rainy, 3-snowy, 4-foggy; and the number of the first and second groups,
sample labels for scene anomaly categories: 0-normal, 1-occlusion, 2-fuzzy, 3-too bright, 4-too dark; the neural network parameter step-by-step training unit is used for classified task training, the cross entropy is used as a loss function, firstly, shared feature layer parameters are trained, and the contribution coefficients of all tasks to weight updating are the same, namely:
Loss＝1/3*L_time+1/3*L_weather+1/3*L_abnormal；
and then, solidifying the convolution parameters of the shared characteristic layer, and updating the weight coefficient of each branch network by each task according to the respective loss function.
2. The deep neural network-based visual scene recognition system of claim 1, wherein the deep convolutional neural network is formed by a three-branch classification network sharing two layers of shallow convolutional features, and comprises:
the time-of-day identification network: the time identification network inputs shared shallow features and outputs three time classifications of day, dusk and night;
weather identification network: the weather identification network inputs shared shallow features and outputs five weather classifications of sunny days, cloudy days, rainy days, snowy days and foggy days; and the number of the first and second groups,
scene anomaly identification network: the scene abnormity identification network inputs shared shallow layer characteristics, and outputs five abnormal scene classifications of normal scene, scene shading, fuzzy scene, too dark scene and too exposed scene.
3. The deep neural network-based visual scene recognition system of claim 1, wherein the online analysis module comprises a network compression unit and a time-sharing parallel analysis unit;
the network compression unit is used for quantizing and thinning the neural network parameters obtained by the offline training, evaluating the output precision loss of the compressed network by using the test set, and determining whether to retrain the quantized model according to the precision of the quantized data;
the time-sharing parallel analysis unit is used for preferentially executing each frame of scene anomaly detection, and the weather and weather detection adopts a frame separation or frame skipping execution mode; in the front-end application, a double-thread or multi-thread parallel mode is adopted, one thread executes scene abnormity identification, the other thread alternately performs time-of-day and weather identification in a time-sharing mode, and the convolutional neural network hardware acceleration unit applies priority to the scene abnormity identification network to take precedence.
4. A visual scene recognition method based on a deep neural network is characterized in that a vehicle forward-looking visual image is collected through a vehicle-mounted visual system, offline network training is carried out on the visual image by using the deep convolutional neural network, then online scene analysis is carried out, and the time of day, the weather and the scene abnormal state of a road scene where the vehicle-mounted visual system is located are output;
the deep convolutional neural network is composed of three-branch classification networks sharing two layers of shallow convolutional characteristics, and network parameters are trained through samples and training tasks;
the offline network training comprises: sample collection and labeling and step-by-step training of neural network parameters;
the sample collection and labeling means: collecting a vehicle forward-looking visual image in an off-line manner, extracting a discrete time sequence training sample, and expanding the sample by utilizing spatial transformation; sample distribution of each category of each task is balanced, labeling is carried out, and a sample label is generated;
the tasks include: identifying time of day, weather and scene abnormity; the categories include: a time of day category, a weather category, and a scene anomaly category;
sample labels for the time of day category: 0-day, 1-dusk, 2-night;
sample labels for weather categories: 0-sunny, 1-cloudy, 2-rainy, 3-snowy, 4-foggy; and the number of the first and second groups,
sample labels for scene anomaly categories: 0-normal, 1-occlusion, 2-fuzzy, 3-too bright, 4-too dark;
the step-by-step neural network parameter training refers to: the classification task training is adopted, the loss function adopts cross entropy, firstly, the shared characteristic layer parameters are trained, and the contribution coefficients of all tasks to weight updating are the same, namely:
Loss＝1/3*L_time+1/3*L_weather+1/3*L_abnormal；
and then, solidifying the convolution parameters of the shared characteristic layer, and updating the weight coefficient of each branch network by each task according to the respective loss function.
5. The visual scene recognition method based on the deep neural network, as claimed in claim 4, wherein the deep convolutional neural network is composed of three-branch classification networks sharing two layers of shallow convolutional features, and comprises:
the time-of-day identification network: the time identification network inputs shared shallow features and outputs three time classifications of day, dusk and night;
weather identification network: the weather identification network inputs shared shallow features and outputs five weather classifications of sunny days, cloudy days, rainy days, snowy days and foggy days; and the number of the first and second groups,
scene anomaly identification network: the scene abnormity identification network inputs shared shallow layer characteristics, and outputs five abnormal scene classifications of normal scene, scene shading, fuzzy scene, too dark scene and too exposed scene.
6. The visual scene recognition method based on the deep neural network is characterized in that the online scene analysis comprises network compression and time-sharing parallel analysis strategies;
the network compression is as follows: quantizing and thinning the neural network parameters obtained by the offline training, wherein the length of quantized data bits and the thinning degree are configuration parameters; evaluating the loss of the compressed network output precision by using the test set, and determining whether to retrain the quantized model according to the quantized data precision;
the time-sharing parallel analysis strategy is as follows: each frame of scene anomaly detection is preferentially executed, and the weather and weather detection adopts a frame separation or frame skipping execution mode; in the front-end application, a double-thread or multi-thread parallel mode is adopted, one thread executes scene abnormity identification, the other thread alternately performs time-of-day and weather identification in a time-sharing mode, and the convolutional neural network hardware acceleration unit applies priority to the scene abnormity identification network to take precedence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810472012.2A CN108921013B (en) | 2018-05-16 | 2018-05-16 | Visual scene recognition system and method based on deep neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810472012.2A CN108921013B (en) | 2018-05-16 | 2018-05-16 | Visual scene recognition system and method based on deep neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108921013A CN108921013A (en) | 2018-11-30 |
| CN108921013B true CN108921013B (en) | 2020-08-18 |
Family
ID=64402563
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810472012.2A Active CN108921013B (en) | 2018-05-16 | 2018-05-16 | Visual scene recognition system and method based on deep neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108921013B (en) |
Families Citing this family (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109858372B (en) * | 2018-12-29 | 2021-04-27 | 浙江零跑科技有限公司 | Lane-level precision automatic driving structured data analysis method |
| CN109740551A (en) * | 2019-01-09 | 2019-05-10 | 贵州宽凳智云科技有限公司 | A kind of night Lane detection method and system based on computer vision |
| CN109934096B (en) * | 2019-01-22 | 2020-12-11 | 浙江零跑科技有限公司 | Automatic driving visual perception optimization method based on characteristic time sequence correlation |
| CN109784298A (en) * | 2019-01-28 | 2019-05-21 | 南京航空航天大学 | A kind of outdoor on-fixed scene weather recognition methods based on deep learning |
| CN109766873B (en) * | 2019-02-01 | 2021-04-06 | 中国人民解放军陆军工程大学 | Pedestrian re-identification method based on hybrid deformable convolution |
| CN110135261A (en) * | 2019-04-15 | 2019-08-16 | 北京易华录信息技术股份有限公司 | A kind of method and system of trained road anomalous identification model, road anomalous identification |
| CN110119768B (en) * | 2019-04-24 | 2023-10-31 | 苏州感测通信息科技有限公司 | Visual information fusion system and method for vehicle positioning |
| CN110186471A (en) * | 2019-05-06 | 2019-08-30 | 平安科技（深圳）有限公司 | Air navigation aid, device, computer equipment and storage medium based on history video |
| CN111950572A (en) * | 2019-05-14 | 2020-11-17 | 北京字节跳动网络技术有限公司 | Method, apparatus, electronic device and computer-readable storage medium for training classifier |
| CN110543600A (en) * | 2019-09-11 | 2019-12-06 | 上海携程国际旅行社有限公司 | Search ranking method, system, device and storage medium based on neural network |
| CN111259719B (en) * | 2019-10-28 | 2023-08-25 | 浙江零跑科技股份有限公司 | Cab scene analysis method based on multi-view infrared vision system |
| CN113011216B (en) * | 2019-12-19 | 2024-04-02 | 合肥君正科技有限公司 | Multi-classification threshold self-adaptive shielding detection method |
| CN111178253B (en) * | 2019-12-27 | 2024-02-27 | 佑驾创新(北京)技术有限公司 | Visual perception method and device for automatic driving, computer equipment and storage medium |
| CN111797763A (en) * | 2020-07-02 | 2020-10-20 | 北京灵汐科技有限公司 | Scene recognition method and system |
| CN111967577B (en) * | 2020-07-29 | 2024-04-05 | 华北电力大学 | Energy Internet scene generation method based on variation self-encoder |
| CN111950656B (en) * | 2020-08-25 | 2021-06-25 | 深圳思谋信息科技有限公司 | Image recognition model generation method and device, computer equipment and storage medium |
| CN112669316B (en) * | 2021-01-29 | 2023-05-30 | 南方电网调峰调频发电有限公司 | Power production abnormality monitoring method, device, computer equipment and storage medium |
| CN113252058B (en) * | 2021-05-24 | 2024-06-28 | 北京航迹科技有限公司 | IMU data processing method, system, device and storage medium |
| CN115107832A (en) * | 2022-08-09 | 2022-09-27 | 中车唐山机车车辆有限公司 | Train headlamp control method, vehicle-mounted terminal, control system and storage medium |
| CN115631482B (en) * | 2022-11-30 | 2023-04-04 | 广汽埃安新能源汽车股份有限公司 | Driving perception information acquisition method and device, electronic equipment and readable medium |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9734425B2 (en) * | 2015-02-11 | 2017-08-15 | Qualcomm Incorporated | Environmental scene condition detection |
| CN105069472B (en) * | 2015-08-03 | 2018-07-27 | 电子科技大学 | A kind of vehicle checking method adaptive based on convolutional neural networks |
| CN105184271A (en) * | 2015-09-18 | 2015-12-23 | 苏州派瑞雷尔智能科技有限公司 | Automatic vehicle detection method based on deep learning |
| CN105539326A (en) * | 2015-12-24 | 2016-05-04 | 百利得汽车主动安全系统（苏州）有限公司 | Weather identification system and method based on vehicle-mounted camera |
| CN105575119B (en) * | 2015-12-29 | 2018-06-19 | 大连楼兰科技股份有限公司 | Road conditions weather deep learning and recognition methods and device |
| US20180040248A1 (en) * | 2016-08-04 | 2018-02-08 | Herman L. Bigham | Anti-Collision, Accident Prevention Car Safety Systems to Provide Visibility when Passing, Pulling or Backing out of Parking Spaces/Driveways and Entering or Crossing Intersections |
-
2018
- 2018-05-16 CN CN201810472012.2A patent/CN108921013B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN108921013A (en) | 2018-11-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108921013B (en) | Visual scene recognition system and method based on deep neural network | |
| CN108304813B (en) | Method for intelligently identifying state of round traffic signal lamp | |
| CN110310241B (en) | Method for defogging traffic image with large air-light value by fusing depth region segmentation | |
| CN103258332B (en) | A kind of detection method of the moving target of resisting illumination variation | |
| EP2549759B1 (en) | Method and system for facilitating color balance synchronization between a plurality of video cameras as well as method and system for obtaining object tracking between two or more video cameras | |
| Zhang et al. | Bridge damage detection using a single-stage detector and field inspection images | |
| CN110866879A (en) | Image rain removing method based on multi-density rain print perception | |
| CN114757925A (en) | Non-contact type high-voltage circuit breaker defect detection method and system | |
| CN111161160A (en) | Method and device for detecting obstacle in foggy weather, electronic equipment and storage medium | |
| CN212009589U (en) | Video identification driving vehicle track acquisition device based on deep learning | |
| Kim et al. | Deep learning based effective surveillance system for low-illumination environments | |
| CN114359196A (en) | Fog detection method and system | |
| CN114663352A (en) | High-precision detection method and system for defects of power transmission line and storage medium | |
| CN113177528A (en) | License plate recognition method and system based on multi-task learning strategy training network model | |
| CN112232226A (en) | Method and system for detecting target object through discriminant model | |
| CN113689399B (en) | Remote sensing image processing method and system for power grid identification | |
| Li et al. | Multiple linear regression haze-removal model based on dark channel prior | |
| CN116309407A (en) | Method for detecting abnormal state of railway contact net bolt | |
| CN112766174B (en) | Railway train carriage group bottom plate loss fault detection method | |
| Wang et al. | Low-light traffic objects detection for automated vehicles | |
| CN114140698A (en) | Water system information extraction algorithm based on FasterR-CNN | |
| CN111275027A (en) | Method for realizing detection and early warning processing of expressway in foggy days | |
| CN111626339A (en) | Method for detecting abnormal die cavity of injection molding machine with light shadow and jitter influence resistance | |
| CN117115097B (en) | TEDS detection method and system based on anomaly detection | |
| Sakaino et al. | DeepSnow/Rain: Light Weather Recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 310051 1st and 6th floors, no.451 Internet of things street, Binjiang District, Hangzhou City, Zhejiang Province Patentee after: Zhejiang Zero run Technology Co.,Ltd. Address before: 310051 1st and 6th floors, no.451 Internet of things street, Binjiang District, Hangzhou City, Zhejiang Province Patentee before: ZHEJIANG LEAPMOTOR TECHNOLOGY Co.,Ltd. |