CN112434730A - GoogleNet-based video image quality abnormity classification method - Google Patents
GoogleNet-based video image quality abnormity classification method Download PDFInfo
- Publication number
- CN112434730A CN112434730A CN202011247921.XA CN202011247921A CN112434730A CN 112434730 A CN112434730 A CN 112434730A CN 202011247921 A CN202011247921 A CN 202011247921A CN 112434730 A CN112434730 A CN 112434730A
- Authority
- CN
- China
- Prior art keywords
- model
- image quality
- video image
- rate
- googlenet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000003062 neural network model Methods 0.000 claims abstract description 22
- 238000012549 training Methods 0.000 claims abstract description 18
- 238000013145 classification model Methods 0.000 claims abstract description 7
- 238000007781 pre-processing Methods 0.000 claims abstract description 7
- 238000013434 data augmentation Methods 0.000 claims abstract description 6
- 238000002372 labelling Methods 0.000 claims abstract description 5
- 238000005457 optimization Methods 0.000 claims description 7
- 230000005856 abnormality Effects 0.000 claims description 5
- 238000012544 monitoring process Methods 0.000 claims description 4
- 238000011156 evaluation Methods 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 238000012795 verification Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 5
- 230000002159 abnormal effect Effects 0.000 description 6
- 238000010606 normalization Methods 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a GoogleNet-based video image quality abnormity classification method, which relates to the technical field of video image quality processing and comprises the following steps: acquiring original data information in advance and carrying out data preprocessing, wherein the data information comprises data information labeling, data augmentation, data set splitting and tfrecrd file generation; building a neural network model and training, wherein the neural network model comprises pre-training weights for calibrating an ImageNet image data set, convolution bases for calibrating the model and a dense connection classifier; and taking the trained neural network model as a video image quality abnormity classification model and outputting a result. The method is simple, high in classification accuracy and high in speed, can detect the image quality abnormity in real time and classify the abnormity, and is convenient to operate and expand.
Description
Technical Field
The invention relates to the technical field of video image quality processing, in particular to a GoogleNet-based video image quality abnormity classification method.
Background
In the prior art, a video image acquired by a video monitoring system has a large number of moving objects, and in all the moving objects, two types of objects, namely people and vehicles, are generally taken as main attention objects. There is a clear distinction between the management requirements for these two types of objects, and there is therefore a need for classification of these two types of objects in video surveillance systems.
At present, the main part in research and development is to use a statistical training based method to classify targets. However, the method needs to collect a large number of image samples of vehicles and people, and has slow recognition speed and high requirement on computing equipment. It can be seen that in the prior art, the above-mentioned problems have severely limited the application of this type of method to object recognition
The patent CN101882217B of the invention discloses a method and a device for classifying objects of video images, which comprises the following steps: after receiving a video image, filtering foreground blocks obtained from the video image, and taking the foreground blocks meeting preset filtering conditions as moving targets; tracking the moving target by a mean iterative drift algorithm, and extracting the moving target at the position of the tracked result; after the extracted moving target is subjected to normalization processing, scanning the contour of the moving target subjected to the normalization processing to obtain a characteristic statistic value; and determining the type of the moving target according to the characteristic statistic value. The targets are classified by utilizing the contour features of the targets, so that the classification accuracy is improved; the method overcomes the defect of inaccurate wide-high ratio characteristics caused by the existing normalization method by carrying out size normalization processing on the moving target through the scaling factor, reduces the data volume of the color histogram by calculating the color histogram through joint probability distribution, but still has the problems of low accuracy and low efficiency.
An effective solution to the problems in the related art has not been proposed yet.
Disclosure of Invention
Aiming at the problems in the related art, the invention provides a video image quality abnormity classification method based on GoogleNet, which is simple in implementation method, high in classification accuracy and high in speed, can detect image quality abnormity in real time and classify the abnormity, is convenient to operate and expand, and solves the technical problems in the prior art.
The technical scheme of the invention is realized as follows:
a video image quality abnormity classification method based on GoogleNet comprises the following steps:
acquiring original data information in advance and carrying out data preprocessing, wherein the data information comprises data information labeling, data augmentation, data set splitting and tfrecrd file generation;
building a neural network model and training, wherein the neural network model comprises pre-training weights for calibrating an ImageNet image data set, convolution bases for calibrating the model and a dense connection classifier;
and taking the trained neural network model as a video image quality abnormity classification model and outputting a result.
Further, the labeling of the data information includes setting classification labels, including normal, color cast abnormal, brightness abnormal and fuzzy abnormal.
Further, the data augmentation includes mirroring, rotation, scaling, clipping, translation, gaussian noise, brightness adjustment, saturation adjustment, and contrast adjustment.
Further, the splitting the data set includes the following steps:
taking a sample data set of model fitting as a training set;
monitoring whether the model is over-fitted or not, and reserving a sample data set independently in the training process for adjusting the hyper-parameters of the model and performing primary evaluation on the capability of the model to serve as a verification set;
and evaluating the generalization capability of the model.
Further, the calibration model convolution base and dense connection classifier comprises the following steps:
running a convolution base on the data set, and saving the output into a Numpy array in the hard disk;
the saved data information is taken as input and input into a separate dense connection classifier.
Further, neural network model optimization is also included, including using a larger learning rate to quickly obtain a better solution, and gradually reducing the learning rate with the continuation of iteration, so that the model is more stable at the later stage of training, which is expressed as:
decay_learning_rate＝learning_rate×decay_rate^(global_step/decay_step)
wherein, the learning rate used in each round of optimization is the learning _ rate, the learning _ rate is the set initial learning rate, the decay _ rate is the attenuation coefficient, and the decay _ step is the attenuation speed.
The invention has the beneficial effects that:
according to the method for classifying video image quality abnormity based on GoogleNet, the original data information is obtained in advance and is subjected to data preprocessing, the neural network model is built and trained, the trained neural network model is used as the video image quality abnormity classification model and outputs the result, the method is simple to implement, high in classification accuracy and high in speed, can detect the image quality abnormity in real time and classify the abnormity, and is convenient to operate and expand.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a schematic flowchart of a GoogleNet-based video image quality anomaly classification method according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a dense connection classifier of a GoogleNet-based video image quality anomaly classification method according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a convolution basis of a GoogleNet-based video image quality anomaly classification method according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments that can be derived by one of ordinary skill in the art from the embodiments given herein are intended to be within the scope of the present invention.
According to the embodiment of the invention, a video image quality abnormity classification method based on GoogleNet is provided.
As shown in fig. 1 to fig. 3, the method for classifying video image quality abnormality based on GoogleNet according to the embodiment of the present invention includes the following steps:
step S1, acquiring original data information in advance and preprocessing the data, wherein the data information is marked, data is augmented, a data set is split, and tfrechrd files are generated;
step S2, building a neural network model and training, wherein the neural network model comprises pre-training weights for calibrating ImageNet image data sets, convolution bases for calibrating the model and a dense connection classifier;
and step S3, taking the trained neural network model as a video image quality abnormity classification model and outputting the result.
And marking data information, wherein the marking data information comprises setting classification labels, including normal, color cast abnormal, brightness abnormal and fuzzy abnormal.
Wherein the data augmentation includes mirroring, rotation, scaling, clipping, translation, Gaussian noise, brightness adjustment, saturation adjustment, and contrast adjustment.
Wherein splitting the data set comprises the following steps:
taking a sample data set of model fitting as a training set;
monitoring whether the model is over-fitted or not, and reserving a sample data set independently in the training process for adjusting the hyper-parameters of the model and performing primary evaluation on the capability of the model to serve as a verification set;
and evaluating the generalization capability of the model.
The convolutional basis and dense connection classifier of the calibration model comprises the following steps:
running a convolution base on the data set, and saving the output into a Numpy array in the hard disk;
the saved data information is taken as input and input into a separate dense connection classifier.
The method also comprises neural network model optimization, wherein the neural network model optimization comprises the steps of rapidly obtaining a better solution by using a larger learning rate, and gradually reducing the learning rate along with the continuation of iteration so that the model is more stable in the later training stage and is represented as follows:
decay_learning_rate＝learning_rate×decay_rate^(global_step/decay_step)
wherein, the learning rate used in each round of optimization is the learning _ rate, the learning _ rate is the set initial learning rate, the decay _ rate is the attenuation coefficient, and the decay _ step is the attenuation speed.
By means of the scheme, the original data information is obtained in advance and is subjected to data preprocessing, the neural network model is built and trained, the trained neural network model is used as a video image quality abnormity classification model, the result is output, the implementation method is simple, the classification accuracy is high, the speed is high, the image quality abnormity can be detected in real time, the abnormity can be classified, the operation is convenient, and the expansion is convenient.
Specifically, for the above-described generation of tfrecrd file, it is
Tfrecord is a binary file format built into TF, but tfrecrd files do not have to be generated. It has the following advantages: storing the data and the label in the same file; the memory is fully utilized, so that the copying and the moving are convenient; unifying the operations of various input files.
In addition, the convolutional neural network for image classification contains two parts: first a series of convolutional and pooling layers and finally a dense connection classifier. The first part is called the convolution base of the model. For convolutional neural networks, feature extraction is to take the convolutional basis of a previously trained network, run new data, and then train a new classifier at the output layer.
In addition, as shown in fig. 3, where InputLayer represents a convolution basis, the feature map of the convolutional neural network indicates whether a general concept exists in the image, and such a feature map may be useful in any computer vision problem. The representation of the densely connected layers does not contain position information of the object in the input image. Dense connected layers give up the notion of space, and if object position is important to the problem, the features of dense connected layers are largely useless.
After the convolution base, a dense connection classifier is added, and two methods can be selected: the convolution base is run on the data set, the output is saved as a Numpy array in the hard disk, and this data is then used as input to the independent dense join classifier. This method is fast and computationally inexpensive, but does not use data enhancement. A density layer is added on top to extend the existing model (i.e., the convolution base) and run the entire model end-to-end on the input data. Data enhancement can be used because each input image enters the model via the convolution basis, but is computationally expensive. I.e., first freeze the convolution base, add a new classifier, and then train.
In addition, when the neural network is trained by adopting a stochastic gradient descent algorithm, the performance of the final model on test data can be improved to a certain extent by using the moving average model. TF provides tf.train.exponentialmovingaverage method to implement the running average model. During initialization, a decay rate is required to control the update speed of the model. Exponental MovingAverage maintains a shadow variable for each variable, the initial value of which is the initial value of the corresponding variable. And each time the running variable is updated, the value of the shadow variable is updated as follows:
shadow_variable＝decay×shadow_variable+(1-decay)×variable
wherein, shadow _ variable is an independent variable, variable is a variable to be updated, decay is an attenuation rate, decay determines the updating speed of the model, and the larger the decay is, the more stable the model tends to. In practice, the decade is typically set to a number very close to 1, e.g., 0.999.
In summary, according to the technical scheme of the invention, the original data information is obtained in advance and is subjected to data preprocessing, the neural network model is built and trained, and the trained neural network model is used as the video image quality abnormity classification model and outputs a result.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (6)
1. A video image quality abnormity classification method based on GoogleNet is characterized by comprising the following steps:
acquiring original data information in advance and carrying out data preprocessing, wherein the data information comprises data information labeling, data augmentation, data set splitting and tfrecrd file generation;
building a neural network model and training, wherein the neural network model comprises pre-training weights for calibrating an ImageNet image data set, convolution bases for calibrating the model and a dense connection classifier;
and taking the trained neural network model as a video image quality abnormity classification model and outputting a result.
2. The GoogleNet-based video image quality abnormality classification method according to claim 1, wherein the labeling of data information includes setting classification labels including normal, color cast abnormality, brightness abnormality and blur abnormality.
3. The GoogleNet-based video image quality anomaly classification method according to claim 2, wherein the data augmentation includes mirroring, rotation, scaling, cropping, translation, gaussian noise, brightness adjustment, saturation adjustment, and contrast adjustment.
4. The GoogleNet-based video image quality anomaly classification method according to claim 1, wherein the splitting of the data set comprises the steps of:
taking a sample data set of model fitting as a training set;
monitoring whether the model is over-fitted or not, and reserving a sample data set independently in the training process for adjusting the hyper-parameters of the model and performing primary evaluation on the capability of the model to serve as a verification set;
and evaluating the generalization capability of the model.
5. The GoogleNet-based video image quality anomaly classification method according to claim 1, wherein the convolutional basis and dense connection classifier of the calibration model comprises the steps of:
running a convolution base on the data set, and saving the output into a Numpy array in the hard disk;
the saved data information is taken as input and input into a separate dense connection classifier.
6. The GoogleNet-based video image quality anomaly classification method according to claim 1, further comprising neural network model optimization, including using a larger learning rate to obtain a better solution quickly, and gradually reducing the learning rate as the iteration continues, so that the model is more stable in the late training stage, and expressed as:
decay_learning_rate＝learning_rate×decay_rate^(global_step/decay_step)
wherein, the learning rate used in each round of optimization is the learning _ rate, the learning _ rate is the set initial learning rate, the decay _ rate is the attenuation coefficient, and the decay _ step is the attenuation speed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011247921.XA CN112434730A (en) | 2020-11-10 | 2020-11-10 | GoogleNet-based video image quality abnormity classification method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011247921.XA CN112434730A (en) | 2020-11-10 | 2020-11-10 | GoogleNet-based video image quality abnormity classification method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112434730A true CN112434730A (en) | 2021-03-02 |
Family
ID=74700841
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011247921.XA Pending CN112434730A (en) | 2020-11-10 | 2020-11-10 | GoogleNet-based video image quality abnormity classification method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112434730A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113743332A (en) * | 2021-09-08 | 2021-12-03 | 中国科学院自动化研究所 | Image quality evaluation method and system based on universal vision pre-training model |
| CN113962995A (en) * | 2021-12-21 | 2022-01-21 | 北京鹰瞳科技发展股份有限公司 | Cataract model training method and cataract identification method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106709511A (en) * | 2016-12-08 | 2017-05-24 | 华中师范大学 | Urban rail transit panoramic monitoring video fault detection method based on depth learning |
| US20180032846A1 (en) * | 2016-08-01 | 2018-02-01 | Nvidia Corporation | Fusing multilayer and multimodal deep neural networks for video classification |
| CN107945182A (en) * | 2018-01-02 | 2018-04-20 | 东北农业大学 | Maize leaf disease recognition method based on convolutional neural networks model GoogleNet |
-
2020
- 2020-11-10 CN CN202011247921.XA patent/CN112434730A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180032846A1 (en) * | 2016-08-01 | 2018-02-01 | Nvidia Corporation | Fusing multilayer and multimodal deep neural networks for video classification |
| CN106709511A (en) * | 2016-12-08 | 2017-05-24 | 华中师范大学 | Urban rail transit panoramic monitoring video fault detection method based on depth learning |
| CN107945182A (en) * | 2018-01-02 | 2018-04-20 | 东北农业大学 | Maize leaf disease recognition method based on convolutional neural networks model GoogleNet |
Non-Patent Citations (2)
| Title |
|---|
| IMU_PANDADE: "深度学习（Python）-- 计算机视觉深度学习", Retrieved from the Internet <URL:https://blog.csdn.net/Pandade520/article/details/97417321> * |
| 啊扑: "神经网络笔记 ----神经网络的优化", Retrieved from the Internet <URL:https://blog.csdn.net/qq_42450404/article/details/88138507> * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113743332A (en) * | 2021-09-08 | 2021-12-03 | 中国科学院自动化研究所 | Image quality evaluation method and system based on universal vision pre-training model |
| CN113743332B (en) * | 2021-09-08 | 2022-03-25 | 中国科学院自动化研究所 | Image quality evaluation method and system based on universal vision pre-training model |
| CN113962995A (en) * | 2021-12-21 | 2022-01-21 | 北京鹰瞳科技发展股份有限公司 | Cataract model training method and cataract identification method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109086811B (en) | Multi-label image classification method and device and electronic equipment | |
| CN107133569B (en) | Monitoring video multi-granularity labeling method based on generalized multi-label learning | |
| CN111767962B (en) | One-stage target detection method, system and device based on generation countermeasure network | |
| CN110889863B (en) | Target tracking method based on target perception correlation filtering | |
| US8099373B2 (en) | Object detector trained using a working set of training data | |
| CN112232371B (en) | American license plate recognition method based on YOLOv3 and text recognition | |
| CN111461145B (en) | Method for detecting target based on convolutional neural network | |
| CN113361326B (en) | Wisdom power plant management and control system based on computer vision target detection | |
| CN114612835A (en) | Unmanned aerial vehicle target detection model based on YOLOv5 network | |
| CN116030396B (en) | Accurate segmentation method for video structured extraction | |
| CN112365497A (en) | High-speed target detection method and system based on Trident Net and Cascade-RCNN structures | |
| CN112434730A (en) | GoogleNet-based video image quality abnormity classification method | |
| CN111178438A (en) | ResNet 101-based weather type identification method | |
| CN111738054A (en) | Behavior anomaly detection method based on space-time self-encoder network and space-time CNN | |
| CN114897782B (en) | Gastric cancer pathological section image segmentation prediction method based on generation type countermeasure network | |
| CN111222546B (en) | Multi-scale fusion food image classification model training and image classification method | |
| CN115937703A (en) | Enhanced feature extraction method for remote sensing image target detection | |
| CN115100497A (en) | Robot-based method, device, equipment and medium for routing inspection of abnormal objects in channel | |
| CN112785610B (en) | Lane line semantic segmentation method integrating low-level features | |
| CN111339950B (en) | Remote sensing image target detection method | |
| CN112581483A (en) | Self-learning-based plant leaf vein segmentation method and device | |
| CN112132207A (en) | Target detection neural network construction method based on multi-branch feature mapping | |
| CN111832435A (en) | Beauty prediction method and device based on migration and weak supervision and storage medium | |
| CN116824135A (en) | Atmospheric natural environment test industrial product identification and segmentation method based on machine vision | |
| CN114283431B (en) | Text detection method based on differentiable binarization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |