CN113221759A - Road scattering identification method and device based on anomaly detection model - Google Patents
Road scattering identification method and device based on anomaly detection model Download PDFInfo
- Publication number
- CN113221759A CN113221759A CN202110532241.0A CN202110532241A CN113221759A CN 113221759 A CN113221759 A CN 113221759A CN 202110532241 A CN202110532241 A CN 202110532241A CN 113221759 A CN113221759 A CN 113221759A
- Authority
- CN
- China
- Prior art keywords
- image
- road surface
- scattering
- road
- detection model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
Abstract
The invention provides a road scattering identification method and a device based on an abnormal detection model, wherein the road scattering identification method comprises the steps of establishing an abnormal detection model comprising a reconstruction unit and an abnormal analysis unit, and improving and training the abnormal detection model; inputting an image to be detected into a trained abnormality detection model, outputting a reconstructed image through a reconstruction unit, calculating to obtain a reconstruction error and a characteristic similarity distance, calculating to obtain an abnormality score through an abnormality analysis unit according to the reconstruction error and the characteristic similarity distance, and judging the state of the road surface in the image to be detected, wherein the road surface has the scattered object through the numerical comparison result of the abnormality score and a road scattered threshold value. The invention solves the problems that the deep learning model for detecting the road surface scattering in the prior art has high training cost and is difficult to train due to high requirements on training sample images, and the recognition rate of the road surface scattering detection is low due to poor precision and high false alarm rate of the trained model.
Description
Technical Field
The invention relates to the technical field of traffic safety prevention and control, in particular to a road scattering identification method and device based on an anomaly detection model.
Background
The highway or the national road has more vehicles and the running speed of the vehicles is high, when scattered and scattered objects are scattered on the road surface of the road, the road surface environment of the road can be influenced, the normal driving of a driver can be interfered, if the scattered and scattered objects are not found in time and are cleaned, traffic accidents are easy to happen, and great potential safety hazards are caused to the normal running of the vehicles.
In the prior art, the most efficient and intelligent method for recognizing the road surface scattering is to recognize a road scene photo by using a recognition algorithm of a deep learning model, namely, the road surface scattering detection of a road scene is made into a detection task in deep learning, but the existing deep learning model for detecting the road surface scattering has the problems of high training cost and difficult training due to high requirements on training sample images, and the trained model has poor precision and high false alarm rate, so that the recognition rate of the road surface scattering detection is very low; taking the YOLO model as an example, in the training process, a large amount of training sample images containing scattered objects need to be used, and it is very difficult to collect a large amount of such training sample images, the collection cost is very high, the number is not up to the standard, a large amount of false alarms can occur in the trained YOLO model, and the operation is very unstable.
Disclosure of Invention
The invention mainly aims to provide a road scattering identification method and device based on an abnormal detection model, so as to solve the problems that in the prior art, a deep learning model for road scattering detection has high training cost and is difficult to train due to high requirements on training sample images, and the identification rate of the road scattering detection is low due to poor precision and high false alarm rate of the trained model.
In order to achieve the above object, according to one aspect of the present invention, there is provided a road scattering identification method based on an anomaly detection model, including: step S1, establishing an abnormality detection model comprising a reconstruction unit and an abnormality analysis unit, and optimizing the reconstruction unit by adjusting the receptive field of a self-encoder of the reconstruction unit and the number of memory characteristic vectors of a memory module of the reconstruction unit so as to improve the abnormality detection model; step S2, collecting road surface frame images without scattering objects as normal sample images, constructing a training sample image set, and training an improved anomaly detection model by using the training sample image set; and step S3, collecting a road surface frame image as an image to be detected, inputting the image into the trained abnormal detection model, outputting a reconstructed image through a reconstruction unit of the abnormal detection model, calculating and acquiring a reconstruction error R and a characteristic similarity distance D according to the reconstructed image and the image to be detected by the reconstruction unit, calculating and acquiring an abnormal score S according to the reconstruction error R and the characteristic similarity distance D by an abnormal analysis unit of the abnormal detection model, and judging the state of the road surface in the image to be detected, wherein the road surface in the image to be detected has the scattering objects according to the numerical comparison result of the abnormal score S and a road scattering threshold value.
Further, the calculation formula of the abnormality score S is:
in the formula (I), the compound is shown in the specification,
to pass through the pixel value I of the image to be measuredtAnd reconstructing pixel values of the image
Calculating the obtained reconstruction error; d (q)tP) is a group q of compressed feature vectors split by the image to be measuredtCalculating the obtained feature similarity distance with a memory feature vector set P in a memory module; λ is a weight coefficient; scale is the size scaling factor; when the abnormal score S is larger than the road scattering threshold value, judging that no scattering objects exist on the road surface in the image to be detected, wherein the image to be detected is a normal image; and when the abnormal score S is less than or equal to the road scattering threshold value, judging that scattering objects exist on the road surface in the image to be detected, wherein the image to be detected is an abnormal image.
Further, the value range of the road scattering threshold is [20,25]](ii) a The value range of the weight coefficient lambda is [0, 1%](ii) a The value range of the size scaling factor is [0.5e ]6,5e6]。
Further, the calculation formula of the reconstruction error R is:
in the formula (I), the compound is shown in the specification,
is the peak signal-to-noise ratio, which is expressed by the formula
Calculating to obtain N, wherein N is the number of pixel points of the image to be detected or the reconstructed image; the calculation formula of the feature similarity distance D is as follows:
in the formula, K is a compressed characteristic vector q split from the image to be detectedtNumber of (1), PpFor the sum compressed feature vector q in the memory modulet kL2 from the nearest memory feature vector.
Furthermore, the self-encoder comprises an encoder and a decoder, the memory module is positioned between the encoder and the decoder, and the image to be detected is compressed and split into compressed feature vectors containing a plurality of implicit spaces after passing through the encoder
Compressed feature vector group q oftCompressing the feature vector
The memory characteristic vector P which is most similar to the memory characteristic vector P in the memory module after passing through the memory modulepAnd splicing to form a composite feature vector, and generating a reconstructed image after a composite feature vector group containing a plurality of composite feature vectors passes through a decoder.
Further, in step S1, adjusting the receptive field of the self-encoder of the reconstruction unit is achieved by adjusting the number of down-sampling units of the encoder and the number of up-sampling units of the decoder; and adjusting the number range of the memory characteristic vectors in the memory characteristic vector set P of the memory module of the reconstruction unit to be [10,100 ].
Furthermore, the number of down-sampling units of the encoder is equal to the number of up-sampling units of the decoder, the number of down-sampling units of the encoder and the number of up-sampling units of the decoder are both 1, and the reception field of the self-encoder of the reconstruction unit is a 14 × 14 pixel region; or the down sampling unit of the coder and the up sampling unit of the decoder are both 2, and the reception field of the self coder of the reconstruction unit is a 32 multiplied by 32 pixel area; or the down sampling unit of the coder and the up sampling unit of the decoder are both 3, and the reception field of the self coder of the reconstruction unit is a 68 multiplied by 68 pixel area; or the down sampling unit of the coder and the up sampling unit of the decoder are both 4, and the reception field of the self coder of the reconstruction unit is a 140 multiplied by 140 pixel area.
Further, the down-sampling unit of the encoder comprises two convolution layers, two nonlinear activation layers, two batch regularization layers and a maximum pooling layer; the up-sampling unit of the decoder comprises three convolutional layers, three nonlinear active layers, two batch regularization layers and one deconvolution layer.
Further, the training sample image set comprises a small-specification training set, a medium-specification training set and a large-specification training set, wherein the small-specification training set comprises one or more road surface frame images of scenes, and the number range of the road surface frame images acquired by each scene is [1,5000 ]; the medium-specification training set comprises road surface frame images of one or more scenes, and the number range of the road surface frame images collected by each scene is [15000,70000 ]; the large-scale training set comprises pavement frame images of one or more scenes, and the number of the pavement frame images collected by each scene ranges from [2000,200000 ].
Further, in step S2, when collecting the road surface frame image without the missing objects, the road surface frame image in the color state is adjusted to the gray state, and the pixel point average value of each road surface frame image is calculated, and the road surface frame image whose pixel point average value is not within the range of the screening threshold is screened out, and the range of the screening threshold is [45,60 ].
Further, in step S3, a road surface frame image is collected to construct a test image set, where the test image set includes a normal image without road surface objects and an abnormal image with road surface objects, and the road surface frame image is randomly selected from the test image set as an image to be detected and input into the trained abnormal detection model.
According to another aspect of the present invention, there is provided a road scattering identification apparatus, including an abnormality detection model obtained by training in steps S1 and S2 of the above-mentioned road scattering identification method based on the abnormality detection model; the anomaly detection model is used for executing the road scattering identification method based on the anomaly detection model.
By applying the technical scheme of the invention, the combination of the self-encoder of the reconstruction unit and the memory module is reasonably utilized through the optimization and improvement of the abnormality detection model, the image to be detected which is subsequently input as a normal sample image is reconstructed, the reconstructed image corresponding to the normal sample image is obtained, the reconstructed image generated by the normal sample image is taken as a standard basis, the abnormality detection model can reconstruct a reconstructed image with high quality, and when the image to be detected does not belong to the predefined normal sample image, the quality of the reconstructed image generated by the abnormality detection model is poor, so that the improved abnormality detection model can be accurately and rapidly identified and judged to be abnormal according to the difference of the reconstructed image.
Specifically, after the collected road surface frame image is input into the trained abnormal detection model as an image to be detected, the reconstruction unit of the abnormal detection model outputs a reconstructed image, the reconstruction unit calculates and obtains a reconstruction error R and a feature similarity distance D according to the reconstructed image and the image to be detected, and the abnormal analysis unit of the abnormal detection model calculates and obtains an abnormal score S according to the reconstruction error R and the feature similarity distance D; the reconstruction error R is the difference of the reconstructed image, the characteristic similarity distance D is the difference between the memory characteristic vector of the normal sample image corresponding to the memory module and the compressed characteristic vector group split from the image to be detected, the two parameters are integrated, and the weight is reasonably distributed to obtain the abnormal score S, so that the state of the road surface in the image to be detected with the scattered objects is accurately judged according to the numerical comparison result of the abnormal score S and the road scattered threshold value, and when the road surface in the image to be detected is judged to have the scattered objects, subsequent rapid alarm is facilitated, the road scattered objects are timely cleaned, and the driving safety of vehicles on the road is ensured. Therefore, the improved anomaly detection model has the advantages of high model precision and low false alarm rate.
Moreover, the anomaly detection task performed by the improved anomaly detection model adopts a weak supervision mode, so that the training sample images only contain the road surface frame images without scattering objects in a concentrated manner when the improved anomaly detection model is trained, the acquisition difficulty of the sample images is greatly reduced, the training difficulty of the improved anomaly detection model is reduced, the training precision of the improved anomaly detection model is improved, and the method has the advantages of high practicability and economy.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate exemplary embodiments of the invention and, together with the description, serve to explain the invention and are not intended to limit the invention. In the drawings:
FIG. 1 is a flow chart illustrating a road scattering identification method based on an anomaly detection model according to the present invention;
FIG. 2 is a schematic diagram showing a work flow of a reconstruction unit of an improved abnormal detection model in the abnormal detection model-based road scattering identification method of the invention;
fig. 3 shows a road surface frame image without any scattering objects on the acquired road surface, which can be used as a normal sample image in a training sample image set or as a normal image in a test image set, according to an alternative embodiment of the road scattering identification method based on the anomaly detection model of the present invention;
FIG. 4 is a road surface frame image of a road surface with scattering objects, which is collected as an abnormal image in a test image set, according to an alternative embodiment of the road scattering identification method based on an abnormal detection model of the present invention;
fig. 5 shows a road surface frame image capable of being used as a collected road surface with scattering objects, in which the scattering objects existing on the road surface are added according to a preset condition and can be used as an abnormal image in a test image set, according to another alternative embodiment of the road scattering identification method based on the abnormality detection model of the invention;
fig. 6 shows an AUC curve graph obtained after the improved anomaly detection model is tested by using a test image set in the method for identifying road scattering based on the anomaly detection model of the present invention, where three AUC curves correspond to the same test set, and are obtained by testing three anomaly detection models trained by using training groups of different specifications of a training sample image set.
Detailed Description
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings.
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged under appropriate circumstances in order to facilitate the description of the embodiments of the invention herein. Furthermore, the terms "comprises," "comprising," "includes," "including," "has," "having," and any variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements explicitly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
In order to solve the problems that in the prior art, a deep learning model for detecting the road surface scattering has high training cost and is difficult to train due to high requirements on training sample images, and the recognition rate of the road surface scattering detection is low due to poor precision and high false alarm rate of the trained model, the invention provides a road scattering recognition method and a road scattering recognition device based on an abnormal detection model, wherein the road scattering recognition device comprises an abnormal detection model, and the abnormal detection model is obtained by training in the steps S1 and S2 of the above or below abnormal detection model-based road scattering recognition method; the anomaly detection model is used for executing the road scattering identification method based on the anomaly detection model.
Fig. 1 is a flowchart of a road scattering identification method based on an anomaly detection model according to an embodiment of the present invention. As shown in fig. 1, the method for identifying the road scattering includes the following steps: step S1, establishing an abnormality detection model comprising a reconstruction unit and an abnormality analysis unit, and optimizing the reconstruction unit by adjusting the receptive field of a self-encoder of the reconstruction unit and the number of memory characteristic vectors of a memory module of the reconstruction unit so as to improve the abnormality detection model; step S2, collecting road surface frame images without any scattering objects as normal sample images, constructing a training sample image set, and training an improved anomaly detection model by using the training sample image set; step S3, collecting a road surface frame image as an image to be detected, inputting the image to be detected into the trained abnormal detection model, outputting a reconstructed image through a reconstruction unit of the abnormal detection model, calculating and obtaining a reconstruction error R and a characteristic similarity distance D according to the reconstructed image and the image to be detected by the reconstruction unit, calculating and obtaining an abnormal score S according to the reconstruction error R and the characteristic similarity distance D by an abnormal analysis unit of the abnormal detection model, and judging the state of the road surface in the image to be detected with the scattering object according to the numerical comparison result of the abnormal score S and the road scattering threshold value.
By optimizing and improving the abnormality detection model, reasonably utilizing the combination of the self-encoder and the memory module of the reconstruction unit, reconstructing an image to be detected which is input as a normal sample image subsequently, acquiring a reconstructed image corresponding to the normal sample image, and taking the reconstructed image generated by the normal sample image as a standard basis, the abnormality detection model can reconstruct a reconstructed image with relatively high quality, and when the image to be detected does not belong to a predefined normal sample image, the quality of the reconstructed image generated by the abnormality detection model is poor, so that the difference of the reconstructed image can enable the improved abnormality detection model to accurately and rapidly identify and judge to detect the abnormality.
Specifically, after the collected road surface frame image is input into the trained abnormal detection model as an image to be detected, the reconstruction unit of the abnormal detection model outputs a reconstructed image, the reconstruction unit calculates and obtains a reconstruction error R and a feature similarity distance D according to the reconstructed image and the image to be detected, and the abnormal analysis unit of the abnormal detection model calculates and obtains an abnormal score S according to the reconstruction error R and the feature similarity distance D; the reconstruction error R is the difference of the reconstructed image, the characteristic similarity distance D is the difference between the memory characteristic vector of the normal sample image corresponding to the memory module and the compressed characteristic vector group split from the image to be detected, the two parameters are integrated, and the weight is reasonably distributed to obtain the abnormal score S, so that the state of the road surface in the image to be detected with the scattered objects is accurately judged according to the numerical comparison result of the abnormal score S and the road scattered threshold value, and when the road surface in the image to be detected is judged to have the scattered objects, subsequent rapid alarm is facilitated, the road scattered objects are timely cleaned, and the driving safety of vehicles on the road is ensured. Therefore, the improved anomaly detection model has the advantages of high model precision and low false alarm rate.
Moreover, the anomaly detection task performed by the improved anomaly detection model adopts a weak supervision mode, so that the training sample images only contain the road surface frame images without scattering objects in a concentrated manner when the improved anomaly detection model is trained, the acquisition difficulty of the sample images is greatly reduced, the training difficulty of the improved anomaly detection model is reduced, the training precision of the improved anomaly detection model is improved, and the method has the advantages of high practicability and economy.
In an alternative embodiment of the invention, the anomaly score S is calculated by the formula:
in the formula (I), the compound is shown in the specification,
to pass through the pixel value I of the image to be measuredtAnd reconstructing pixel values of the image
Calculating the obtained reconstruction error; d (q)tP) is a group q of compressed feature vectors split by the image to be measuredtCalculating the obtained feature similarity distance with a memory feature vector set P in a memory module; λ is a weight coefficient; scale is the size scaling factor.
The detection algorithm for the image to be detected by using the numerical result of the formula (1) is as follows:
when the abnormal score S is larger than the road scattering threshold value, judging that no scattering objects exist on the road surface in the image to be detected, wherein the image to be detected is a normal image (shown in figure 3); when the abnormal score S is less than or equal to the road scattering threshold, it is determined that there is a scattering object on the road surface in the image to be measured, and the image to be measured is an abnormal image (as shown in fig. 4).
Optionally, the value range of the road scattering threshold is [20,25]](ii) a The value range of the weight coefficient lambda is [0, 1%](ii) a The value range of the size scaling factor is [0.5e ]6,5e6]。
In the preferred embodiment of the present invention, the weight coefficient λ is 0.5, which is favorable for the weight balance between the reconstruction error R and the feature similarity distance D, and the size scaling factor is e6。
In the process of calculating the abnormal score S by weighted summation of the reconstruction error R and the feature similarity distance D through setting the weight coefficient λ, weights of the reconstruction error R and the feature similarity distance D are distributed, so that the input different images to be measured are emphasized according to actual conditions, and the result of the calculated abnormal score S is ensured to have use reliability. The setting of the size scaling factor scale can ensure that the calculation values of the reconstruction error R and the feature similarity distance D are in the same order of magnitude, ensure that the weighted sum value of the reconstruction error R and the feature similarity distance D is changed and floated, and ensure that the numerical result of the abnormal score S has practical application significance.
Specifically, the calculation formula of the reconstruction error R is:
in the formula (I), the compound is shown in the specification,
is the peak signal-to-noise ratio, which is expressed by the formula
And calculating to obtain the image, wherein N is the number of pixel points of the image to be detected or the reconstructed image.
The calculation formula of the feature similarity distance D is as follows:
in the formula, K is a compressed characteristic vector q split from the image to be detectedtNumber of (1), PpFor compressing feature vectors in a memory module
L2 from the nearest memory feature vector.
Fig. 2 is a schematic view of a work flow of a reconstruction unit of an improved abnormal detection model in the abnormal detection model-based road scattering identification method of the present invention; as shown in fig. 2, the self-Encoder of the reconstruction unit includes an Encoder (Encoder) and a Decoder (Decoder), and a Memory Module (Memory Module) is located between the Encoder and the Decoder; InputI in FIG. 2tThe image to be measured as an input abnormality detection model is passed through an Encoder (Encoder)) Then compressed and split into compressed feature vectors containing a plurality of implicit spaces
Compressed feature vector group q oftCompressing the feature vector
Passing through the Memory Module (Memory Module) and the Memory feature vector P in the Memory Module (Memory Module) most similar to the Memory feature vector PpSplicing to form a composite feature vector, and generating a reconstructed image as an Output after a composite feature vector group containing a plurality of composite feature vectors passes through a Decoder (Decoder), namely Output in the image
It should be noted that, as shown in fig. 2, K split compressed feature vectors q are obtainedtAre respectively as
Where K is W × H, i.e., the width and height of the module, and each compressed feature vector q defined by the Encoder (Encoder)tAre all C, i.e. the compressed feature vector qtDigital information of the order of C is contained in this dimension. The Memory feature vector set P in the Memory Module (Memory Module) includes M Memory feature vectors, P1、P2…PMThe number of M may be related to the compressed feature vector qtThe number of the memory feature vectors is equal or different, but the length of each memory feature vector is equal to the compressed feature vector qtThe length of the two feature vectors is equal to C, so that the length of the composite feature vector formed by splicing the two feature vectors is 2C.
In improving the abnormality detection model, in step S1, the number of memory feature vectors in the memory feature vector set P of the memory modules of the reconstruction unit is adjusted to the range of [10,100 ]. In a preferred embodiment of the present invention, the number of memory feature vectors in the memory feature vector set P of the memory module of the reconstruction unit is 50. The number of the memory feature vectors is determined according to the complexity of a task scene, and in the current expressway scene, the number of 50 memory feature vectors can enable the improved anomaly detection model to achieve the optimal performance.
In improving the anomaly detection model, in step S1, adjusting the receptive field of the self-encoder of the reconstruction unit is achieved by adjusting the number of downsampling units of the encoder and the number of upsampling units of the decoder.
Optionally, the number of down-sampling units of the encoder is equal to the number of up-sampling units of the decoder, both the down-sampling units of the encoder and the up-sampling units of the decoder are 1, and the reception field of the self-encoder of the reconstruction unit is a 14 × 14 pixel region; or the down sampling unit of the coder and the up sampling unit of the decoder are both 2, and the reception field of the self coder of the reconstruction unit is a 32 multiplied by 32 pixel area; or the down sampling unit of the coder and the up sampling unit of the decoder are both 3, and the reception field of the self coder of the reconstruction unit is a 68 multiplied by 68 pixel area; or the down sampling unit of the coder and the up sampling unit of the decoder are both 4, and the reception field of the self coder of the reconstruction unit is a 140 multiplied by 140 pixel area.
In the illustrated embodiment of the invention, the field of view of the self-encoder of the reconstruction unit is a 14 × 14 pixel region. The improved anomaly detection model has excellent performance at this time.
It should be noted that the down-sampling unit of the encoder in the present invention includes two convolution layers, two nonlinear active layers, two batch regularization layers and a max-pooling layer; the up-sampling unit of the decoder comprises three convolutional layers, three nonlinear active layers, two batch regularization layers and one deconvolution layer.
In this embodiment, the training sample image set includes a small-specification training set, a medium-specification training set, and a large-specification training set when training the improved anomaly detection model, where the small-specification training set includes road frame images of one or more scenes, and the number range of the road frame images acquired by each scene is [1,5000 ]; the medium-specification training set comprises road surface frame images of one or more scenes, and the number range of the road surface frame images collected by each scene is [15000,70000 ]; the large-scale training set comprises pavement frame images of one or more scenes, and the number of the pavement frame images collected by each scene ranges from [2000,200000 ]. In this way, the interaction between test scenarios and the influence of the number of samples on the training effect of the improved anomaly detection model are avoided.
In this embodiment, the small-format training set includes road surface frame images of 3 scenes, the number of the road surface frame images collected by each scene is 5000, and 15000 road surface frame images are obtained in total; the medium-specification training set comprises pavement frame images of single scenes, and the number of the pavement frame images acquired by the single scenes is 70000; the large-specification training set comprises 100 scenes of pavement frame images, the number of the pavement frame images collected by each scene is 2000, and 200000 pavement frame images are obtained in total. The road surface frame images are normal sample images without any scattering objects. In order to ensure the accuracy of the trained and improved abnormality detection model, the above-mentioned road surface frame image needs to ensure that the time information or the location information is not displayed, and the redundant time information or the redundant location information can be removed by cutting the road surface frame image.
In addition, in the illustrated embodiment of the present invention, the acquired road surface frame images are expressway scenes, that is, required normal sample images can be collected from road surface frame images captured at different expressway gates to form a training sample image set, and preferably a close-range angle of the expressway gates. In addition, in order to make the memory module memorize more and more comprehensive characteristics of the normal sample image, when the training sample image set is made, enough road surface frame images containing vehicles of different brands and different colors are required to be collected. Moreover, the resolution of all the collected normal sample images needs to be unified, the phenomenon of proportion maladjustment of the road surface frame images is prevented, and the method can be realized by adjusting the proportion and the scaling of the road surface frame images. In the present embodiment, the pixels of the normal sample image in the road surface frame image training sample image set are uniformly adjusted to 256 × 256.
In addition, when collecting the road surface frame image without the scattering objects, the road surface frame image in the color state is adjusted to be in the gray state, the pixel point mean value of each road surface frame image is calculated, the road surface frame image of which the pixel point mean value is not in the range of the screening threshold value is screened, and the range of the screening threshold value is [45,60 ]. The method can remove the road surface frame image shot at night, and avoid the problem of poor working stability of the improved abnormity detection model trained due to unclear road surface. Optionally, the road surface frame image in the color state converts the bgr color image into a grayscale image through a color space conversion interface of opencv.
And respectively training the improved anomaly detection model of the adjusted receptive field of the self-encoder and the memory characteristic vector of the memory module through the three training groups of the manufactured training sample image set, wherein an Adam optimizer is adopted by a model optimizer, and a cosine annealing strategy is adopted to adjust the learning rate at the learning rate of 0.00002. The model was trained in 60 rounds (each round represents the transfer of all samples of the training set into the model once).
Moreover, in order to further ensure the reliability of the trained and improved abnormal detection model, a pre-trained detection model is required to filter the acquired normal sample image, so as to filter the abnormal image sample marked by the detection model (i.e. the image with the road surface scattering objects).
It should be noted that, by using the improved anomaly detection model trained by the present invention, it is possible to reliably detect a randomly input image to be detected. Of course, in order to further obtain the verification, in step S3, the road surface frame images are collected, a test image set is constructed, the test image set includes normal images without road surface scattering objects and abnormal images with road surface scattering objects, and the road surface frame images are randomly selected from the test image set as the to-be-detected images to be input into the trained abnormal detection model.
In this embodiment, the test image set is one, and 100 abnormal images randomly selected for 100 scenes and 20 normal images of the same 100 scenes are selected. The abnormal image can also be a normal image obtained by manually adding typical scattering objects, such as tires, broken stones or goods frames.
The results of testing the optimized and improved anomaly detection model using the test image set described above are shown in FIG. 6. In the figure, the abscissa represents the false positive rate, representing the proportion of the normal image recognized as the abnormal image; the ordinate is the true positive rate, representing the probability that an abnormal image is identified as a normal image. The curves in the graph represent the prediction results of the anomaly detection models under different thresholds, and specifically, the curves small, middle and large in the graph correspond to the AUC curves obtained after the improved anomaly detection model is tested by using the test image set in the small-specification training set, the middle-specification training set and the large-specification training set respectively, and the larger the area under the curves is, the better the performance of the anomaly detection model is. In this embodiment, the area of the abnormal detection model under the characteristic curve (AUC curve) of the test image set reaches 77%, which indicates that the abnormal detection model has good detection performance.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
The integrated unit in the above embodiments, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present invention may be substantially or partially contributed by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing one or more computer devices (which may be personal computers, servers, network devices, or the like) to execute all or part of the steps of the method according to the embodiments of the present invention.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the unit is only one logic function division, and there may be other divisions when the actual implementation is performed, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (12)
1. A road scattering identification method based on an anomaly detection model is characterized by comprising the following steps:
step S1, establishing an abnormality detection model comprising a reconstruction unit and an abnormality analysis unit, and optimizing the reconstruction unit by adjusting the receptive field of the self-encoder of the reconstruction unit and the number of the memory characteristic vectors of the memory module of the reconstruction unit so as to improve the abnormality detection model;
step S2, collecting road surface frame images without any scattering objects as normal sample images, constructing a training sample image set, and training the improved anomaly detection model by using the training sample image set;
step S3, collecting a road surface frame image as an image to be detected, inputting the image to be detected into the trained abnormal detection model, outputting a reconstructed image through a reconstruction unit of the abnormal detection model, calculating and acquiring a reconstruction error R and a characteristic similarity distance D according to the reconstructed image and the image to be detected by the reconstruction unit, calculating and acquiring an abnormal score S according to the reconstruction error R and the characteristic similarity distance D by an abnormal analysis unit of the abnormal detection model, and judging the state of the road surface in the image to be detected with the scattering objects according to the numerical comparison result of the abnormal score S and a road scattering threshold value.
2. The method according to claim 1, wherein the abnormal score S is calculated by the formula:
in the formula (I), the compound is shown in the specification,
to pass through the pixel value I of the image to be measuredtAnd pixel values of the reconstructed image
Calculating the obtained reconstruction error; d (q)tAnd P) is a compressed characteristic vector group q divided by the image to be detectedtCalculating the obtained feature similarity distance with a memory feature vector set P in the memory module; λ is a weight coefficient; scale is the size scaling factor;
when the abnormal score S is larger than the road scattering threshold value, judging that no scattering objects exist on the road surface in the image to be detected, wherein the image to be detected is a normal image;
and when the abnormal score S is smaller than or equal to the road scattering threshold value, judging that scattering objects exist on the road surface in the image to be detected, wherein the image to be detected is an abnormal image.
3. The method of recognizing road scattering according to claim 2,
the value range of the road scattering threshold value is [20,25 ];
the value range of the weight coefficient lambda is [0,1 ];
the value range of the size scaling factor is [0.5e ]6,5e6]。
4. The method of recognizing road scattering according to claim 2,
the calculation formula of the reconstruction error R is as follows:
in the formula (I), the compound is shown in the specification,
is the peak signal-to-noise ratio, which is expressed by the formula
Calculating to obtain N, wherein N is the number of pixel points of the image to be detected or the reconstructed image;
the calculation formula of the feature similarity distance D is as follows:
in the formula, K is a compressed characteristic vector q split from the image to be detectedtNumber of (1), PpFor and compressed feature vectors in a memory module
L2 is closest to the memory feature vector.
5. The method according to claim 1, wherein the self-encoder comprises an encoder and a decoder, the memory module is located between the encoder and the decoder, and the image to be detected is compressed and split into compressed eigenvectors including a plurality of implicit spaces after passing through the encoder
Compressed feature vector group q oftThe compressed feature vector
The memory characteristic vector P which is most similar to the memory characteristic vector P in the memory module after passing through the memory modulepAnd splicing to form a composite feature vector, and generating the reconstructed image after a composite feature vector group containing a plurality of composite feature vectors passes through the decoder.
6. The method according to claim 5, wherein in step S1, the adjusting of the receptive field of the self-encoder of the reconstruction unit is achieved by adjusting the number of down-sampling units of the encoder and the number of up-sampling units of the decoder; and adjusting the number range of the memory feature vectors in the memory feature vector set P of the memory module of the reconstruction unit to be [10,100 ].
7. The road scattering recognition method of claim 6, wherein the number of down-sampling units of the encoder is equal to the number of up-sampling units of the decoder,
the down-sampling unit of the encoder and the up-sampling unit of the decoder are both 1, and the reception field of the self-encoder of the reconstruction unit is a 14 × 14 pixel region; or
The down-sampling units of the encoder and the up-sampling units of the decoder are both 2, and the reception field of the self-encoder of the reconstruction unit is a 32 x 32 pixel area; or
The down-sampling units of the encoder and the up-sampling units of the decoder are 3, and the receptive field of the self-encoder of the reconstruction unit is a 68 x 68 pixel area; or
The down sampling unit of the encoder and the up sampling unit of the decoder are both 4, and the reception field of the self-encoder of the reconstruction unit is a 140 × 140 pixel area.
8. The method for identifying road scattering of claim 6, wherein the down-sampling unit of the encoder comprises two convolution layers, two nonlinear activation layers, two batch regularization layers and one maximum pooling layer; the up-sampling unit of the decoder comprises three convolution layers, three nonlinear activation layers, two batch regularization layers and one deconvolution layer.
9. The method of claim 1, wherein the training sample image set comprises a small-size training set, a medium-size training set, and a large-size training set, wherein,
the small-specification training set comprises road surface frame images of one or more scenes, and the number range of the road surface frame images acquired by each scene is [1,5000 ];
the medium-specification training set comprises road surface frame images of one or more scenes, and the number range of the road surface frame images acquired by each scene is [15000,70000 ];
the large-specification training set comprises pavement frame images of one or more scenes, and the number of the pavement frame images acquired by each scene ranges from [2000,200000 ].
10. The method for identifying road scattering of claim 1, wherein in step S2, when the road surface frame image without scattering is collected, the road surface frame image in a color state is adjusted to a gray state, the average value of pixels of each road surface frame image is calculated, and the road surface frame image whose average value of pixels is not within a screening threshold range is screened out, and the screening threshold range is [45,60 ].
11. The method for identifying road scattering of claim 1, wherein in step S3, a road surface frame image is collected to construct a test image set, the test image set includes a normal image without road surface scattering objects and an abnormal image with road surface scattering objects, and the road surface frame image is randomly selected from the test image set as the image to be tested and input into the trained abnormal detection model.
12. A road scattering recognition apparatus, comprising an abnormality detection model obtained by training in steps S1 and S2 of the road scattering recognition method based on the abnormality detection model according to any one of claims 1 to 11; the anomaly detection model is used for executing the road scattering identification method based on the anomaly detection model according to any one of claims 1 to 11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110532241.0A CN113221759A (en) | 2021-05-17 | 2021-05-17 | Road scattering identification method and device based on anomaly detection model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110532241.0A CN113221759A (en) | 2021-05-17 | 2021-05-17 | Road scattering identification method and device based on anomaly detection model |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113221759A true CN113221759A (en) | 2021-08-06 |
Family
ID=77092122
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110532241.0A Pending CN113221759A (en) | 2021-05-17 | 2021-05-17 | Road scattering identification method and device based on anomaly detection model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113221759A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114663742A (en) * | 2022-04-01 | 2022-06-24 | 北京优创新港科技股份有限公司 | Road surface abnormity detection method and system based on unsupervised learning |
| CN114972350A (en) * | 2022-08-01 | 2022-08-30 | 深圳市信润富联数字科技有限公司 | Method, device and equipment for detecting abnormality of mold and storage medium |
| WO2024024263A1 (en) * | 2022-07-29 | 2024-02-01 | 株式会社日立産機システム | Inspection device and method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109711280A (en) * | 2018-12-10 | 2019-05-03 | 北京工业大学 | A kind of video abnormality detection method based on ST-Unet |
| CN111340056A (en) * | 2018-12-18 | 2020-06-26 | 富士通株式会社 | Information processing method and information processing apparatus |
| CN111915691A (en) * | 2019-05-07 | 2020-11-10 | 上海科技大学 | Image processing system, method, terminal and medium based on neural network |
| CN111967571A (en) * | 2020-07-07 | 2020-11-20 | 华东交通大学 | MHMA-based anomaly detection method and equipment |
| CN112766342A (en) * | 2021-01-12 | 2021-05-07 | 安徽容知日新科技股份有限公司 | Abnormity detection method for electrical equipment |
-
2021
- 2021-05-17 CN CN202110532241.0A patent/CN113221759A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109711280A (en) * | 2018-12-10 | 2019-05-03 | 北京工业大学 | A kind of video abnormality detection method based on ST-Unet |
| CN111340056A (en) * | 2018-12-18 | 2020-06-26 | 富士通株式会社 | Information processing method and information processing apparatus |
| CN111915691A (en) * | 2019-05-07 | 2020-11-10 | 上海科技大学 | Image processing system, method, terminal and medium based on neural network |
| CN111967571A (en) * | 2020-07-07 | 2020-11-20 | 华东交通大学 | MHMA-based anomaly detection method and equipment |
| CN112766342A (en) * | 2021-01-12 | 2021-05-07 | 安徽容知日新科技股份有限公司 | Abnormity detection method for electrical equipment |
Non-Patent Citations (1)
| Title |
|---|
| HYUNJONG PARK等，: "Learning Memory-Guided Normality for Anomaly Detection", 《2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114663742A (en) * | 2022-04-01 | 2022-06-24 | 北京优创新港科技股份有限公司 | Road surface abnormity detection method and system based on unsupervised learning |
| WO2024024263A1 (en) * | 2022-07-29 | 2024-02-01 | 株式会社日立産機システム | Inspection device and method |
| CN114972350A (en) * | 2022-08-01 | 2022-08-30 | 深圳市信润富联数字科技有限公司 | Method, device and equipment for detecting abnormality of mold and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110363182B (en) | Deep learning-based lane line detection method | |
| CN113221759A (en) | Road scattering identification method and device based on anomaly detection model | |
| CN107657237A (en) | Car crass detection method and system based on deep learning | |
| CN110570613A (en) | Fence vibration intrusion positioning and mode identification method based on distributed optical fiber system | |
| CN110992683A (en) | Dynamic image perception-based intersection blind area early warning method and system | |
| CN111160249A (en) | Multi-class target detection method of optical remote sensing image based on cross-scale feature fusion | |
| CN110210350A (en) | A kind of quick parking space detection method based on deep learning | |
| CN111735524A (en) | Tire load obtaining method based on image recognition, vehicle weighing method and system | |
| CN113569756B (en) | Abnormal behavior detection and positioning method, system, terminal equipment and readable storage medium | |
| CN104680554A (en) | SURF-based compression tracing method and system | |
| CN112990314A (en) | Hyperspectral image anomaly detection method and device based on improved isolated forest algorithm | |
| CN110263836B (en) | Bad driving state identification method based on multi-feature convolutional neural network | |
| CN113724286A (en) | Method and device for detecting saliency target and computer-readable storage medium | |
| CN114596316A (en) | Road image detail capturing method based on semantic segmentation | |
| CN111563468A (en) | Driver abnormal behavior detection method based on attention of neural network | |
| CN114708532A (en) | Monitoring video quality evaluation method, system and storage medium | |
| CN116805409A (en) | Method for identifying road surface state and evaluating flatness by using driving video | |
| CN116309407A (en) | Method for detecting abnormal state of railway contact net bolt | |
| CN112232102A (en) | Building target identification method and system based on deep neural network and multitask learning | |
| CN115631457A (en) | Man-machine cooperation abnormity detection method and system in building construction monitoring video | |
| CN112507327B (en) | Weather detection and processing method based on machine learning | |
| CN115937828A (en) | Fatigue driving detection method and device and vehicle | |
| CN112380986B (en) | Vehicle type detection method, system and storage medium for resisting complex street background interference | |
| CN115019201A (en) | Weak and small target detection method based on feature refined depth network | |
| CN116524726B (en) | New energy automobile overload detection analysis method and system based on big data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20210806 |
|
| WD01 | Invention patent application deemed withdrawn after publication |