CN111126248A - Method and device for identifying shielded vehicle - Google Patents
Method and device for identifying shielded vehicle Download PDFInfo
- Publication number
- CN111126248A CN111126248A CN201911327614.XA CN201911327614A CN111126248A CN 111126248 A CN111126248 A CN 111126248A CN 201911327614 A CN201911327614 A CN 201911327614A CN 111126248 A CN111126248 A CN 111126248A
- Authority
- CN
- China
- Prior art keywords
- image
- vehicle
- foreground
- module
- identified
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000013528 artificial neural network Methods 0.000 claims abstract description 22
- 238000001514 detection method Methods 0.000 claims abstract description 11
- 238000005192 partition Methods 0.000 claims abstract description 6
- 230000000877 morphologic effect Effects 0.000 claims description 24
- 238000012545 processing Methods 0.000 claims description 23
- 238000004422 calculation algorithm Methods 0.000 claims description 21
- 238000004364 calculation method Methods 0.000 claims description 20
- 238000004590 computer program Methods 0.000 claims description 19
- 230000011218 segmentation Effects 0.000 claims description 18
- 230000000873 masking effect Effects 0.000 claims description 8
- 230000003628 erosive effect Effects 0.000 claims description 6
- 238000009499 grossing Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 4
- 238000011176 pooling Methods 0.000 claims description 4
- 230000010339 dilation Effects 0.000 claims description 3
- 238000012821 model calculation Methods 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 10
- 230000008569 process Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000010191 image analysis Methods 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Biomedical Technology (AREA)
- Evolutionary Biology (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method and a device for identifying a shielded vehicle, wherein the method comprises the following steps: obtaining a first mask image of a vehicle region in an image to be identified through object foreground detection, and obtaining a second mask image of the vehicle region through the first mask image; subtracting the first mask map from the second mask map to obtain a non-vehicle blank map; acquiring inner vertexes of the plurality of first areas; dividing continuous areas in the first mask image into a plurality of second areas by using partition lines constructed by taking the inner vertexes as end points, and constructing a plurality of rectangular frames according to partition lines formed by outer vertexes and adjacent inner vertexes of the plurality of second areas; acquiring a corresponding single vehicle image; and inputting the single vehicle image into a trained recognition neural network for recognition to obtain the classification information of the vehicle. The method can effectively solve the problem of identifying the shielded vehicle, has small model calculation amount, and can be used for identifying the shielded vehicle in real time.
Description
Technical Field
The invention relates to the field of vehicle identification, in particular to a method and a device for identifying a vehicle under shielding, terminal equipment and a computer readable medium.
Background
With the development of computer vision technology, vehicle type identification through monitoring and snapping vehicle images has become an important component of traffic intelligent management, and has been widely researched. However, sometimes, the vehicle in the captured image is shielded by other vehicles or other objects, so that a single vehicle image cannot be segmented, and the vehicle identification accuracy is reduced. Most of the existing methods for processing the vehicle shielding are based on complex models, the calculation complexity is high, the calculation cost is high, and the practical application of the methods is severely restricted.
Disclosure of Invention
In view of the above, the present invention provides a method, an apparatus, a terminal device and a computer readable medium for identifying a vehicle under occlusion, which can improve the accuracy of vehicle identification and solve the problems in the prior art that a vehicle under occlusion cannot segment a single vehicle image, or that the segmentation computation complexity is high and the computation cost is high.
The first aspect of the embodiment of the invention provides a method for identifying a vehicle under occlusion, which comprises the following steps:
acquiring a first mask image of a vehicle region in an image to be identified through object foreground detection, and acquiring a second mask image of the vehicle region by connecting the outer edge and the outer vertex of each continuous region in the first mask image;
subtracting the first mask map from the second mask map to obtain a non-vehicle blank map, wherein the non-vehicle blank map comprises a plurality of first areas;
acquiring inner vertexes of the plurality of first areas;
dividing continuous areas in the first mask image into a plurality of second areas by using partition lines constructed by taking the inner vertexes as end points, and constructing a plurality of rectangular frames according to partition lines formed by outer vertexes and adjacent inner vertexes of the plurality of second areas;
acquiring a single vehicle image corresponding to the positions of the plurality of rectangular frames in the image to be identified;
and inputting the single vehicle image into a trained recognition neural network for recognition to obtain the classification information of the vehicle.
Further, acquiring a first mask map of a vehicle region in the image to be identified through object foreground detection, including:
acquiring a video stream of an image to be identified, and extracting a foreground image from the video stream by adopting a Codebook algorithm;
performing morphological operations on the foreground image, wherein the morphological operations comprise dilation and erosion;
and acquiring a contour of the foreground image after morphological operation by adopting a Canny algorithm, wherein an area in the contour is a first mask image of a vehicle area.
Further, the extracting foreground images from the video stream by using a Codebook algorithm includes:
extracting an image to be identified from the video stream, and calculating the global average gray value of the image to be identified;
and searching the corresponding code book according to the global average gray value, if the searching is successful, all pixels of the image to be recognized are foreground, if the searching is failed, traversing each code element of each pixel of the image to be recognized, if the code element matched with the current pixel value exists, the current pixel is background, otherwise, the current pixel is foreground.
Further, the contour of the foreground image after morphological operation is obtained by adopting a Canny algorithm, and the contour obtaining method comprises the following steps:
smoothing the foreground image using gaussian filtering;
removing the noise of the foreground image by applying Gaussian blur, and reducing the identification of a false edge;
calculating the gradient amplitude and the gradient direction of the foreground image, wherein the calculation formula of a gradient model G is as follows:
in the formula: gx is the difference in the horizontal direction, Gy is the difference in the vertical direction;
the calculation formula of the gradient direction θ is as follows:
θ＝atan2(Gy,Gx)。
traversing pixel points of the fuzzy part of the contour curve in the foreground image, comparing the gradient strength of the current pixel point with the gradient strength in the positive and negative gradient directions, if the gradient strength of the current pixel point is greater than that of other points in the same direction, keeping the value of the current pixel point, otherwise, setting the value to be 0, and splicing all the kept pixel points into a contour.
Further, the recognition neural network comprises a convolution module, a self-coding module and a recognition module;
the convolution module comprises N sub-convolution modules which are connected in parallel, the sub-convolution modules comprise convolution layers and a maximum pooling layer, and the convolution module is used for extracting feature information of a single vehicle image in the N sub-convolution modules respectively; the self-coding module comprises 2 layers of convolution layers and is used for fusing the characteristic information extracted by the N sub-convolution modules; the identification module is used for identifying the vehicle according to the extracted characteristic information.
A second aspect of an embodiment of the present invention provides an obscured vehicle identification device, including:
the first acquisition module is used for acquiring a first mask image of a vehicle region in an image to be identified through object foreground detection, and acquiring a second mask image of the vehicle region by connecting the outer edge and the outer vertex of each continuous region in the first mask image;
the first processing module is used for subtracting the first masking map from the second masking map to obtain a non-vehicle blank map, and the non-vehicle blank map comprises a plurality of first areas;
the second acquisition module is used for acquiring inner vertexes of the plurality of first areas;
the segmentation module is used for dividing continuous areas in the first mask image into a plurality of second areas by using the segmentation lines constructed by taking the inner vertexes as end points, and constructing a plurality of rectangular frames according to the segmentation lines formed by the outer vertexes and the adjacent inner vertexes of the plurality of second areas;
the third acquisition module is used for acquiring a single vehicle image corresponding to the positions of the plurality of rectangular frames in the image to be identified;
and the classification identification module is used for inputting the single vehicle image into a trained identification neural network for identification to obtain the classification information of the vehicle.
Further, the first obtaining module comprises:
the foreground obtaining module is used for obtaining a video stream of an image to be identified and extracting a foreground image from the video stream by adopting a Codebook algorithm;
the morphological operation module is used for performing morphological operation on the foreground image, and the morphological operation comprises expansion and erosion;
and the contour acquisition module is used for acquiring a contour of the foreground image after morphological operation by adopting a Canny algorithm, wherein an area in the contour is a first mask image of a vehicle area.
Further, the foreground obtaining module comprises:
the gray level calculation module is used for extracting an image to be identified from the video stream and calculating the global average gray level value of the image to be identified;
and the second processing module is used for searching the corresponding code book according to the global average gray value, if the searching is successful, all pixels of the image to be identified are foreground, if the searching is failed, each coding element of each pixel of the image to be identified is traversed, if a coding element matched with the current pixel value exists, the current pixel is background, and otherwise, the current pixel is foreground.
A third aspect of the embodiments of the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the method for identifying a vehicle under occlusion described above when executing the computer program.
A fourth aspect of the embodiments of the present invention provides a computer-readable medium, which stores a computer program that, when being processed and executed, implements the steps of the method for identifying a vehicle under occlusion described above.
In the embodiment of the invention, the non-vehicle blank map is obtained by subtracting the first mask map from the second mask map; the segmentation lines are obtained through the inner vertexes of the plurality of first areas in the non-vehicle blank picture, so that the segmentation of the shielded vehicle picture is realized, and the problem that the shielded vehicle cannot be identified or the identification accuracy is low due to the fact that a single vehicle image cannot be segmented in the prior art is effectively solved. Meanwhile, by adopting the light-weight recognition neural network, the problem of vehicle recognition can be processed in real time by a model based on sparse coding and self-coding, and the problems of complex model, high calculation complexity and high calculation cost of the method for processing vehicle shielding in the prior art are solved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
FIG. 1 is a flow chart of a method for identifying an occluded vehicle according to an embodiment of the present invention;
FIG. 2 is a schematic processing diagram of a method for identifying an occluded vehicle according to an embodiment of the present invention;
FIG. 3 is another schematic diagram of a processing procedure of an under-occlusion vehicle identification method according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a neural network for recognition provided by an embodiment of the present invention;
FIG. 5 is a schematic structural diagram of an obscured vehicle identification device according to an embodiment of the present invention;
FIG. 6 is a detailed block diagram of the first acquisition module of FIG. 5;
FIG. 7 is a detailed block diagram of the foreground acquisition module of FIG. 5;
fig. 8 is a schematic diagram of a terminal device according to an embodiment of the present invention.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
In order to explain the technical means of the present invention, the following description will be given by way of specific examples.
Referring to fig. 1, fig. 1 is a flowchart of a method for identifying a vehicle under occlusion according to an embodiment of the present invention. As shown in fig. 1, the method for identifying a vehicle under occlusion according to the present embodiment includes the following steps:
step S102, a first mask image of a vehicle region in the image to be identified is obtained through object foreground detection, and a second mask image of the vehicle region is obtained through connecting the outer edge and the outer vertex of each continuous region in the first mask image.
And step S104, subtracting the first mask map from the second mask map to obtain a non-vehicle blank map, wherein the non-vehicle blank map comprises a plurality of first areas.
Step S106, obtaining inner vertexes of a plurality of first areas.
Step S108, dividing the continuous area in the first mask image into a plurality of second areas by the dividing lines constructed by taking the inner vertexes as the end points, and constructing a plurality of rectangular frames according to the dividing lines constructed by the outer vertexes and the adjacent inner vertexes of the plurality of second areas.
Step S110, obtaining a single vehicle image corresponding to the positions of the plurality of rectangular frames in the image to be identified.
And step S112, inputting the single vehicle image into the trained recognition neural network for recognition, and obtaining the classification information of the vehicle.
Referring to fig. 2 and 3, fig. 2 and 3 are schematic processing diagrams of a method for identifying a vehicle under occlusion according to an embodiment of the present invention. The method includes the steps that M is a first masking image, M 'is a second masking image, an area of the first masking image M is removed from the second masking image M', a non-vehicle blank image D is obtained, the non-vehicle blank image D comprises a plurality of discontinuous first areas, and inner vertexes of the plurality of first areas are obtained, as shown in fig. 2 and 3, 2 inner vertexes are arranged in fig. 2, 4 inner vertexes are arranged in fig. 3, and the inner vertexes of the first areas are connected, so that connecting lines are dividing lines for shielding vehicles in an image to be identified; overlapping the dividing line with the first mask map M to obtain the division of the first mask map M of the vehicle area, namely, the continuous area positioned on the same side of the dividing line in the first mask map M is a single vehicle area; and determining a rectangular frame according to the two end points of the dividing line and the outer vertex of the continuous area on the same side of the dividing line, wherein the rectangular frame is the minimum rectangle comprising the dividing line and the outer vertex of the continuous area in the first mask image M on the same side of the dividing line, and the image corresponding to the position of the rectangular frame is a single vehicle image.
When the first mask map M has only one dividing line, the first mask map M may be divided into 2 individual vehicle regions, as shown in fig. 2, the dividing line formed by 2 inner vertices may divide the image to be recognized into 2 individual vehicle regions up and down; when there are a plurality of dividing lines in the first mask map M, the continuous area included in the small corner area of the adjacent 2 dividing lines is a single vehicle area, and as shown in fig. 3, the image to be recognized may be divided into 3 separate vehicle areas.
In the embodiment of the invention, the non-vehicle blank map is obtained by subtracting the first mask map from the second mask map; the segmentation lines are obtained through the inner vertexes of the plurality of first areas in the non-vehicle blank picture, so that the segmentation of the shielded vehicle picture is realized, and the problem that the shielded vehicle cannot be identified or the identification accuracy is low due to the fact that a single vehicle image cannot be segmented in the prior art is effectively solved. Meanwhile, by adopting the light-weight recognition neural network, the problem of vehicle recognition can be processed in real time by a model based on sparse coding and self-coding, and the problems of complex model, high calculation complexity and high calculation cost of the method for processing vehicle shielding in the prior art are solved.
Further, the step S102 of obtaining the first mask map of the vehicle region in the image to be recognized through object foreground detection includes:
step S1021, acquiring a video stream of an image to be identified, and extracting a foreground image from the video stream by adopting a Codebook algorithm;
step S1022, performing morphological operation on the foreground image, wherein the morphological operation comprises expansion and erosion;
and S1023, acquiring a contour of the foreground image subjected to the morphological operation by adopting a Canny algorithm, wherein an area in the contour is a first mask image of a vehicle area.
The Codebook algorithm is to create a Codebook for any pixel of an image, the Codebook is composed of a plurality of coding elements, the upper and lower limits of the box of each coding element accommodate small amplitude changes of pixel values, and the plurality of coding elements accommodate a plurality of possibilities of pixels.
Further, in step S1021, extracting a foreground image from the video stream by using a Codebook algorithm, including:
step A, extracting an image to be identified from a video stream, and calculating the global average gray value of the image to be identified;
and B, searching the corresponding code book according to the global average gray value, if the search is successful, all pixels of the image to be recognized are foreground, if the search is failed, traversing each code element of each pixel of the image to be recognized, if code elements matched with the current pixel value exist, the current pixel is background, and otherwise, the current pixel is foreground.
Further, in step S1023, a Canny algorithm is used to obtain the contour of the foreground image after morphological operation, including:
step 1, smoothing a foreground image by using Gaussian filtering;
step 2, removing the noise of the foreground image by applying Gaussian blur, and reducing the identification of a false edge;
and 3, calculating the gradient amplitude and the gradient direction of the foreground image, wherein the calculation formula of a gradient model G is as follows:
in the formula: gx is the difference in the horizontal direction, Gy is the difference in the vertical direction;
the calculation formula of the gradient direction θ is as follows:
θ＝atan2(Gy,Gx)。
and 4, traversing pixel points of the fuzzy part of the contour curve in the foreground image, comparing the gradient strength of the current pixel point with the gradient strength in the positive and negative gradient directions, if the gradient strength of the current pixel point is greater than that of other points in the same direction, keeping the value of the current pixel point, otherwise, setting the value to be 0, and splicing all the kept pixel points into the contour.
The edge has only one exact dot width by suppressing all other gradient values by using non-maximum suppression to help preserve local maximum gradients.
As shown in fig. 4, the identifying neural network may include a convolution module, a self-encoding module and an identifying module;
the convolution module comprises N sub-convolution modules which are connected in parallel, a sparse coding mode is adopted, the sub-convolution modules comprise convolution layers and a maximum pooling layer, and the images are respectively extracted from the N sub-convolution modules;
the self-coding module comprises 2 layers of convolution layers and is used for fusing the feature information extracted by the N sub-convolution modules and reducing information loss caused by convolution feature extraction;
the identification module is used for identifying the vehicle according to the extracted characteristic information.
Compared with other neural network models, the recognition network model adopts a lightweight structural design, the image analysis time can reach 20ms per image, and the real-time processing of the images can be realized.
The loss function of the identified neural network is
L＝λL1+(1-λ)L2
Wherein the content of the first and second substances,
xifeature vector, z, representing randomly sampled images from a training setiIs xiIs given as a regular parameter, gamma is a basis vector parameter, B is a basis vector (dictionary), and epsilon is a hyperparameter.
h＝ReLU(W0v+b0)
V is the input to the present layer and,
is the input of the next layer, W0、
Are encoder and decoder parameters, b0、
Is an offset value, ReLU (·) denotes activation with a ReLU function.
Referring to fig. 5, fig. 5 is a block diagram of a hidden vehicle identification apparatus according to an embodiment of the present invention. As shown in fig. 5, the occluded vehicle identification 20 of the present embodiment includes a first acquisition module 202, a first processing module 204, a second acquisition module 206, a segmentation module 208, a third acquisition module 210, and a classification identification module 212. The first obtaining module 202, the first processing module 204, the second obtaining module 206, the segmentation module 208, the third obtaining module 210, and the classification identifying module 212 are respectively configured to perform specific methods in S102, S104, S106, S108, S110, and S112 in fig. 1, and details can be referred to in the related description of fig. 1, which is only briefly described here:
a first obtaining module 202, configured to obtain a first mask map of a vehicle region in an image to be identified through object foreground detection, and obtain a second mask map of the vehicle region by connecting outer edges and outer vertices of each continuous region in the first mask map;
the first processing module 204 is configured to subtract the first mask map from the second mask map to obtain a non-vehicle blank map, where the non-vehicle blank map includes a plurality of first areas;
a second obtaining module 206, configured to obtain inner vertices of the plurality of first regions;
a dividing module 208, configured to divide the continuous region in the first mask map into a plurality of second regions by using dividing lines constructed by using the inner vertices as end points, and construct a plurality of rectangular frames according to dividing lines constructed by outer vertices and adjacent inner vertices of the plurality of second regions;
the third obtaining module 210 obtains a single vehicle image corresponding to each of the positions of the plurality of rectangular frames in the image to be recognized;
and the classification identification module 212 is used for inputting the single vehicle image into the trained identification neural network for identification to obtain the classification information of the vehicle.
Further, referring to fig. 6, the first obtaining module 202 may include:
the foreground obtaining module 2022 is configured to obtain a video stream of the image to be identified, and extract a foreground image from the video stream by using a Codebook algorithm;
a morphological operation module 2024, configured to perform morphological operations on the foreground image, where the morphological operations include dilation and erosion;
the contour obtaining module 2026 is configured to obtain a contour from the foreground image after the morphological operation by using a Canny algorithm, where an area in the contour is a first mask map of the vehicle area.
The Codebook algorithm is to create a Codebook for any pixel of an image, the Codebook is composed of a plurality of coding elements, the upper and lower limits of the box of each coding element accommodate small amplitude changes of pixel values, and the plurality of coding elements accommodate a plurality of possibilities of pixels.
Further, as can be seen in fig. 7, the foreground obtaining module 2022 may include:
the gray level calculation module 2021 is configured to extract an image to be identified from the video stream, and calculate a global average gray level of the image to be identified;
the second processing module 2023 is configured to search for a corresponding codebook according to the global average gray value, if the search is successful, all pixels of the image to be identified are foreground, if the search is unsuccessful, each coding element of each pixel of the image to be identified is traversed, if a coding element matching the current pixel value exists, the current pixel is a background, and otherwise, the current pixel is a foreground.
Further, the contour acquisition module may include:
a smoothing module for smoothing the foreground image using gaussian filtering;
the noise removal module is used for removing the noise of the foreground image by applying Gaussian blur and reducing the identification of a false edge;
the gradient calculation module is used for calculating the gradient amplitude and the gradient direction of the foreground image, wherein the calculation formula of the gradient model G is as follows:
in the formula: gx is the difference in the horizontal direction, Gy is the difference in the vertical direction;
the calculation formula of the gradient direction θ is as follows:
θ＝atan2(Gy,Gx)。
and the contour splicing module is used for traversing pixel points of the fuzzy part of the contour curve in the foreground image, comparing the gradient strength of the current pixel points with the gradient strength in the positive and negative gradient directions, if the gradient strength of the current pixel points is greater than that of other points in the same direction, keeping the value of the current pixel points, otherwise, setting the value to be 0, and splicing all the kept pixel points into a contour.
The edge has only one exact dot width by suppressing all other gradient values by using non-maximum suppression to help preserve local maximum gradients.
The neural network identification device comprises a neural network identification module, a neural network identification module and a neural network identification module, wherein the neural network identification module can comprise a convolution module, a self-coding module and an identification module;
the convolution module comprises N sub-convolution modules which are connected in parallel, a sparse coding mode is adopted, the sub-convolution modules comprise convolution layers and a maximum pooling layer, and the images are respectively extracted from the N sub-convolution modules;
the self-coding module comprises 2 layers of convolution layers and is used for fusing the feature information extracted by the N sub-convolution modules and reducing information loss caused by convolution feature extraction;
the identification module is used for identifying the vehicle according to the extracted characteristic information.
Compared with other neural network models, the recognition network model adopts a lightweight structural design, the image analysis time can reach 20ms per image, and the real-time processing of the images can be realized.
The loss function of the identified neural network is
L＝λL1+(1-λ)L2
Wherein the content of the first and second substances,
xifeature vector, z, representing randomly sampled images from a training setiIs xiIs given as a regular parameter, gamma is a basis vector parameter, B is a basis vector (dictionary), and epsilon is a hyperparameter.
h＝ReLU(W0v+b0)
V is the input to the present layer and,
is the input of the next layer, W0、
Are encoder and decoder parameters, b0、
Is an offset value, ReLU (·) denotes activation with a ReLU function.
In the embodiment of the invention, the first processing module 204 subtracts the second mask map from the first mask map to obtain a non-vehicle blank map; the segmentation lines are obtained through the inner vertexes of the plurality of first areas in the non-vehicle blank picture, so that the segmentation of the shielded vehicle picture is realized, and the problem that the shielded vehicle cannot be identified or the identification accuracy is low due to the fact that a single vehicle image cannot be segmented in the prior art is effectively solved. Meanwhile, the light recognition neural network adopted by the classification recognition module 212 can process the vehicle recognition problem in real time based on the model of sparse coding and self-coding, and the problems of complex model, high calculation complexity and high calculation cost of the vehicle shielding processing method in the prior art are solved.
Fig. 8 is a schematic diagram of a terminal device according to an embodiment of the present invention. As shown in fig. 8, the terminal device 10 of this embodiment includes: a processor 100, a memory 101 and a computer program 102 stored in said memory 101 and executable on said processor 100, such as a program for performing identification of vehicles under occlusion. The processor 100, when executing the computer program 102, implements the steps in the above-described method embodiments, e.g., the steps of S102, S104, S106, S108, S110, and S112 shown in fig. 1. Alternatively, the processor 100, when executing the computer program 102, implements the functions of the modules/units in the above-mentioned device embodiments, such as the functions of the first obtaining module 202, the first processing module 204, the second obtaining module 206, the dividing module 208, the third obtaining module 210, and the classification identifying module 212 shown in fig. 5.
Illustratively, the computer program 102 may be partitioned into one or more modules/units that are stored in the memory 101 and executed by the processor 100 to implement the present invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program 102 in the terminal device 10. For example, the computer program 102 may be partitioned into a first acquisition module 202, a first processing module 204, a second acquisition module 206, a partitioning module 208, a third acquisition module 210, and a classification identification module 212. (modules in the virtual device), the specific functions of each module are as follows:
a first obtaining module 202, configured to obtain a first mask map of a vehicle region in an image to be identified through object foreground detection, and obtain a second mask map of the vehicle region by connecting outer edges and outer vertices of each continuous region in the first mask map;
the first processing module 204 is configured to subtract the first mask map from the second mask map to obtain a non-vehicle blank map, where the non-vehicle blank map includes a plurality of first areas;
a second obtaining module 208, configured to obtain inner vertices of the plurality of first regions;
the segmentation module is used for dividing continuous areas in the first mask image into a plurality of second areas by using segmentation lines constructed by using the inner vertexes as end points, and constructing a plurality of rectangular frames according to segmentation lines constructed by the outer vertexes and the adjacent inner vertexes of the plurality of second areas;
the third obtaining module 210 obtains a single vehicle image corresponding to each of the positions of the plurality of rectangular frames in the image to be recognized;
and the classification identification module 212 is used for inputting the single vehicle image into the trained identification neural network for identification to obtain the classification information of the vehicle.
The terminal device 10 may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. Terminal device 10 may include, but is not limited to, a processor 100, a memory 101. Those skilled in the art will appreciate that fig. 8 is merely an example of a terminal device 10 and does not constitute a limitation of terminal device 10 and may include more or fewer components than shown, or some components may be combined, or different components, e.g., the terminal device may also include input-output devices, network access devices, buses, etc.
The Processor 100 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 101 may be an internal storage unit of the terminal device 10, such as a hard disk or a memory of the terminal device 10. The memory 101 may also be an external storage device of the terminal device 10, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the terminal device 10. Further, the memory 101 may also include both an internal storage unit of the terminal device 10 and an external storage device. The memory 101 is used for storing the computer program and other programs and data required by the terminal device 10. The memory 101 may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.
Claims (10)
1. A method for identifying a vehicle under occlusion, comprising:
acquiring a first mask image of a vehicle region in an image to be identified through object foreground detection, and acquiring a second mask image of the vehicle region by connecting the outer edge and the outer vertex of each continuous region in the first mask image;
subtracting the first mask map from the second mask map to obtain a non-vehicle blank map, wherein the non-vehicle blank map comprises a plurality of first areas;
acquiring inner vertexes of the plurality of first areas;
dividing continuous areas in the first mask image into a plurality of second areas by using partition lines constructed by taking the inner vertexes as end points, and constructing a plurality of rectangular frames according to partition lines formed by outer vertexes and adjacent inner vertexes of the plurality of second areas;
acquiring a single vehicle image corresponding to the positions of the plurality of rectangular frames in the image to be identified;
and inputting the single vehicle image into a trained recognition neural network for recognition to obtain the classification information of the vehicle.
2. The method for identifying the shielded vehicle according to claim 1, wherein the step of obtaining the first mask map of the vehicle area in the image to be identified through object foreground detection comprises the steps of:
acquiring a video stream of an image to be identified, and extracting a foreground image from the video stream by adopting a Codebook algorithm;
performing morphological operations on the foreground image, wherein the morphological operations comprise dilation and erosion;
and acquiring a contour of the foreground image after morphological operation by adopting a Canny algorithm, wherein an area in the contour is a first mask image of a vehicle area.
3. The method for identifying vehicles under occlusion of claim 2, wherein the extracting foreground images from the video stream using a Codebook algorithm comprises:
extracting an image to be identified from the video stream, and calculating the global average gray value of the image to be identified;
and searching the corresponding code book according to the global average gray value, if the searching is successful, all pixels of the image to be recognized are foreground, if the searching is failed, traversing each code element of each pixel of the image to be recognized, if the code element matched with the current pixel value exists, the current pixel is background, otherwise, the current pixel is foreground.
4. The method for identifying vehicles under occlusion according to claim 2, wherein the contour acquisition of the foreground image after morphological operation by Canny algorithm comprises:
smoothing the foreground image using gaussian filtering;
removing the noise of the foreground image by applying Gaussian blur, and reducing the identification of a false edge;
calculating the gradient amplitude and the gradient direction of the foreground image, wherein the calculation formula of a gradient model G is as follows:
in the formula: gx is the difference in the horizontal direction, Gy is the difference in the vertical direction;
the calculation formula of the gradient direction θ is as follows:
θ＝atan2(Gy,Gx)；
traversing pixel points of the fuzzy part of the contour curve in the foreground image, comparing the gradient strength of the current pixel point with the gradient strength in the positive and negative gradient directions, if the gradient strength of the current pixel point is greater than that of other points in the same direction, keeping the value of the current pixel point, otherwise, setting the value to be 0, and splicing all the kept pixel points into a contour.
5. The method of under-occlusion vehicle identification of claim 1,
the recognition neural network comprises a convolution module, a self-coding module and a recognition module;
the convolution module comprises N sub-convolution modules which are connected in parallel, the sub-convolution modules comprise convolution layers and a maximum pooling layer, and the convolution module is used for extracting feature information of a single vehicle image in the N sub-convolution modules respectively; the self-coding module comprises 2 layers of convolution layers and is used for fusing the characteristic information extracted by the N sub-convolution modules; the identification module is used for identifying the vehicle according to the extracted characteristic information.
6. An obscured vehicle identification device, comprising:
the first acquisition module is used for acquiring a first mask image of a vehicle region in an image to be identified through object foreground detection, and acquiring a second mask image of the vehicle region by connecting the outer edge and the outer vertex of each continuous region in the first mask image;
the first processing module is used for subtracting the first masking map from the second masking map to obtain a non-vehicle blank map, and the non-vehicle blank map comprises a plurality of first areas;
the second acquisition module is used for acquiring inner vertexes of the plurality of first areas;
the segmentation module is used for dividing continuous areas in the first mask image into a plurality of second areas by using the segmentation lines constructed by taking the inner vertexes as end points, and constructing a plurality of rectangular frames according to the segmentation lines formed by the outer vertexes and the adjacent inner vertexes of the plurality of second areas;
the third acquisition module is used for acquiring a single vehicle image corresponding to the positions of the plurality of rectangular frames in the image to be identified;
and the classification identification module is used for inputting the single vehicle image into a trained identification neural network for identification to obtain the classification information of the vehicle.
7. The obscured vehicle identification device according to claim 6, wherein the first obtaining module comprises:
the foreground obtaining module is used for obtaining a video stream of an image to be identified and extracting a foreground image from the video stream by adopting a Codebook algorithm;
the morphological operation module is used for performing morphological operation on the foreground image, and the morphological operation comprises expansion and erosion;
and the contour acquisition module is used for acquiring a contour of the foreground image after morphological operation by adopting a Canny algorithm, wherein an area in the contour is a first mask image of a vehicle area.
8. The occluded vehicle identification device of claim 7, wherein the foreground acquisition module comprises:
the gray level calculation module is used for extracting an image to be identified from the video stream and calculating the global average gray level value of the image to be identified;
and the second processing module is used for searching the corresponding code book according to the global average gray value, if the searching is successful, all pixels of the image to be identified are foreground, if the searching is failed, each coding element of each pixel of the image to be identified is traversed, if a coding element matched with the current pixel value exists, the current pixel is background, and otherwise, the current pixel is foreground.
9. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any of claims 1-5 when executing the computer program.
10. A computer-readable medium, in which a computer program is stored which, when being processed and executed, carries out the steps of the method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911327614.XA CN111126248A (en) | 2019-12-20 | 2019-12-20 | Method and device for identifying shielded vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911327614.XA CN111126248A (en) | 2019-12-20 | 2019-12-20 | Method and device for identifying shielded vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111126248A true CN111126248A (en) | 2020-05-08 |
Family
ID=70500744
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911327614.XA Pending CN111126248A (en) | 2019-12-20 | 2019-12-20 | Method and device for identifying shielded vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111126248A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113378976A (en) * | 2021-07-01 | 2021-09-10 | 深圳市华汉伟业科技有限公司 | Target detection method based on characteristic vertex combination and readable storage medium |
| CN116311106A (en) * | 2023-05-24 | 2023-06-23 | 合肥市正茂科技有限公司 | Training method, device, equipment and medium for occlusion image recognition model |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103871082A (en) * | 2014-03-31 | 2014-06-18 | 百年金海科技有限公司 | Method for counting people stream based on security and protection video image |
| CN103914842A (en) * | 2014-04-04 | 2014-07-09 | 上海电机学院 | Foreground detecting method based on Codebook background differencing |
| CN104680557A (en) * | 2015-03-10 | 2015-06-03 | 重庆邮电大学 | Intelligent detection method for abnormal behavior in video sequence image |
| CN108759667A (en) * | 2018-05-29 | 2018-11-06 | 福州大学 | Front truck distance measuring method based on monocular vision and image segmentation under vehicle-mounted camera |
-
2019
- 2019-12-20 CN CN201911327614.XA patent/CN111126248A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103871082A (en) * | 2014-03-31 | 2014-06-18 | 百年金海科技有限公司 | Method for counting people stream based on security and protection video image |
| CN103914842A (en) * | 2014-04-04 | 2014-07-09 | 上海电机学院 | Foreground detecting method based on Codebook background differencing |
| CN104680557A (en) * | 2015-03-10 | 2015-06-03 | 重庆邮电大学 | Intelligent detection method for abnormal behavior in video sequence image |
| CN108759667A (en) * | 2018-05-29 | 2018-11-06 | 福州大学 | Front truck distance measuring method based on monocular vision and image segmentation under vehicle-mounted camera |
Non-Patent Citations (1)
| Title |
|---|
| 朱世松 等: "基于轮廓特征点的重叠车辆检测与分割" * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113378976A (en) * | 2021-07-01 | 2021-09-10 | 深圳市华汉伟业科技有限公司 | Target detection method based on characteristic vertex combination and readable storage medium |
| CN113378976B (en) * | 2021-07-01 | 2022-06-03 | 深圳市华汉伟业科技有限公司 | Target detection method based on characteristic vertex combination and readable storage medium |
| CN116311106A (en) * | 2023-05-24 | 2023-06-23 | 合肥市正茂科技有限公司 | Training method, device, equipment and medium for occlusion image recognition model |
| CN116311106B (en) * | 2023-05-24 | 2023-08-22 | 合肥市正茂科技有限公司 | Training method, device, equipment and medium for occlusion image recognition model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110414507B (en) | License plate recognition method and device, computer equipment and storage medium | |
| CN106778705B (en) | Pedestrian individual segmentation method and device | |
| WO2018072102A1 (en) | Method and apparatus for removing spectacles in human face image | |
| CN112287912B (en) | Deep learning-based lane line detection method and device | |
| WO2023082453A1 (en) | Image processing method and device | |
| CN113988112B (en) | Method, device and equipment for detecting lane line and storage medium | |
| CN110751024A (en) | User identity identification method and device based on handwritten signature and terminal equipment | |
| CN111079764A (en) | Low-illumination license plate image recognition method and device based on deep learning | |
| CN114387591A (en) | License plate recognition method, system, equipment and storage medium | |
| CN107704797A (en) | Real-time detection method and system and equipment based on pedestrian in security protection video and vehicle | |
| CN114331951A (en) | Image detection method, image detection device, computer, readable storage medium, and program product | |
| CN111126248A (en) | Method and device for identifying shielded vehicle | |
| CN111539341A (en) | Target positioning method, device, electronic equipment and medium | |
| CN111161348B (en) | Object pose estimation method, device and equipment based on monocular camera | |
| CN114444565A (en) | Image tampering detection method, terminal device and storage medium | |
| CN113191189A (en) | Face living body detection method, terminal device and computer readable storage medium | |
| CN112115737B (en) | Vehicle orientation determining method and device and vehicle-mounted terminal | |
| CN111311610A (en) | Image segmentation method and terminal equipment | |
| CN117115117A (en) | Pathological image recognition method based on small sample, electronic equipment and storage medium | |
| CN115731179A (en) | Track component detection method, terminal and storage medium | |
| CN113313124B (en) | Method and device for identifying license plate number based on image segmentation algorithm and terminal equipment | |
| CN110633705A (en) | Low-illumination imaging license plate recognition method and device | |
| CN113628148A (en) | Infrared image noise reduction method and device | |
| CN113792671A (en) | Method and device for detecting face synthetic image, electronic equipment and medium | |
| CN114596210A (en) | Noise estimation method, device, terminal equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |