CN109635733B - Parking lot and vehicle target detection method based on visual saliency and queue correction - Google Patents

Parking lot and vehicle target detection method based on visual saliency and queue correction Download PDF

Info

Publication number
CN109635733B
CN109635733B CN201811517627.9A CN201811517627A CN109635733B CN 109635733 B CN109635733 B CN 109635733B CN 201811517627 A CN201811517627 A CN 201811517627A CN 109635733 B CN109635733 B CN 109635733B
Authority
CN
China
Prior art keywords
bbsm
parking lot
image
map
vehicle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811517627.9A
Other languages
Chinese (zh)
Other versions
CN109635733A (en
Inventor
陈浩
陈玲艳
陈稳
高通
赵静
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Harbin Institute of Technology
Original Assignee
Harbin Institute of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harbin Institute of Technology filed Critical Harbin Institute of Technology
Priority to CN201811517627.9A priority Critical patent/CN109635733B/en
Publication of CN109635733A publication Critical patent/CN109635733A/en
Application granted granted Critical
Publication of CN109635733B publication Critical patent/CN109635733B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/13—Satellite images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10032—Satellite or aerial image; Remote sensing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/08—Detecting or categorising vehicles

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Astronomy & Astrophysics (AREA)
  • Remote Sensing (AREA)
  • Geometry (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)

Abstract

A parking lot and vehicle target detection method based on visual saliency and queue correction belongs to the technical field of vehicle detection in parking lots. The invention solves the problems of low processing speed and poor vehicle target detection effect of the existing remote sensing image vehicle target detection method. According to the method, a saliency map BBSM based on brightness features is designed according to brightness features of a parking lot area and is used for rough extraction of the parking lot area, and then the color features and the surface features of the parking lot area are used for accurately extracting the parking lot outline; in each precisely extracted parking lot outline, suspected areas possibly containing vehicles are extracted, a calculation method of vehicle queue arrangement directions based on an edge statistical model is designed to correct the vehicle queue arrangement directions, finally, a sliding window method is used for cutting a vehicle queue to extract suspected vehicle slices, after HOG characteristics of the slices are extracted, an SVM classifier is used for secondary classification, targets classified into the vehicles are marked back to original images, and vehicle detection is achieved; the invention is applied to the technical field of vehicle detection in parking lots.

Description

Parking lot and vehicle target detection method based on visual saliency and queue correction
Technical Field
The invention belongs to the technical field of vehicle detection in parking lots, and particularly relates to a parking lot and a vehicle target detection method.
Background
The detection of the vehicle target in the remote sensing image has important application significance in the aspects of city planning, traffic management and the like. However, the existing vehicle detection research is more focused on the detection of road vehicles, and the vehicle detection research for parking in a parking lot is relatively less, and the methods can be roughly divided into two types: a target classification method based on template matching; a method for classifying objects based on feature extraction.
The template matching-based method is one of basic methods for target classification, and similarity of pixels of a template image and a target image to be recognized is measured by calculating Euclidean distance and the like, so that the type of a region to be recognized or a target is judged. However, the method can only be matched with samples contained in a template library, so that the generalization capability of the algorithm is low, the method does not have good illumination invariance, rotation invariance and view angle transformation invariance, and the point-by-point operation has high complexity and long operation time and cannot be used for real-time processing. The characteristic-based classification method is a universal and effective method for remote sensing region extraction and target detection and identification. By analyzing the difference of the target and the false alarm on some features, such as a scale-invariant feature transform feature, a Histogram of Oriented Gradients (HOG) feature, a geometric invariant moment, an aspect ratio, a texture feature and the like, selecting a feature description slice which is favorable for classification, and finally classifying the features by using a machine learning method and the like.
The main problems of vehicle target detection of the existing remote sensing image are that the area span is wide, the data volume is large, and the target detail characteristics are not obvious under the sub-meter resolution. This will result in large memory occupation of the processing system, slow processing, and less features that can be extracted from the vehicle sample slice, which will affect the classification effect of the classifier, resulting in poor vehicle target detection effect.
Therefore, it is necessary to develop a method for accurately detecting a vehicle target while reducing the processing of unnecessary data.
Disclosure of Invention
The invention aims to solve the problems that the processing speed is low due to large data volume of the existing remote sensing image vehicle target detection method, and the vehicle target detection effect is poor due to few characteristics which can be extracted by a vehicle sample slice.
The technical scheme adopted by the invention for solving the technical problems is as follows: a method for parking lot and vehicle object detection based on visual saliency and alignment correction, the method comprising the steps of:
step one, inputting a sub-meter-level high-resolution optical remote sensing image, calculating a BBSM salient image of the input sub-meter-level high-resolution optical remote sensing image based on the brightness characteristics and the visual saliency of the parking lot, and binarizing the calculated BBSM salient image;
step two, carrying out superpixel segmentation on the BBSM salient map after binarization according to the surface characteristics of the parking lot to obtain all the partitioned superpixel blocks, setting screening conditions to screen all the partitioned superpixel blocks, and calculating the centroid map CDM of each screened superpixel block(i′,j′)CDM with calculated centroid map for each super-pixel block(i′,j′)Updating the BBSM significant map after binarization to obtain a centroid map CDM of the BBSM significant map after binarization;
calculating a mass Center Density Distribution Index (CDDI) image of the BBSM salient image after binarization according to the obtained CDM, and obtaining an ROI image according to the CDDI image, namely obtaining a rough-extraction parking lot area image;
thirdly, obtaining a precisely extracted parking lot area image according to the roughly extracted parking lot area image and the color characteristics and the surface characteristics of the parking lot, and completing parking lot detection;
step four, calculating an SR saliency map of the parking lot region image which is extracted in a precise mode, and extracting a suspected vehicle region in the SR saliency map;
calculating the angle of the queue arrangement direction of each suspected vehicle area, and rotating all the suspected vehicle areas to the horizontal arrangement direction according to the calculated angle to finish the queue correction of the suspected vehicle areas;
and fifthly, cutting all the rotated suspected vehicle areas into slices by using a sliding window cutting method, extracting HOG features of the cut slices, classifying the extracted HOG features by using an SVM classifier, marking the slices classified into the vehicles back to the original image, and finishing vehicle detection.
The invention has the beneficial effects that: the invention provides a parking lot and vehicle target detection method based on visual saliency and queue correction, which is characterized in that firstly, a saliency map BBSM based on brightness features is designed according to the brightness features of a parking lot area with high brightness and a brightness distribution interval with a small proportion relative to an input map for rough extraction of the parking lot area, and then, the color features of colorless parking lot area and high brightness and the surface features of large area of the parking lot area are utilized to further accurately extract the contour of the parking lot so as to finish parking lot detection; in each precisely extracted parking lot outline, extracting suspected areas possibly containing vehicles by using an SR saliency map, designing a calculation method of vehicle queue arrangement directions based on an edge statistical model for correcting the arrangement directions of vehicle queues, finally cutting the vehicle queues by using a sliding window method, extracting a large number of suspected vehicle slices, extracting HOG (histogram of oriented gradient) features from the slices, then classifying the true and false vehicle slices by using an SVM (support vector machine) classifier, marking targets classified into the vehicles back to original images, and finally realizing vehicle detection; the parking lot area extraction method and the vehicle detection method based on the queue correction and SR saliency map can ensure that the accuracy of vehicle detection reaches more than 85%.
Compared with the traditional method, the calculation method of the BBSM saliency map designed by the invention can remarkably accelerate the processing speed of vehicle detection.
Drawings
FIG. 1 is a flow chart of a method of parking lot and vehicle object detection based on visual saliency and alignment correction of the present invention;
fig. 2 is a schematic diagram of an input sub-meter-level high-resolution optical remote sensing image according to a second embodiment of the present invention;
FIG. 3 is a BBSM saliency map calculated according to a second embodiment of the present invention;
fig. 4 is a schematic diagram of a roughly extracted parking lot area image according to a third embodiment of the present invention;
fig. 5 is a schematic diagram of a parking lot area image extracted in detail according to the fourth embodiment of the present invention;
fig. 6 is a SR-saliency map of a parking lot area according to a fifth embodiment of the present invention;
fig. 7 is a binarized SR saliency map of a certain parking lot area according to the fifth embodiment of the present invention;
fig. 8 is a schematic diagram illustrating a situation where a direction of a suspected vehicle area is deviated according to a fifth embodiment of the present invention;
fig. 9 is a diagram of an edge detection result of a suspected vehicle area according to a fifth embodiment of the present invention;
fig. 10 is a diagram of the AngleNum statistical result according to the fifth embodiment of the present invention;
wherein: the abscissa is an angle value, and the ordinate is the number of point pairs;
FIG. 11 is a graph of a vehicle target detection result 1 of the present invention;
FIG. 12 is a graph of a vehicle target detection result 2 of the present invention;
FIG. 13 is a graph of a vehicle target detection result 3 of the present invention;
FIG. 14 is a diagram of a vehicle target detection result 4 of the present invention;
Detailed Description
The first embodiment is as follows: as shown in fig. 1, the parking lot and vehicle object detection method based on visual saliency and alignment correction according to the present embodiment includes the following steps:
step one, inputting a sub-meter-level high-resolution optical remote sensing image, calculating a BBSM salient image of the input sub-meter-level high-resolution optical remote sensing image based on the brightness characteristics and the visual saliency of the parking lot, and binarizing the calculated BBSM salient image;
step two, carrying out superpixel segmentation on the BBSM salient map after binarization according to the surface characteristics of the parking lot to obtain all the partitioned superpixel blocks, setting screening conditions to screen all the partitioned superpixel blocks, and calculating the centroid map CDM of each screened superpixel block(i′,j′)CDM with calculated centroid map for each super-pixel block(i′,j′)Updating the BBSM significant map after binarization to obtain a centroid map CDM of the BBSM significant map after binarization;
calculating a mass Center Density Distribution Index (CDDI) image of the BBSM salient image after binarization according to the obtained CDM, and obtaining an ROI image according to the CDDI image, namely obtaining a rough-extraction parking lot area image;
thirdly, obtaining a precisely extracted parking lot area image according to the roughly extracted parking lot area image and the color characteristics and the surface characteristics of the parking lot, and completing parking lot detection;
step four, calculating an SR saliency map of the parking lot region image which is extracted in a precise mode, and extracting a suspected vehicle region in the SR saliency map;
calculating the angle of the queue arrangement direction of each suspected vehicle area, and rotating all the suspected vehicle areas to the horizontal arrangement direction according to the calculated angle to finish the queue correction of the suspected vehicle areas;
and fifthly, cutting all the rotated suspected vehicle areas into slices by using a sliding window cutting method, extracting HOG features of the cut slices, classifying the extracted HOG features by using an SVM classifier, marking the slices classified into the vehicles back to the original image, and finishing vehicle detection.
The second embodiment is as follows: the first difference between the present embodiment and the specific embodiment is: the specific process of the step one is as follows:
step one, as shown in fig. 2, a sub-meter high-resolution optical remote sensing image is input, the input sub-meter high-resolution optical remote sensing image is converted into a brightness channel, and a brightness image I is obtained, wherein the size of the brightness image I is MXN, and M and N respectively represent the height and width of the brightness image I;
counting the brightness values u, u being 0, corresponding to all pixel points in the brightness image I.
Statistical result PuIndicating the number of pixels with a brightness value u in the brightness image I, e.g. P14And (3) representing the number of the pixel points with the brightness value of 14 in the image I, calculating the BBSM value of each pixel point in the brightness image I as follows:
Figure BDA0001902397550000041
wherein: (I, j) represents any pixel point in the brightness image I, I(i,j)Representing the brightness value, BBSM, of any one pixel point (I, j) in the brightness image I(i,j)The value of BBSM representing pixel point (i, j);
the BBSM value of each pixel point forms a BBSM saliency map; the BBSM saliency map is shown in fig. 3;
step two, selecting a binary threshold value T1To BBSM displayBinarizing the written image to obtain a binarized BBSM significant image BBSM':
T1＝0.3×max(BBSM)+0.7×min(BBSM)
wherein: max (BBSM) is the maximum value of the BBSM saliency map, min (BBSM) is the minimum value of the BBSM saliency map;
the BBSM value after binarization of the pixel point (i, j) is:
Figure BDA0001902397550000042
the traditional method for calculating the BBSM saliency map is: inputting a sub-meter high-resolution optical remote sensing image, converting the input sub-meter high-resolution optical remote sensing image into a brightness channel to obtain a brightness image I, wherein the size of the brightness image I is MXN, and the brightness image I comprises the following components: if I is 1, …, M, j is 1, …, N, the BBSM value of the pixel in the luminance image I is:
Figure BDA0001902397550000051
wherein: m and N respectively represent the height and width of the brightness image I, and (I, j) represents any pixel point in the brightness image Ii,jRepresenting the brightness value, BBSM, of any pixel in the brightness image I(i,j)Is the BBSM value of pixel (I, j), D (I)(i,j),I(m,n)) Is a brightness value I(i,j)And a brightness value I(m,n)Is an absolute difference of
D(I(i,j),I(m,n))＝|I(i,j)-I(m,n)|
The BBSM value of each pixel point forms a BBSM saliency map;
since the pixels with the same brightness in the brightness image I should have the same BBSM value, compared with the conventional method, the BBSM saliency map acquisition method of the present embodiment speeds up the processing speed.
The third concrete implementation mode: the first difference between the present embodiment and the specific embodiment is: the specific process of the second step is as follows:
step two, BBSM only uses the brightness characteristic of the parking lot in step one, and then uses the surface characteristic of the parking lot;
on the binarized BBSM saliency map, taking the cutting step length in the height direction as W and the cutting step length in the width direction as H, and cutting the binarized BBSM saliency map into blocks by utilizing W and H
Figure BDA0001902397550000052
Each superpixel block SP is actually a small binary image, and when W cannot be divided by M or H cannot be divided by N, a small number of edge regions of the image are removed to ensure that W can be divided by M and H can be divided by N.
Defining a weight W for each superpixel blockSPComprises the following steps:
Figure BDA0001902397550000053
wherein: SP(i′,j′)The BBSM value after binarization of the pixel points (i ', j') in the super pixel block is obtained;
step two, setting the screening condition as WSPβ · W · H, wherein: beta is a screening coefficient, the super-pixel blocks obtained by the segmentation in the second step are screened by using the set screening conditions, the super-pixel blocks with the weight values larger than the screening conditions are screened, and the super-pixel blocks with the weight values not larger than the screening conditions are removed;
step two and three, calculating the CDM of the centroid map of each selected super-pixel block(i′,j′): CDM refers to a binary map identifying the centroid of each super-pixel block,
the CDM is calculated by
Figure BDA0001902397550000054
Wherein:
Figure BDA0001902397550000061
and
Figure BDA0001902397550000062
representing the centroid of a super-pixel block, i.e. eachThe average abscissa and ordinate values of all points whose median in the super-pixel block is 1. That is, a point of the centroid diagram with a value of 1 indicates that a parking lot exists in a certain size of area centered on the point. CDM with calculated centroid map for each super-pixel block(i′,j′)Updating the BBSM significant map after binarization to obtain a centroid map CDM of the BBSM significant map after binarization;
step two and four, using unit matrix Te with the size of k multiplied by kk×kCarrying out CDM convolution with the mass center image of the BBSM salient map after binarization to obtain a mass center density distribution index CDDI image of the BBSM salient map after binarization;
Figure BDA0001902397550000063
wherein:
Figure BDA0001902397550000064
the value of k is 120, which considers that the corresponding size of 120 × 120 pixels is 480 × 480m which is larger than the size of most parking lots under the resolution of 4 meters, so that the generated ROI image can cover all the parking lots without omission during convolution, and excessive alarm omission is prevented during final vehicle detection.
Step two and five, selecting a binary threshold value T2And binarizing the mass center density distribution index CDDI image of the binarized BBSM saliency map to obtain an ROI image and obtain a roughly extracted parking lot region image. The parking lot area image extracted roughly is shown in fig. 4, that is, all 1 s are taken for points where CDDI is not 0.
The fourth concrete implementation mode: the first difference between the present embodiment and the specific embodiment is: the specific process of the third step is as follows:
thirdly, removing the non-road irrelevant areas by utilizing the color characteristics of the parking lot, wherein the color characteristics of the parking lot are represented by the fact that the parking lot presents neutral gray color and has higher brightness,
in the parking area image of crudely extracting, screening the pixel that satisfies the screening condition through setting up the value of R, G, B three wave bands, wherein: the screening conditions are as follows: r, G, B the values of the three bands are all in the range of (130,250), and the difference between any two bands must not exceed 40;
step two, removing the road area by using the surface characteristics of the parking lot, wherein the surface characteristics have larger area in the parking lot compared with other road surfaces and field areas;
filling holes in the screening result obtained in the step three, removing roads with the width less than 10 meters by using corrosion operation, and then calculating the area of each connected domain to remove the connected domains with the area less than 800 square meters; and obtaining a parking lot area fine extraction result of the sub-meter high-resolution optical remote sensing image after expansion reduction, obtaining a fine extraction parking lot area image, and completing parking lot detection. The parking lot area fine extraction result is shown in fig. 5;
the fifth concrete implementation mode: the first difference between the present embodiment and the specific embodiment is: the specific process of the step four is as follows:
fourthly, Fourier transform is carried out on the parking lot area image which is extracted in a fine mode to calculate an amplitude spectrum A (f) and a phase spectrum P (f):
Figure BDA0001902397550000071
Figure BDA0001902397550000072
wherein: i' (x) is a luminance image in the finely extracted parking field region;
Figure BDA0001902397550000073
indicating that I' (x) takes the magnitude of the fourier transform,
Figure BDA0001902397550000074
the phase value representing the fourier transform of I' (x);
Figure BDA0001902397550000075
represents a fourier transform;
changing the amplitude spectrum A (f) into a log spectrum L (f), then carrying out linear spatial filtering (the filtering mode is 3 multiplied by 3 mean value filtering) on the log spectrum, and carrying out difference on the log spectrum L (f) and the linear spatial filtering result to obtain a residual spectrum R (f);
L(f)＝log(A(f))
R(f)＝L(f)-hn(f)*L(f)
wherein: h isn(f) Is a mean filtering operator;
performing inverse Fourier transform on the residual spectrum R (f) and the phase spectrum P (f) to obtain an inverse Fourier transform result, and performing linear spatial filtering on the inverse Fourier transform result by using a Gaussian filter to obtain an SR saliency map: the resulting SR saliency map is shown in figure 6,
Figure BDA0001902397550000076
wherein: g (x) is a gaussian filter operator; SR (x) represents the SR saliency map;
Figure BDA0001902397550000077
represents an inverse fourier transform;
normalizing the obtained SR saliency map, binarizing the normalized SR saliency map, and setting a binarization threshold value T3Obtaining a binarized SR saliency map, wherein max (SR) is the maximum value of the SR saliency map, and the region with the value of 1 in the binarized SR saliency map is a suspected vehicle region; that is, the suspected vehicle area is a binary map identifying all suspected vehicle areas, as shown in fig. 7.
In a parking lot, most vehicles are distributed in an aggregated manner, namely a plurality of vehicles are parked in a pile in the same direction, and in order to enable the arrangement direction of an area containing the vehicles in an obtained image of a suspected area to be horizontal, the invention provides a method for calculating the arrangement direction of objects in the area. Fig. 8 is one of images of suspected areas in a parking lot, and it can be seen from the figure that another vehicle is located beside a vehicle queue in the suspected area, and the saliency of the independent vehicle in the SR saliency map is higher, so that the vehicle queue in the binary map obtained during binarization is connected to the identification map of the independent vehicle, which causes the long side direction of the minimum external rectangle to be inconsistent with the original arrangement direction, because the vehicle queue is closer to the independent vehicle, which causes the binarization result of the SR saliency map to be connected, and if the minimum external quadrangle of the connected domain is directly taken, the final direction value is affected by the queue and the independent target in the same way. The human judgment actually considers the influence of all vehicles in a comprehensive way, namely if the last angle is regarded as a voting, 15 vehicles in the queue respectively have a voting right, and the independent vehicles also have a voting right, and finally vote together to select the angle with the highest vote number. Based on this principle, the present embodiment designs an angle calculation method.
Step two, for the vehicles parked in the queue, extracting the edge of the suspected vehicle area by using a Sobel operator to obtain an edge map of the suspected vehicle area, wherein the result of the edge map of the suspected vehicle area is shown in FIG. 9, calculating the angle value of the included angle between the connecting line of any two points in the edge map of the suspected vehicle area and the horizontal direction by using a statistical method, recording the number of point pairs corresponding to all the angle values, and marking as Anglenumu′And u' is 0, so, 179, (an included angle between any two point connecting line and the horizontal direction in the edge image of the suspected vehicle area is defined as that an intersection point of an extension line of any two point connecting line and the horizontal direction is a circle center, the right side of the circle center is a positive direction in the horizontal direction, the left side of the circle center is a negative direction in the horizontal direction, an angle which is formed by starting from the positive direction in the horizontal direction and passing through any two point connecting line is defined as an "included angle between any two point connecting line and the horizontal direction" in the edge image of the suspected vehicle area, and the value range of the included angle is 0-179 degrees): anglenumu′Representing the number of the point pairs corresponding to the angle value u'; such as Anglenum5Representing the total number of all point pairs with the connecting line angle of 5 (+ -0.5) degrees in the edge map of the suspected vehicle area;
fig. 10 is a graph of the AngleNum statistics of fig. 9, where the maximum value of AngleNum corresponds to an angle of 6 degrees,
maximum Anglenumu′The corresponding angle value u' is used as the angle of the queue arrangement direction of the suspected vehicle area, and the suspected vehicle area is rotated according to the obtained angleObtaining suspected vehicle areas arranged in the horizontal direction;
for example: if the angle corresponding to a certain suspected vehicle area is 8 °, the suspected vehicle area is rotated 8 ° toward the edge direction (i.e., horizontal direction) of the image, so that the suspected vehicle area is parallel to the edge of the image, and the suspected vehicle areas arranged in the horizontal direction are obtained.
Step four, for the vehicles which are not parked in the queue (the vehicles which are not parked in the queue refer to the area of the suspected vehicle area and are not larger than 25 square meters), namely the suspected vehicle area is provided with only one vehicle and almost occupies the whole area, because the area is small, the direction of the vehicle can be generally and accurately represented by the long side direction of the minimum circumscribed rectangle, the angle value of the included angle between the wide side of the suspected vehicle area and the horizontal direction is directly taken as the angle of the queue arrangement direction, and the suspected vehicle area is rotated according to the obtained angle value to obtain the suspected vehicle area which is arranged in the horizontal direction;
and step four, repeating the processes of the step four and the step four, calculating the angles of the arrangement directions of all the suspected vehicle areas in the whole parking lot, rotating all the suspected vehicle areas according to the obtained corresponding angles to obtain all the suspected vehicle areas arranged in the horizontal direction, and finishing the queue correction of the suspected vehicle areas.
The sixth specific implementation mode: the first difference between the present embodiment and the specific embodiment is: the concrete process of the step five is as follows:
because the size of the vehicle is small under the sub-meter resolution, the vehicle is difficult to be initially positioned by utilizing the characteristics, and meanwhile, the condition that the alarm missing is as little as possible is also considered;
setting the window size of a sliding window to be 20 multiplied by 12 pixels, setting the step length to be 2 pixels, cutting all the rotated suspected vehicle areas into slices, extracting HOG features of the cut slices, carrying out secondary classification on the extracted HOG features by using an SVM classifier, marking the slices classified into vehicles back to an original image, and finishing vehicle detection.
The window size of the sliding window is larger than that of all conventional vehicles, and it is considered that when the window contains the vehicle, the complete vehicle and the edge thereof should exist in the sample slice, so that the HOG feature of the vehicle and the HOG feature of the vehicle outline can be obtained simultaneously when the HOG feature is extracted, the feature information of the vehicle can be more fully utilized, and the classification accuracy can be improved.
The SVM classifier of the second classification adopted in the invention is a trained SVM classifier, wherein a positive sample used for training the SVM classifier is an HOG feature extracted from a real vehicle slice selected by 400 personnel, and a negative sample is an HOG feature extracted from a non-vehicle slice selected by 500 personnel.
The seventh embodiment: the first or sixth differences from the present embodiment are: the parameters during the HOG feature extraction are set as follows: the size of the cell is 4 pixels, the number of divided angle blocks is 5, and the step length is 2 pixels. And the final HOG characteristic dimension obtained by extraction is 420 dimensions.
The specific implementation mode is eight: the third difference between the present embodiment and the specific embodiment is that: the value ranges of the cutting step length W in the height direction and the cutting step length H in the width direction are both [10,100 ].
The specific implementation method nine: the third difference between the present embodiment and the specific embodiment is that: the value range of the screening coefficient beta is [0.3, 0.8 ].
The detailed implementation mode is ten: the third difference between the present embodiment and the specific embodiment is that: the binary threshold value T2Has a value range of [0.1, 0.9 ]]。
The following examples were used to demonstrate the beneficial effects of the present invention:
the experimental image is a large scene image of an area where a large parking lot in texas, usa is located, which is acquired from Google Earth, and the size of the image is 13000 × 13000 pixels, the resolution is 0.5 meter, and the image contains 5145 vehicle targets. The test result shows that 93.8% of parking lot pixels can be extracted according to the pixel point calculation by the BBSM saliency map-based parking lot extraction method, and a parking lot area can be extracted more accurately; the vehicle detection algorithm based on the queue correction and the SR saliency map is calculated according to the vehicle target, the accuracy is 85.12%, the recall rate is 89.76%, and the detection rate is high. As shown in fig. 11, 12, 13 and 14, all are partial detection result graphs.
The above-described calculation examples of the present invention are merely to explain the calculation model and the calculation flow of the present invention in detail, and are not intended to limit the embodiments of the present invention. It will be apparent to those skilled in the art that other variations and modifications of the present invention can be made based on the above description, and it is not intended to be exhaustive or to limit the invention to the precise form disclosed, and all such modifications and variations are possible and contemplated as falling within the scope of the invention.

Claims (9)

1. Parking lot and vehicle object detection method based on visual saliency and alignment correction, characterized in that the method comprises the following steps:
step one, inputting a sub-meter-level high-resolution optical remote sensing image, calculating a BBSM salient image of the input sub-meter-level high-resolution optical remote sensing image based on the brightness characteristics and the visual saliency of the parking lot, and binarizing the calculated BBSM salient image; the specific process comprises the following steps:
step one, inputting a sub-meter high-resolution optical remote sensing image, converting the input sub-meter high-resolution optical remote sensing image into a brightness channel to obtain a brightness image I, wherein the size of the brightness image I is MXN, and M and N respectively represent the height and width of the brightness image I;
counting the brightness values u, u being 0, corresponding to all pixel points in the brightness image I.
Statistical result PuAnd (3) representing the number of pixel points with the brightness value of u in the brightness image I, calculating the BBSM value of each pixel point in the brightness image I as follows:
Figure FDA0002536231760000011
wherein: (I, j) represents any pixel point in the brightness image I, I(i,j)Representing the brightness value, BBSM, of any one pixel point (I, j) in the brightness image I(i,j)The value of BBSM representing pixel point (i, j);
the BBSM value of each pixel point forms a BBSM saliency map;
step two, selecting a binary threshold value T1And binarizing the BBSM significant map to obtain a binarized BBSM significant map BBSM':
T1＝0.3×max(BBSM)+0.7×min(BBSM)
wherein: max (BBSM) is the maximum value of the BBSM saliency map, min (BBSM) is the minimum value of the BBSM saliency map;
the BBSM value after binarization of the pixel point (i, j) is
Figure FDA0002536231760000012
Step two, carrying out superpixel segmentation on the BBSM salient map after binarization according to the surface characteristics of the parking lot to obtain all the partitioned superpixel blocks, setting screening conditions to screen all the partitioned superpixel blocks, and calculating the centroid map CDM of each screened superpixel block(i′,j′)(i ', j') represents a pixel point (i ', j') in the superpixel block, using the calculated centroid map CDM of each superpixel block(i′,j′)Updating the BBSM significant map after binarization to obtain a centroid map CDM of the BBSM significant map after binarization;
calculating a mass Center Density Distribution Index (CDDI) image of the BBSM salient image after binarization according to the obtained CDM, and obtaining an ROI image according to the CDDI image, namely obtaining a rough-extraction parking lot area image;
thirdly, obtaining a precisely extracted parking lot area image according to the roughly extracted parking lot area image and the color characteristics and the surface characteristics of the parking lot, and completing parking lot detection;
step four, calculating an SR saliency map of the parking lot region image which is extracted in a precise mode, and extracting a suspected vehicle region in the SR saliency map;
calculating the angle of the queue arrangement direction of each suspected vehicle area, and rotating all the suspected vehicle areas to the horizontal arrangement direction according to the calculated angle to finish the queue correction of the suspected vehicle areas;
and fifthly, cutting all the rotated suspected vehicle areas into slices by using a sliding window cutting method, extracting HOG features of the cut slices, classifying the extracted HOG features by using an SVM classifier, marking the slices classified into the vehicles back to the original image, and finishing vehicle detection.
2. The visual saliency and queue correction based parking lot and vehicle object detection method as claimed in claim 1, wherein said step two specific process is:
step two, on the binarized BBSM saliency map, taking the cutting step length in the height direction as W and the cutting step length in the width direction as H, and cutting the binarized BBSM saliency map into the BBSM saliency map by using W and H
Figure FDA0002536231760000021
A super pixel block SP;
defining a weight W for each superpixel blockSPComprises the following steps:
Figure FDA0002536231760000022
wherein: SP(i′,j′)The BBSM value after binarization of the pixel points (i ', j') in the super pixel block is obtained;
step two, setting the screening condition as WSPβ · W · H, wherein: beta is a screening coefficient, the super-pixel blocks obtained by the segmentation in the second step are screened by using the set screening conditions, the super-pixel blocks with the weight values larger than the screening conditions are screened, and the super-pixel blocks with the weight values not larger than the screening conditions are removed;
step two and three, calculating the CDM of the centroid map of each selected super-pixel block(i′,j′)：
Figure FDA0002536231760000023
Wherein:
Figure FDA0002536231760000024
and
Figure FDA0002536231760000025
representing the centroid of a super-pixel block, with the calculated centroid map CDM of each super-pixel block(i′,j′)Updating the BBSM significant map after binarization to obtain a centroid map CDM of the BBSM significant map after binarization;
step two and four, using unit matrix Te with the size of k multiplied by kk×kCarrying out CDM convolution with the mass center image of the BBSM salient map after binarization to obtain a mass center density distribution index CDDI image of the BBSM salient map after binarization;
Figure FDA0002536231760000031
wherein:
Figure FDA0002536231760000032
step two and five, selecting a binary threshold value T2And binarizing the mass center density distribution index CDDI image of the binarized BBSM saliency map to obtain an ROI image and obtain a roughly extracted parking lot region image.
3. The visual saliency and queue correction based parking lot and vehicle object detection method as claimed in claim 1, wherein said step three specific process is:
step three, in the parking area image of thick extraction, filter the pixel that satisfies the screening condition through setting for the value of R, G, B three wave bands, wherein: the screening conditions are as follows: r, G, B the values of the three wave bands are all in the range of (130,250), and the difference value of any two wave bands is less than or equal to 40;
step two, filling holes in the screening result obtained in the step one, and calculating the area of each connected domain after removing roads with the width less than 10 meters by using corrosion operation to remove the connected domains with the area less than 800 square meters; and obtaining a parking lot area fine extraction result of the sub-meter high-resolution optical remote sensing image after expansion reduction, obtaining a fine extraction parking lot area image, and completing parking lot detection.
4. The visual saliency and queue correction based parking lot and vehicle object detection method as claimed in claim 1, wherein said step four is specifically processed by:
fourthly, Fourier transform is carried out on the parking lot area image which is extracted in a fine mode to calculate an amplitude spectrum A (f) and a phase spectrum P (f):
Figure FDA0002536231760000033
Figure FDA0002536231760000034
wherein: i' (x) is a luminance image in the finely extracted parking field region;
Figure FDA0002536231760000035
indicating that I' (x) takes the magnitude of the fourier transform,
Figure FDA0002536231760000036
the representation I' (x) takes the fourier transformed phase value,
Figure FDA0002536231760000037
represents a fourier transform;
changing the amplitude spectrum A (f) into a log spectrum L (f), then carrying out linear spatial filtering on the log spectrum, and carrying out difference on the log spectrum L (f) and a linear spatial filtering result to obtain a residual spectrum R (f);
L(f)＝log(A(f))
R(f)＝L(f)-hn(f)*L(f)
wherein: h isn(f) Is a mean filtering operator;
performing inverse Fourier transform on the residual spectrum R (f) and the phase spectrum P (f) to obtain an inverse Fourier transform result, and performing linear spatial filtering on the inverse Fourier transform result by using a Gaussian filter to obtain an SR saliency map:
Figure FDA0002536231760000041
wherein: g (x) is a gaussian filter operator; SR (x) represents the SR saliency map,
Figure FDA0002536231760000042
represents an inverse fourier transform;
normalizing the obtained SR saliency map, binarizing the normalized SR saliency map, and setting a binarization threshold value T3Obtaining a binarized SR saliency map, wherein max (SR) is the maximum value of the SR saliency map, and the region with the value of 1 in the binarized SR saliency map is a suspected vehicle region;
step two, for the vehicles parked in the queue, extracting the edge of the suspected vehicle area by using a Sobel operator to obtain an edge map of the suspected vehicle area, calculating the angle value of the included angle between the connecting line of any two points in the edge map of the suspected vehicle area and the horizontal direction, recording the number of point pairs corresponding to all the angle values, and marking as Anglenumu′0, 179, wherein: anglenumu′Representing the number of the point pairs corresponding to the angle value u';
maximum Anglenumu′The corresponding angle value u' is used as the angle of the queue arrangement direction of the suspected vehicle area, and the suspected vehicle area is rotated according to the obtained angle to obtain the suspected vehicle area arranged in the horizontal direction;
step four, for the vehicles which are not parked in the queue, directly taking the angle value of the included angle between the wide side of the suspected vehicle area and the horizontal direction as the angle of the queue arrangement direction, and rotating the suspected vehicle area according to the obtained angle value to obtain the suspected vehicle area which is arranged in the horizontal direction;
and step four, repeating the processes of the step four and the step four, calculating the angles of the arrangement directions of all the suspected vehicle areas in the whole parking lot, rotating all the suspected vehicle areas according to the obtained corresponding angles to obtain all the suspected vehicle areas arranged in the horizontal direction, and finishing the queue correction of the suspected vehicle areas.
5. The visual saliency and queue correction based parking lot and vehicle object detection method as claimed in claim 1, wherein said step five specific process is:
setting the window size of a sliding window to be 20 multiplied by 12 pixels, setting the step length to be 2 pixels, cutting all the rotated suspected vehicle areas into slices, extracting HOG features of the cut slices, carrying out secondary classification on the extracted HOG features by using an SVM classifier, marking the slices classified into vehicles back to an original image, and finishing vehicle detection.
6. The visual saliency and queue correction based parking lot and vehicle object detection method according to claim 1 or 5, characterized in that the parameters at the time of HOG feature extraction are set as: the size of the cell is 4 pixels, the number of divided angle blocks is 5, and the step length is 2 pixels.
7. The visual saliency and queue correction based parking lot and vehicle object detection method as claimed in claim 2, characterized in that said height direction cutting step W and width direction cutting step H both range [10,100 ].
8. The visual saliency and queue correction based parking lot and vehicle object detection method of claim 2 wherein the screening coefficient β is in the range of [0.3, 0.8 ].
9. The visual saliency and alignment correction based parking lot and vehicle object detection method as claimed in claim 2, characterized in that said binarization threshold value T2Has a value range of [0.1, 0.9 ]]。
CN201811517627.9A 2018-12-12 2018-12-12 Parking lot and vehicle target detection method based on visual saliency and queue correction Active CN109635733B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811517627.9A CN109635733B (en) 2018-12-12 2018-12-12 Parking lot and vehicle target detection method based on visual saliency and queue correction

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811517627.9A CN109635733B (en) 2018-12-12 2018-12-12 Parking lot and vehicle target detection method based on visual saliency and queue correction

Publications (2)

Publication Number Publication Date
CN109635733A CN109635733A (en) 2019-04-16
CN109635733B true CN109635733B (en) 2020-10-27

Family

ID=66073158

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811517627.9A Active CN109635733B (en) 2018-12-12 2018-12-12 Parking lot and vehicle target detection method based on visual saliency and queue correction

Country Status (1)

Country Link
CN (1) CN109635733B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110488280B (en) * 2019-08-29 2022-03-11 广州小鹏自动驾驶科技有限公司 Method and device for correcting parking space profile, vehicle and storage medium
CN111444806B (en) * 2020-03-19 2023-06-20 成都云盯科技有限公司 Commodity touch information clustering method, device and equipment based on monitoring video
CN111753692B (en) * 2020-06-15 2024-05-28 珠海格力电器股份有限公司 Target object extraction method, product detection method, device, computer and medium
CN113033408B (en) * 2021-03-26 2023-10-20 北京百度网讯科技有限公司 Data queue dynamic updating method and device, electronic equipment and storage medium
CN115438702A (en) * 2022-10-18 2022-12-06 国网山东省电力公司营销服务中心(计量中心) Power line carrier channel noise detection method and system

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103310195A (en) * 2013-06-09 2013-09-18 西北工业大学 LLC-feature-based weak-supervision recognition method for vehicle high-resolution remote sensing images

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8655016B2 (en) * 2011-07-29 2014-02-18 International Business Machines Corporation Example-based object retrieval for video surveillance
CN107133558B (en) * 2017-03-13 2020-10-20 北京航空航天大学 Infrared pedestrian significance detection method based on probability propagation

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103310195A (en) * 2013-06-09 2013-09-18 西北工业大学 LLC-feature-based weak-supervision recognition method for vehicle high-resolution remote sensing images

Also Published As

Publication number Publication date
CN109635733A (en) 2019-04-16

Similar Documents

Publication Publication Date Title
CN109635733B (en) Parking lot and vehicle target detection method based on visual saliency and queue correction
CN115861135B (en) Image enhancement and recognition method applied to panoramic detection of box body
CN106651872B (en) Pavement crack identification method and system based on Prewitt operator
CN109657632B (en) Lane line detection and identification method
CN106683119B (en) Moving vehicle detection method based on aerial video image
CN104778721A (en) Distance measuring method of significant target in binocular image
CN102708356A (en) Automatic license plate positioning and recognition method based on complex background
Choi et al. Vehicle detection from aerial images using local shape information
CN107610114A (en) Optical satellite remote sensing image cloud snow mist detection method based on SVMs
CN111382658B (en) Road traffic sign detection method in natural environment based on image gray gradient consistency
CN110569751B (en) High-resolution remote sensing image building extraction method
CN114549981A (en) Intelligent inspection pointer type instrument recognition and reading method based on deep learning
CN110070545B (en) Method for automatically extracting urban built-up area by urban texture feature density
CN109635722B (en) Automatic identification method for high-resolution remote sensing image intersection
CN111915583A (en) Vehicle and pedestrian detection method based on vehicle-mounted thermal infrared imager in complex scene
CN114596551A (en) Vehicle-mounted forward-looking image crack detection method
CN109389167A (en) Traffic sign recognition method and system
CN110321855A (en) A kind of greasy weather detection prior-warning device
CN106845458A (en) A kind of rapid transit label detection method of the learning machine that transfinited based on core
CN113239733A (en) Multi-lane line detection method
CN110516666B (en) License plate positioning method based on combination of MSER and ISODATA
CN115841669A (en) Pointer instrument detection and reading identification method based on deep learning technology
CN111091071A (en) Underground target detection method and system based on ground penetrating radar hyperbolic wave fitting
CN108520252B (en) Road sign identification method based on generalized Hough transform and wavelet transform
CN103065296B (en) High-resolution remote sensing image residential area extraction method based on edge feature

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant