KR20220112947A - Surveillance Video Driven Status Analysis System for Parking Lots - Google Patents

Abstract

The present invention relates to a parking status analysis system and method using a CCTV footage. The parking status analysis system and method can effectively enhance efficiency for operating a parking lot by analyzing a model of a vehicle entering the parking lot using a latest technology such as deep learning and the like for a footage filmed by a CCTV located in the parking lot and by obviously recognizing data on the vehicle to improve accuracy of calculating variable values required for recognizing an available parking space such as a distance between vehicles, a distance between a vehicle and a parking line, a distance between the vehicle and a boundary of a parking space and the like in the CCTV footage.

Description

Parking status analysis system and method using CCTV camera image {Surveillance Video Driven Status Analysis System for Parking Lots}

By accurately detecting and recognizing the location and size of the vehicle, the distance between the vehicle and the vehicle, the distance between the vehicle and the parking line, and the distance between the vehicle and the fence, the number of vehicles parked and the number of empty parking surfaces, as well as the number of parking spaces that can actually be parked This is to present a parking status analysis system and method that can accurately detect and recognize the number of vehicles without additional equipment or construction.

The existing smart parking management system installs and operates ultrasonic, roof, and optical sensors in units of individual parking surfaces to analyze whether a specific parking surface is occupied, the number of additional parking spaces available, and the number of vehicles currently parked. do.

Recently, with the development of computer vision technology, a technology for detecting whether a parking surface is occupied by an object recognition method has been introduced, but due to the diversity of vehicle types, the initial technology detects a stopper installed on the parking surface A method of judging whether or not there is a vehicle has been adopted, and recently, a technology for determining whether a corresponding parking surface is occupied based on the existence of a vehicle regardless of vehicle type has been introduced.

However, even using such a technology, there is a limitation in not being able to accurately detect cases of so-called unmanned parking, such as parking across two parking surfaces or partially invading other parking surfaces.

In the case of shooting a parking surface in a wide area using the minimum camera in the limit situation as described above, the perspective effect of looking large (longer) in the near area and small (short) in the far area and when a small vehicle is obscured by a large vehicle , it is known that it appears more prominently when the vehicle is obscured by an artificial structure.

Accordingly, the present invention accurately detects (or recognizes) the location and size of the vehicle, the distance between the vehicle and the vehicle, the distance between the vehicle and the parking line, the distance between the vehicle and the fence, etc. , to present a parking status analysis system and method that can accurately detect (or recognize) the number of parking surfaces that can actually be parked without a separate device or construction.

Patent Publication 10-2011-0139503 Vehicle model analysis system and method therefor Registered Patent 10-2019047 Apparatus and method for estimating vehicle speed in video

Accordingly, in the present invention, by accurately detecting (or recognizing) the location and size of the vehicle, the distance between the vehicle and the vehicle, the distance between the vehicle and the parking line, and the distance between the vehicle and the fence, as well as the number of parked vehicles and the number of empty parking surfaces, , to present a parking status analysis system and method that can accurately detect (or recognize) the number of parking surfaces that can actually be parked without a separate device or construction.

The parking status analysis system using the CCTV camera image according to the present invention for realizing the above technical problem is one or more cameras 100 for shooting an image of the parking lot including the parking surface 101 and the entry zone 102 ; a detection and tracking unit 200 for detecting and tracking a vehicle entering the parking lot through the entry zone 102 from the image taken by the camera 100; Recognition unit 300 for recognizing the size of the vehicle being tracked by the inspection and tracking unit 200; a position detection unit 400 for detecting a position of the vehicle being tracked by the detection and tracking unit 200 when the vehicle is stopped on a parking surface; a correction unit 500 for correcting the position of the vehicle detected by the position detection unit 400 in consideration of the real-world magnification of the image of the camera 100 according to perspective; and a distance between vehicles, a distance between a vehicle and a parking line, and a distance between a vehicle and a parking surface boundary using the size of the vehicle recognized by the recognition unit 300 and the position of the vehicle corrected by the correction unit 500 An analysis unit 600 that calculates an abnormality and analyzes the number of parking surfaces that can be additionally parked using the calculation result; as an essential component, the detection and tracking unit 200 and the position detection unit 400 are It is located in the parking lot for the purpose of more accurately detecting the size and location of the measurement standard 700 that knows the size and location; provides a parking status analysis system as an additional component. As the measurement standard 700, one or more symbols, figures, characters, or entry and exit breaker, exit breaker, booth, (in case of an underground parking lot) parking lot are located on the ground of the parking lot and/or on the surface of an artificial structure located in the parking lot. (pillars, etc.) may be used.

In addition, the parking status analysis method using the CCTV camera image according to the present invention for realizing the above technical problem is an image shooting step ( 10); a detection and tracking step (20) of detecting and tracking a vehicle entering the parking lot through the entry zone (102) from the image obtained in the image capturing step (10); a vehicle recognition step 30 of recognizing the size of the vehicle detected by the detection and tracking step 20; a position detection step (40) of detecting the position when the vehicle being tracked in the detection and tracking step (20) stops on a parking surface; a vehicle position correction step (50) of correcting the position of the vehicle detected in the position detection step (40) in consideration of the magnification of the image obtained in the image capturing step (10) according to perspective; Using the size of the vehicle according to the model of the vehicle recognized in the vehicle recognition step 30 and the location of the vehicle corrected in the vehicle location correction step 50, the distance between vehicles, the distance between the vehicle and the parking line, and the vehicle and parking A parking surface analysis step (60) of calculating any one or more of the distances between the surface boundaries and analyzing the number of parking surfaces that can be additionally parked using the calculation result is an essential component (step), and the detection and tracking The measurement standard detection step 70 for more accurately detecting the size and location of the vehicle in the step 20 and the location detection step 40; as an additional component, the measurement standard detection step 70 is the ground level of the parking lot And/or provides a parking status analysis method, characterized in that for detecting the measurement standard 700, which is one or more marks located on the surface of the artificial structure located in the parking lot.

The present invention analyzes a model of a vehicle entering a parking lot by using the latest technology such as deep learning, and clearly identifies the vehicle's specifications accordingly, so that the distance between the vehicles in the CCTV image, the distance between the vehicle and the parking line, and the vehicle There is an effect that can effectively improve the efficiency of parking lot operation by improving the calculation accuracy of the variable values required to identify the parking surface that can be parked, such as the distance between the boundaries of the parking surfaces.

Figure 1 Example of a schematic configuration of a parking status analysis system using a CCTV camera
Figure 2 Example of implementation of position detection
Fig. 3 An example of the correction carried out by the correction unit

The present invention will be described in detail with reference to the accompanying drawings as follows.

1 is a schematic configuration of a parking status analysis system using a CCTV camera according to the present invention, and the camera 100 may be a PTZ camera and a multi-function camera that is currently widely sold as a commercial product, and the parking surface in the parking lot ( 101) and an image of the parking lot including the entry zone 102 is photographed, and the acquired image is transmitted to the detection and tracking unit 200 .

The detection and tracking unit 200 detects the vehicle entering the parking lot through the entry zone 102 from the image taken by the camera 100, multi-object detection (hereinafter MOD) and multi-object tracking (Multi Object Tracking). : It detects and tracks using computer vision technology such as MOT), and transmits the image of the detected vehicle to the recognition unit 300 , and when the vehicle being tracked stops, the image of the vehicle is transmitted to the position detection unit 400 .

In order to proceed with the MOD and MOT, in the detection and tracking unit 200, R-CNN (Region-based Convolutional Neural Network), Fast R-CNN, Faster R-CNN, YOLO (You Only Look Once), YOLO 9000, After object recognition through MOD using any one or more algorithms of YOLO v3, SSD (Single Shot Multi-box Detector), Deep SORT (Simple Online and Real-Time Tracking), Hungarian (Kuhn-Munkres), BOOSTING Tracker, MIL Tracker , KCF Tracker, Median Flow Tracker, TLD Tracker, MOSSE Tracker, CSRT Tracker, etc. are used to track the movement of multiple objects. After the vehicle is stopped, an image of the vehicle that has taken a predetermined time, such as 1 minute, is separately divided and transmitted to the position detection unit 400 .

Among the MOD algorithms, R-CNN is the “Rich feature hierarchies for accurate object detection and semantic segmentation Tech report (v5)” announced through CVPR in 2014, and the Fast R-CNN algorithm is “ Through “Faster R-CNN”, the Faster R-CNN algorithm has been developed through “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks” announced at NIPS in 2015, YOLO v1, 9000 (including v2), The v3 algorithms are “You Only Look Once: Unified, Real-Time Object Detection” presented at CVPR in 2016, “YOLO9000: Better, Faster, Stronger” presented at CVPR in 2017, and “YOLOv3: An Incremental Improvement”, etc., “SSD: Single Shot MultiBox Detector” announced at ECCV in 2016, etc., Deep SORT, “Simple Online and Realtime Tracking with a Deep Association Metric” announced at arXiv in 2017, etc. Hungarian (Kuhn-Munkres) has been announced through “A Distributed Version of the Hungarian Method for Multi-Robot Assignment” announced on arXiv in 2018, and BOOSTING among the trackers for MOT was announced in 2006 With Helmut Grabner et al., "Real-time tracking via on-line boosting", the MIL Tracker was published in 2010 by Boris Babenko, " Through “Robust Object Tracking with Online Multiple Instance Learning”, KCF Tracker (kernelized correlation filter), through Joao F. Henriques et al., “High-Speed Tracking with Kernelized Correlation Filters” published in 2014, Median Flow Tracker Through Zdenek Kalal et al., "Forward-Backward Error: Automatic Detection of Tracking Failures" published in 2010, Tracking - Learning - Detection (TLD Tracker) was published in Zdenek Kalal et al., "Tracking- Learning-Detection", the MOSSE (Minimum Output Sum of Squared Error) Tracker, from David S. Bolme et al., published in 2010, "Visual Object Tracking using Adaptive Correlation Filters", CSRT Tracker by Lukezic, 2018 It has been published in A et al.'s "Correlation Filter Tracker with Channel and Spatial Reliablity".

The recognition unit 300 analyzes the image of the vehicle entering the entry zone 102 received from the detection and tracking unit 200 to find the vehicle model, and analyzes the specifications for each vehicle model accordingly. send to Traditionally, as a method for analyzing a vehicle model in an image, Clady, X., et al. Search for the most similar vehicle model using the Nearest Neighbor algorithm on the front image of the vehicle described in “Multi-class vehiicle type recognition system” presented at the IAPR workshop on artificial neural networks in pattern recognition in 2008. There is a method using the Deformable Part Model (DPM) algorithm described in “Fine-grained categorization for 3D scene understanding” published by Stark, M., et al. in the International Journal of Robotics Research in 2011. As a method using deep learning, it is initially through pre-trained weights of the ImageNet dataset described in “A large scale car dataset for fine-grained categorization and verification” published in IEEE by Yang, L., et al. in 2015. A method using the established Overfeat model, part detection and topological relationships using multiple CNN layers described in “Global Topology Constraint Network for Fine-Grained Vehicle Recognition” published by Ye Xiang et al. in 2020 IEEE method and the like may be used.

The position detection unit 400 receives the image of the vehicle determined to be parked from the detection and tracking unit 200 , and detects the position of the vehicle in the image. At this time, the position detecting unit 400 adopts the coordinate information of each vertex of the closest quadrangle of the lower edge of the vehicle as the position of the vehicle, as in the embodiment of the position detection of FIG. 2 .

)를 입력하여 구현된 주차장의 평면도 내에 새로이 맵핑/설정된 직교좌표계의 위치좌표(

)로 보정한다. 이 때, 상기 보정부(500)는 상기 측정표준(700)을 활용하여 각 좌표가 해당 좌표에 존재할 확률(수식 1)을 계산하며,The correction unit 500 corrects the position of the vehicle detected by the position detection unit 400 in consideration of the real-world magnification of the image of the camera 100 according to perspective. As an example, the vanishing point and the ground surface in the image are identified, the plan view of the parking lot is implemented through homography using the corresponding plane, and the position coordinates of the vehicle detected by the correction unit 500 (

) to input the newly mapped/set Cartesian coordinates in the floor plan of the parking lot (

) to be corrected. At this time, the correction unit 500 uses the measurement standard 700 to calculate the probability (Equation 1) that each coordinate exists in the corresponding coordinate,

[Formula 1]

차량 i의 하단부에 최인접한 bounding box 의 j번째 꼭지점 실제 위치의 평면도 내 좌표계로 치환된 좌표

The coordinates replaced by the coordinate system in the plan view of the actual position of the j-th vertex of the bounding box closest to the bottom of vehicle i

위치를 알고있는 측정표준의 평면도 내 좌표계로 치환된 좌표

Coordinates replaced by a coordinate system in a plan view of a known measurement standard

The information on the vertex having the highest value of the corresponding formula (the substitution coordinate, the j value, and the unit vector to the adjacent vertex) is transmitted to the analysis unit 600 .

The analysis unit 600 uses the vehicle specification (size) according to the model of the vehicle received from the recognition unit 300 and the position of the vehicle corrected by the correction unit 500 to determine the distance between the vehicles, the vehicle and the parking area. Any one or more of the distance between the lines and the distance between the vehicle and the parking surface boundary is calculated, and the number of parking surfaces that can be additionally parked is analyzed using the results.

The measurement standard 700 is one or more marks (symbols, figures, and letters) located on the ground of the parking lot and / or an artificial structure located in the parking lot (any one of an entry and exit breaker, a booth) ), in order to provide accurate information in configuring a probability density function in the process of being corrected in the correction unit 500, the detection and tracking unit 200 and the position detection unit 400 are configured together. Information on the size and location of the vehicle detected by the camera 100 and/or the vehicle being tracked from the image acquired by the camera 100 is secured and then provided to the compensator 500 .

In the parking status analysis method using the cctv camera image according to the present invention, the image shooting step 10 is the camera 100 to secure the image of the parking lot including the parking surface 101 and the entry zone 102 The detection and tracking unit 200 , the recognition unit 300 , the position detection unit 400 , the correction unit 500 , and the analysis unit 600 that detects and tracks a vehicle entering the parking lot can be implemented easily by utilizing it. , the measurement standard 700, etc., are respectively the detection and tracking step 20, the vehicle recognition step 30, the location detection step 40, the vehicle location correction step 50, the parking surface analysis step 60, the measurement standard detection step As a component (step) corresponding to (70), the technical content is substantially the same, and based on the description of the parking status analysis system using the CCTV camera image, a person with ordinary knowledge in the technical field to which this technology belongs It can be implemented easily.

Claims (8)

One or more cameras 100 for taking an image of the parking lot including the parking surface 101 and the entry zone (102);

a detection and tracking unit 200 for detecting and tracking a vehicle entering the parking lot through the entry zone 102 from the image taken by the camera 100;

a recognition unit 300 for recognizing the size of the vehicle being tracked by the detection and tracking unit 200;

a position detection unit 400 for detecting the position of the vehicle being tracked by the detection and tracking unit 200 when the vehicle is stopped on a parking surface;

a correction unit 500 for correcting the position of the vehicle detected by the position detection unit 400 in consideration of the real-world magnification of the image of the camera 100 according to perspective;

Any one or more of a distance between vehicles, a distance between a vehicle and a parking line, and a distance between a vehicle and a parking surface boundary using the size of the vehicle recognized by the recognition unit 300 and the position of the vehicle corrected by the correction unit 500 An analysis unit 600 that calculates and analyzes the number of parking surfaces that can be additionally parked using the calculation result; Parking status analysis system using an image, characterized in that it comprises a

The method according to claim 1, wherein
The detection and tracking unit 200 and the position detection unit 400 are located in the parking lot for the purpose of more accurately detecting the size and position of the vehicle and the measurement standard 700 that knows the size and position; Parking status analysis system using an image, characterized in that it is composed
The method according to claim 2, wherein
The measurement standard 700 is a parking status analysis system using an image, characterized in that one or more marks are located on the ground of the parking lot and/or on the surface of an artificial structure located in the parking lot.
According to claim 3,
Parking status analysis system using an image, characterized in that the mark is any one of a symbol, a figure, and a character
The method according to claim 2, wherein
The measurement standard 700 is a parking status analysis system using an image, characterized in that any one of an entrance breaker, an exit breaker, and a booth
An image taking step (10) of securing an image of the parking lot including the parking surface (101) and the entry zone (102);

a detection and tracking step (20) of detecting and tracking a vehicle entering the parking lot through the entry zone (102) from the image obtained in the image capturing step (10);

a vehicle recognition step (30) of recognizing the size of the vehicle detected in the detection and tracking step (20);

a position detection step (40) of detecting the position when the vehicle being tracked in the detection and tracking step stops on a parking surface;

a vehicle position correction step (50) of correcting the position of the vehicle detected by the position detection step (40) in consideration of the magnification of the image obtained in the image capturing step (10) according to perspective;

The distance between vehicles, the distance between the vehicle and the parking line, and the boundary between the vehicle and the parking surface using the size of the vehicle according to the model of the vehicle recognized by the vehicle recognition step 30 and the location of the vehicle corrected by the vehicle location correction step Calculating any one or more of the distances and analyzing the parking surface using the calculation result to analyze the number of parking surfaces where additional parking is possible (60); Way
7. The method according to claim 6,
The detection and tracking step 20 and the position detection step 40 use the image characterized in that it further comprises a measurement standard detection step 70 for more accurately detecting the size and location of the vehicle. Parking status analysis method
The method according to claim 7,
The measurement standard detection step 70 is a parking status analysis method using an image, characterized in that detecting the measurement standard 700, which is one or more marks located on the ground of the parking lot and/or on the surface of an artificial structure located in the parking lot

Images