KR20150034398A - A Parking Event Detection System Based on Object Recognition - Google Patents

Abstract

The present invention relates to a parking lot event detecting system based on an object recognition. The object recognition-based parking lot event detecting system of the present invention includes: a CCTV (10) to collect real-time images; an analysis server (20) which analyzes a real-time image received from the CCTV (10); and an operation terminal (30) and a voice expression device (40) to notify an event of the image analyzed by the analysis server (20) to an operator. The analysis server (20) performs: a setting of an area detection for a real-time image reception; a real-time CCTV image collection; a background recognition of the collected image; and a notification to an operator when performing a detection of an event generation by an object division (object recognition) and an object tracking. Thereby the present invention is able to: reduce time consumed when people using a parking lot unnecessarily wander in the parking lot; increase the efficiency of a parking usage by rapidly guiding to an empty parking space; and increase safety and minimize damage through a rapid detection of an event situation in the parking lot.

Description

[0001] The present invention relates to an object recognition based parking event detection system,

The present invention relates to a parking event detection system based on object recognition, and more specifically, to a parking lot event detection system based on object recognition, in which the time for wandering a parking lot unnecessarily to people using a parking lot is reduced, The present invention also relates to a parking event detection system based on an object recognition method for providing safety and minimizing loss through rapid detection of an event situation in a parking lot.

The service using intelligent image in the parking lot can be divided into the representative areas such as information collection about the parking area in the parking lot, monitoring of the parking lot passage, and vehicle information guidance for the use vehicle.

Therefore, in the related technology field, a system for detecting events by real-time image analysis through object recognition technology is used in the images collected as a function of the mobile path monitoring, and such a system is installed and used in an actual parking lot.

As a result, it is possible to reduce the time required to unnecessarily move the parking lot to the people using the parking lot, to increase the efficiency of parking use by providing parking guidance in a vacant parking space within a preset time, and to increase the safety by detecting the event situation in the parking lot , And minimization of losses.

[Related Technical Literature]

1. INTELLIGENT PARKING MANAGEMENT METHOD AND SYSTEM BASED ON CAMERA (Patent Application No. 10-2012-0019498)

2. Parking management system and method thereof (Patent Application No. 10-2012-0072169)

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and it is an object of the present invention to provide a parking space management system and a parking lot management system in which parking time is unnecessarily reduced to people using a parking lot, The present invention is to provide a parking event detection system based on object recognition in order to provide an increase in safety through rapid detection of a vehicle, a minimization of loss, and the like.

The present invention also provides an object recognition-based parking event detection system for reducing the expense due to an automation system, reducing operator fatigue, and providing a system that meets the increasing demand for a convenient and quick parking guidance system.

The present invention also provides an object recognition-based parking event detection system for enabling detection of an illegal parking (passage parking) in a parking lot including an unexpected situation in a parking lot, a riding car or a rover, .

However, the objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

According to an embodiment of the present invention, there is provided an object recognition based parking lot event detection system including: a CCTV (10) for real-time image collection; An analysis server 20 for analyzing the real-time image received from the CCTV 10; And an operating terminal (30) and a voice display device (40) for notifying an operator of an event analyzed by the analysis server (20); The parking lot management system according to claim 1, wherein the analysis server (20) comprises: an area detection setting unit for receiving the real time image, a real time CCTV image collection, a background recognition for the collected image, And notifying the operator when event occurrence detection is performed according to the object tracking.

At this time, the analysis server 20 performs foreground detection for recognizing the image as a background in the case of the background recognition and the object classification (object recognition), and object separation for learning the background and separating the objects from the learned background .

In addition, in the case of the object tracking, the analysis server 20 divides the object tracked by the category including the vehicle, the person, and the fire and determines the object, and tracks the object by reflecting the property of the determined object .

In addition, the analysis server 20 judges a stopped vehicle as being within a predetermined time over detection area at the time of performing the event determination in the event occurrence detection, judges a person who wanders in the parking lot in the night time zone, And a fire event for judging a dangerous situation in the parking lot.

In addition, the analysis server 20 performs region segmentation and image downsampling at the time of object recognition, performs background modeling using a Gaussian Mix Model, removes noise using a median filter, And then performs object detection through label detection and contour detection in that order.

In addition, the analysis server 20 performs a process of considering regions, which are connected to the currently detected pixels of the labeling at the time of label detection, or regions having the same color range as one object, (N is a natural number of 2 or more) from 1 to n in order to obtain the contour information of the object to be labeled, the process of finding the outline by the corresponding number is repeated.

In addition, the analysis server 20 calculates a movement distance, a movement speed, and a magnitude variation of an object being tracked to determine an event when the event occurrence is detected, A calculation process, and an event generation condition analysis process for detecting an event when a change coinciding with a preset condition (movement distance, movement speed, size variation) occurs in the object to be traced.

The parking lot event detection system based on object recognition according to the embodiment of the present invention reduces the time for unnecessarily wandering the parking lot to the people using the parking lot and quickly introduces parking guidance to the free parking space, In addition, it provides the safety enhancement and the minimization of loss through prompt detection of the event situation in the parking lot.

In addition, the object recognition-based parking event detection system according to another embodiment of the present invention provides a system that reduces the expense due to the automation system, reduces the fatigue of the operator, and meets the increasing demand of a convenient and quick parking induction system .

In addition, the object recognition-based parking event detection system according to another embodiment of the present invention can be applied to a parking lot event detection system including a parking lot including an unexpected situation in a parking lot, illegal parking in a parking lot, And the like.

1 is a diagram illustrating a physical configuration of a system for detecting a parking lot event based on an object recognition according to an embodiment of the present invention.
FIG. 2 is a flowchart illustrating an overall process performed on an object recognition-based parking lot event detection system according to an embodiment of the present invention.
FIG. 3 is a flowchart illustrating an object recognition algorithm in an object recognition-based parking lot event detection system according to an exemplary embodiment of the present invention. Referring to FIG.
4 is a diagram for explaining the labeling process among the object recognition algorithms of FIG.
FIG. 5 is a diagram for explaining a process of acquiring contour information for an object among the object recognition algorithms of FIG. 3. FIG.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a detailed description of preferred embodiments of the present invention will be given with reference to the accompanying drawings. In the following description of the present invention, detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

In the present specification, when any one element 'transmits' data or signals to another element, the element can transmit the data or signal directly to the other element, and through at least one other element Data or signal can be transmitted to another component.

1 is a diagram illustrating a physical configuration of a system for detecting a parking lot event based on an object recognition according to an embodiment of the present invention. Referring to FIG. 1, the parking event detection system based on object recognition includes a CCTV 10 for real-time image collection, an analysis server 20 for analyzing the CCTV 10, an operation terminal (not shown) 30) and a voice display device (40).

Here, the transmission of the CCTV image for real-time image capturing is performed by using the transmission method using TCP / IP rather than the transmission using the conventional coaxial cable, so that the system configuration at the site is simplified and maintenance is facilitated .

On the other hand, by adopting the server system as the image analysis by the analysis server 20, it is possible to take immediate action in the management center (disaster prevention room) in case of a failure, and to transmit and receive data with the operation terminal 30 also by TCP / IP communication So that the configuration in the same intranet is easy.

FIG. 2 is a flowchart illustrating an overall process performed on an object recognition-based parking lot event detection system according to an embodiment of the present invention. Referring to FIG. 2, in the parking lot event detection system based on the object recognition, the analysis server 20 logically collects CCTV images transmitted from the CCTV 10 in real time and finally notifies the operator of the CCTV images. The object recognition step S4, the object tracking step S5, the event occurrence detection step S6, and the notification process to the operator (S7).

More specifically, in the background recognition step (S3) and the object classification step (S4), the analysis server 20 learns foreground detection techniques for recognizing images as backgrounds, and object separation technology for separating objects from learned backgrounds Is used.

In the object tracing process (S5), it is preferable that the object can be traced by reflecting characteristics of the object judged as a technology for judging the object, which is judged by the object, by a vehicle, a person, a fire or the like.

In the case of event determination in the event occurrence detection process (S6), it is determined that the vehicle stopped in the detection area for a predetermined time or longer, the person who wanders in the parking lot in the night time zone, It is preferable to classify them into events and notify the operator.

FIG. 3 is a flowchart illustrating an object recognition algorithm in an object recognition-based parking lot event detection system according to an exemplary embodiment of the present invention. Referring to FIG. 1 to 3, a description will be given of a specific process in which the analysis server 20 is a main body. In general, the object recognition algorithm includes an image acquisition process S10, an object recognition process S20, an object tracking algorithm S30, and an event determination algorithm S40.

<Object recognition process ( S20 )>

In the area segmentation and image downsampling in step S21 of the object recognition process (S20) after the image acquisition process (S10), since the background detection algorithm for image analysis requires a large amount of computation, The higher the cost, the more exponentially the cost of the analytical system increases. Accordingly, the analysis server 20 performs downsampling stepwise so that image analysis of as many channels as possible can be performed in a general-purpose CPU without decreasing the performance of the image analysis.

When comparing the results of the image analysis by the step down downsampling, it is possible to analyze about 8 channels using the I7 class CPU without degrading the performance of the image analysis when using the D1 resolution, and in the case of the CIF image A problem of low object detection probability such as falling objects is derived. In case of CIF class image, it is possible to solve the problem of degradation of analytical reliability by applying the method of dividing the region for the remote portion and processing it with high resolution.

Next, the background modeling using the Gaussian Mix Model of step S22 will be described. The analysis server 20 of the present invention uses a Gaussian Mixture Model background removal method as a background removal method for moving object detection, which is robust against noises such as shaking or rolling waves of branches. The Gaussian mixture model used in the present invention utilizes a method of dividing foreground and background by modeling each pixel of a background image using a Gaussian probability distribution.

Meanwhile, the analysis server 20 preferably applies a Gaussian MIX model of the LAB color model. More specifically, when a Gaussian mixture model (GMM) is performed using only a luminance component of an image, relatively stable results can be obtained in the case where there is no abrupt illumination change while the computational load is small. Therefore, Is effective. On the other hand, in the case of the outdoor parking lot, since the result is unstable due to the illumination change, the Gaussian mixture model (GMM) for the color component is simultaneously performed and the result of the Gaussian mixture model (GMM) This is because the result is obtained.

Next, the noise removal process using the median filter of step S23 performed after step S22 will be described. In the noise removal process using the median filter of step S23, the foreground image is a binarized image in which the background is separated, but noise is mixed in the foreground image. Noise is a set of one or two small unit pixels, and this noise is called "Salt-and-Pepper" noise.

In the present invention, a method of removing noise by the analysis server 20 uses a morphology method and a method using a median filter, and a median filtering method having a greater noise removal effect than a calculation amount is more effective.

If the label detection of the step S24 and the object extraction using the contour detection algorithm are performed after the step S23, the analysis server 20 is connected to the currently detected pixel of the labeling, We assume that the regions are considered as one object. The labeling is performed in the following process as shown in FIG.

In order to obtain the contour information of the object, if the object of the labeled object is 1 to n (n is a natural number of 2 or more), the process of finding the outline by the corresponding number is repeated. It is possible to carry out the process as shown in FIG.

Therefore, it is desirable to simplify the bounding box that surrounds the contour line to the final target object because the amount of coordinate data is large and the calculation process for tracking is complicated.

<Object tracking algorithm ( S30 )>

The object tracking algorithm (S30) after the sequential processing for the image acquisition process (S10) and the object recognition process (S20) will be described in detail. In the present invention, various object tracking methods are used in order to ensure practicality by reducing the amount of calculation compared with the accuracy of tracking.

More specifically, 'Block Matching Algorithm' which judges and tracks block similarity of objects, 'MEAN Shift Algorithm' which uses color based tracking method, 'CAM Shift Algorithm' which uses color based tracking method, 'Motion Template Algorithm' , A dense optical flow algorithm that tracks changes in light intensity over the entire image, and an LKT method that extracts feature points and tracks the extracted feature points.

First of all, the block matching method is a method used mainly in predicting the motion of an object in a moving picture. It is assumed that there are only two-dimensional moving motions in two consecutive moving picture frames. Mean Absolute Difference (MAD) is mainly used to evaluate the similarity of two blocks. In the block matching motion estimation method, a method of calculating an average absolute value difference for all displacements around a specific block is called a full search method. Using the global search task, we can find the most accurate block matching based motion vector. However, as the window size increases, the global search method requires a long computation time.

Next, the Mean Shift algorithm is a method of moving the center to the highest similarity point by comparing the feature points in the region of interest and the surrounding features based on the color information. In the case of the mean shift algorithm, it is difficult to determine the size of the search window. Therefore, it affects the performance of the image tracking which needs to appropriately change the size or shape of the search window according to the shape change. Is rapidly deteriorating. Also, since the amount of computation increases as the number of objects to be traced increases, it is not practical for a system in which a plurality of objects must be tracked at the same time.

Next, the Cam Shift algorithm is an algorithm that optimizes the size of the search window by compensating for the disadvantage of the mean shift algorithm that can not appropriately change the size or shape of the search window. The Mean Shit algorithm is performed in the following steps of Step-1 to Step-5. First, the initial position and size of the search window are determined in step-1, the center of gravity distribution in the search window is found in step-2, the MEAN shift algorithm is repeated in step-3, The position and size are changed to the center of the distribution of darkness obtained in 3), and then step-2 to step-4 are repeatedly performed in step-5. Since the Cam Shift algorithm can obtain an improved result compared to the mean shift algorithm, but it can not overcome the problem that the amount of computation increases as the number of objects to be traced increases, it is practical for a system in which a plurality of objects must be tracked simultaneously .

Next, when we look at the Motion Template algorithm, it is one of the Temporal Templates, which corresponds to an algorithm that tracks human motion. The Motion Template algorithm represents the scalar image of the black and white contrast of the area of the object's recent movement, and it can calculate the movement trajectory and direction of the object with this scalar image. However, the Motion Template algorithm has a disadvantage in that it is not suitable for detecting sudden situation because it has a small computational complexity but it can not be traced continuously to a non - moving object.

Next, in the dense optical flow algorithm, an algorithm that estimates a motion pattern that occurs in an object, a surface, and an outline from a scene viewed by an observer and an observer. In this case, the computation amount is large. In a rare optical flow method, The accuracy is lowered.

In the object tracking (S31) by the simplified block matching method among the object tracking algorithm (S30), when the size and the overlap ratio of the bounding box information extracted through the first object recognition process are examined, It is considered as the same object and it is excluded from the application of the detailed tracking algorithm, and the block matching method is applied only to the object which can not be separated from the primary, thereby enabling efficient tracking while reducing the amount of computation. The object tracking process proposed by the present invention is performed by steps 1 to 6. First, the initial position and size of the search window are determined by using the bounding box information for the extracted object, and the same object is firstly separated by comparing the size of the bounding box information with the size of the bounding box information in step-2, 3, the same object is separated by the block matching method. In case of the object which can not be separated into the second by Step-4, it is compared with the object information stored in the queue, And the objects that can not be separated through Step-5 to Step-4 are registered in the queue as a new object, and the process from Step-2 to Step-4 is repeated in the last Step-6.

After step S31, correction is performed based on the object size of the camera view (S32), filtering of objects by category is performed (S33), and event detection area determination of step S34 is performed. When performing the event detection area discrimination in step S34, it is possible to reduce the computation load by excluding the area where there is no possibility of moving objects, such as the ceiling, from the tracking object, and the area for event detection is also limited to the area of interest It is desirable to reduce the detection probability and reduce the amount of computation.

<Event identification algorithm ( S40 )>

The image acquisition process S10, the object recognition process S20, and the event determination algorithm S40 after the object tracking algorithm S30 will be described.

The process of calculating the moving distance, speed, and size variation of the object during the event determination algorithm (S40) will be described. In order to identify the event, the moving distance, the moving speed, .

In the case of the moving distance, the moving distances of the corners along with the center distance are individually calculated to improve the accuracy of the event discrimination by a minute change.

In the case of the moving speed, by calculating the distance from the first identified point, it is possible to provide a deviation that occurs when the speed is calculated frame by frame.

Changes in size can be used to identify events that change irregularly along the time axis, such as fire, by managing area and aspect ratio changes together.

In the event occurrence condition analysis process (S42) during the event determination algorithm (S40), an event is detected when a change that coincides with a given condition occurs in the object to be tracked. Here, the event detection method applies a method of analyzing the following conditions for each event. That is, the event detection of the present invention can be applied to a stop vehicle (an object that does not move for more than a specified time among objects classified as a vehicle), a pedestrian (an object that roams for more than a specified time among objects classified as a person) (Objects that pass through a specified boundary of objects), falling objects (objects that are not classified within an unknown period of time and that do not move for a specified time or more), and fire It is possible.

Here, it is possible to designate the detection schedule for each event type so as to take into consideration the situation depending on the time, and it is possible to apply the individual analysis parameter to the illumination change situation such as the weather.

In the event alarm and storage (S43) during the event determination algorithm (S40), it is possible to notify the situation through audio output to the voice display device (40) or operation of a warning light during event detection, Allow video for events to be stored in the database. And provides the effect of post-management of event occurrence status through retrieval and inquiry of stored event data.

The present invention can also be embodied as computer-readable codes on a computer-readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored.

Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device and the like, and also implemented in the form of a carrier wave (for example, transmission over the Internet) .

The computer readable recording medium may also be distributed over a networked computer system so that computer readable code can be stored and executed in a distributed manner. And functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers skilled in the art to which the present invention pertains.

As described above, preferred embodiments of the present invention have been disclosed in the present specification and drawings, and although specific terms have been used, they have been used only in a general sense to easily describe the technical contents of the present invention and to facilitate understanding of the invention , And are not intended to limit the scope of the present invention. It is to be understood by those skilled in the art that other modifications based on the technical idea of the present invention are possible in addition to the embodiments disclosed herein.

10: CCTV
20: Analysis server
30: operating terminal
40: voice output device

Claims (7)

CCTV (10) for real time image collection;
An analysis server 20 for analyzing the real-time image received from the CCTV 10; And
An operating terminal (30) and a voice display device (40) for notifying an operator of an event analyzed by the analysis server (20); The parking lot event detection system according to claim 1, wherein the analysis server (20)
The notification to the operator is performed when the event occurrence detection is performed according to the area detection setting for real-time image reception, the real-time CCTV image collection, the background recognition of the collected image, the object classification (object recognition) Based parking event detection system.
The system according to claim 1, wherein the analysis server (20)
Wherein the object recognition is performed through foreground detection for recognizing the image as a background in the case of the background recognition and the object classification (object recognition), and object separation for learning the background and separating the object from the learned background. Event detection system.
The system according to claim 2, wherein the analysis server (20)
Wherein the object tracking unit classifies the object tracked by the category including the vehicle, the person, and the fire, and tracks the object by reflecting the characteristics of the determined object. .
The system according to claim 3, wherein the analysis server (20)
A fire event for judging a dangerous situation in a parking lot due to a smoke, judging a person who stops in the detection area for a predetermined time or more at the time of performing the event determination in the event occurrence detection, And notifying the operator of the parking lot event.
The system according to claim 1, wherein the analysis server (20)
It performs area segmentation and image downsampling in object recognition, performs background modeling using Gaussian Mix Model, performs noise removal using Median Filter, and performs label detection and contour detection. And the object extraction is performed in order.
The system according to claim 5, wherein the analysis server (20)
Performing a process of recognizing, as a single object, regions having the same color range or being connected to the currently detected pixels of the labeling upon label detection,
The contour detection method according to claim 1, wherein, in order to obtain contour information of the object at the time of detecting the contour, if the object of the labeled object is 1 to n (n is a natural number of 2 or more) Recognition based parking event detection system.
The system according to claim 1, wherein the analysis server (20)
A moving distance, a speed, and a size change amount calculation step of calculating an object moving distance, a moving speed, and a size change amount to an object being tracked in order to determine an event when the event occurrence is detected, And an event occurrence condition analysis process for detecting an event when a change that coincides with a condition (movement distance, movement speed, size change amount) occurs.

Images