KR20190037770A - Apparatus, system and method for real-time parking lot video object recognition from cctv parking lot video data - Google Patents

Abstract

The present invention relates to an apparatus for identifying an image object from a parking section in real time using CCTV image information of a parking lot, which is capable of easily checking whether the parking section is occupied, and a system and a method thereof. According to the present invention, the system for identifying an image object in a parking section in real time using CCTV image information comprises: a CCTV-based device photographing a parking/stoppage state of a parking lot in real time; an external source including an open data source and also connected to at least a real-time parking section image object identification device installed in a plurality of ordinary parking lots; a parking lot management device installed in a predetermined parking lot; and the parking section image object identification device. The parking section image object identification device comprises: an input module including an external source data input unit to input open data including at least image information related to an ordinary parking lot from the open source and a real-time data input unit to input real-time image data from the CCTV-based device of the predetermined parking lot; an artificial intelligence module using the open data inputted from the external source data input unit of the input module to learn a shape and environment of an expected parking section so as to generate trained neural network artificial intelligence capable of determining whether an arbitrary parking section is occupied; and a control module applying the parking lot image, which is provided in real time from the CCTV-based device of the predetermined parking lot among the ordinary parking lots, to the trained neural network artificial intelligence to determine in real time whether the parking sections of the predetermined parking lot are occupied and, based on the determination result, providing parking section state information in real time representing whether the parking sections of the predetermined parking lot are occupied.

Description

FIELD OF THE INVENTION [0001] The present invention relates to an apparatus, system and method for recognizing a real-time parking space image using a CCTV parking lot image information,

The present invention relates to an apparatus, a system and a method for recognizing a parking space image object using CCTV parking space image information. More particularly, the present invention relates to a parking space space object recognition method, The learned neural network has the ability to accumulate the intelligence information, and based on the learned neural network artificial intelligence, from the real-time information input through the CCTV of the specific parking lot, the parking lot of the parking lot is empty parking lot, The present invention relates to a parking box video object recognition apparatus, a parking box video object recognition system, and a parking box video object recognition method using CCTV parking lot image information providing a function of determining whether or not a parking space is available.

The terms used in the present invention are as follows.

Empty parking spaces: Parking spaces where the vehicle is not currently parked

Carpark: Carpark where the car is currently parked

In the present invention, by using the open CCTV image information as a known open data of a public parking lot photographed in advance, it is possible to use the artificial intelligent learning to make the parking space empty The learned neural network artificial intelligence capable of judging whether or not it is a parking space and whether or not it is a parking space is accumulated and provided.

A specific parking lot: An individual parking lot that provides CCTV image information in real time. By applying the learned neural network artificial intelligence of the present invention to real-time CCTV image information, it is possible to determine which parking spaces are empty parking lots or cold parking spaces It can be judged in real time.

Open source: CCTV-related pre-recorded video, video data, and analysis data input into the parking lot object recognition system or parking space video object recognition device from an external source.

Conventionally, in a conventional parking lot, a technique of determining whether there is an empty parking space or a cold parking space includes a sensor detecting method installed in each parking space, and a camera is installed in each parking space or in a group of one to three or four parking spaces. There is a way.

For example, in Japanese Patent Application Laid-Open No. 10-2016-0077977 (published on Jul. 26, 2014), a direct light source is disposed on the bottom of a parking compartment, and a target area on which a light receiving element for recognizing a straight- The target area recognizes the light, and when the vehicle is parked, the light from the rectilinear light source is blocked by the parked vehicle. Therefore, it is possible to determine whether or not the ascending car is an empty car park by using this difference.

Another example is that in the case of Patent Registration No. 10-0956400 (disclosed on Apr. 28, 2010), one camera is installed for each of the three parking spaces small group, and three parking spaces for one parking space And provides a technique for recognizing whether or not it is a car parking lot.

In addition, various types of conventionally disclosed technologies have been proposed, but basically, the conventional art has a problem in that a large number of sensors such as a sensor and a camera are used for each parking space or for a small group (three or four parking spaces) A unit sensing device is disposed.

Therefore, in the prior art, the parking lot management system is provided with a plurality of unit sensing devices for recognizing whether or not the parking spaces are parked, so that it is necessary to construct a huge sensing device system and physically, , And the material input is considerably large. Particularly, a lot of manpower is put into managing the contamination or failure of the photosensor lens due to dust, vibration, etc. for each individual apparatus, and the failure rate which can not judge whether or not the parking space is substantially empty parking space or empty parking space The distrust of the parking management system is also high. In particular, the camera system that identifies whether a parking space is parked by a camera for each small group of parking spaces is inferior in terms of efficiency, economy, and management.

In addition, in the case of outdoor, in order to install a unit monitoring device for detecting whether or not a parking space is parked, difficulties due to environmental constraints such as temperature, humidity, and theft and difficulty in human and material management, The installation cost and the management cost are further increased because the sensor or the camera must be arranged and managed in preparation for the environmental conditions.

As a result of this, many large-sized outdoor parking lots are often used by drivers to search for empty parking spaces. As a result, It is difficult to provide an effective parking guide for the user.

Patent Registration No. 10-2016-0077977 (published on Jul. 07, 2016) Patent Registration No. 10-0956400 (Published on Apr. 28, 2010)

SUMMARY OF THE INVENTION The present invention has been made in order to solve the above problems of the prior art, and it is an object of the present invention to provide a parking space management system, CCTV that can distinguish between empty parking spaces and cold parking spaces in a parking lot, regardless of indoor or outdoor, without installing a sensing device, An object recognition apparatus, a system, and a method.

Another object of the present invention is to provide a parking space management system and a parking lot management system, which can drastically reduce the installation cost and the maintenance cost of the related devices for determining whether or not the parking space is parked, And a system and method for recognizing a parking space image object using CCTV parking space image information which can reliably provide identification of an empty parking space with a low failure rate.

Another object of the present invention is to provide a CCTV parking lot video image capable of reducing the wobble of a driver or a vehicle necessary for parking and simplifying the arrangement of the installation device to reduce exhaust pollution, noise pollution, The present invention also provides an apparatus, system, and method for recognizing a parking space video object using information.

BRIEF DESCRIPTION OF THE DRAWINGS The foregoing and other objects are more clearly understood from the following detailed description taken in conjunction with the accompanying drawings.

An embodiment of the present invention for realizing the above object is an apparatus for recognizing a real-time parking space image object using CCTV image information, which is used in a public parking lot, and which has open data including at least image information related to the public parking lot, An input module including an external source data input unit for inputting the open data from an external source provided in the form of a real time data input unit and a real time data input unit for inputting real time image data from a CCTV system of a specific parking lot; A learned neural network having a function capable of determining whether or not the vehicle is parked with respect to an arbitrary parking space by learning the shape of the expected parking space and the parking space environment using the open data input from the external source data input unit of the input module, An artificial intelligence module for generating artificial intelligence; A parking lot image provided in real time from a CCTV system of a specific parking lot among the public parking lots is applied to the learned neural network artificial intelligence to judge in real time whether or not the parking lot is parked in the specific parking lot, And a parking space state determiner for providing parking space state information indicating whether or not parking spaces in the specific parking space are parked in real time.

Preferably, the artificial intelligence unit uses, as an open data from an external source data input unit of the input module, a CCTV image related to the parking status of the public parking lot as a still image or a moving image, A shape learning pre-preparation unit for determining a shape around a box, generating and accumulating a plurality of learning shape data based on the determined shape; And a time-based change using a plurality of images provided from at least one of the learning shape data generated from the shape learning preparation unit and the still image or moving image captured from the CCTV, An environment learning preparation unit for generating and accumulating a learning complex data set reflecting environmental changes; Based on the learning shape data from the preparation unit for pre-shape learning and the compound learning data set from the pre-environment learning preparation unit, determines whether there is an empty parking space or a cold parking space in the parking spaces of the common parking lots Wherein the learning shape data and the learning complex data set are generated based on data processed for use as an input for a learning method in a parking space learning unit, Wherein the parking space learning unit and the parking space state determination unit learn or determine whether the parking space is an empty parking space or a cold parking space using machine learning using at least one of autonomous learning and map learning, The learning includes at least one of Deep Learning and Shallow learning.

Preferably, the control module is further configured to display an alarm, a message, a color change, a polygonal image, and the like reflecting the change of the empty parking space and the free parking space in the specific parking lot, based on the real time parking space state information provided from the parking space state determination portion And a parking space status change notification unit for providing the parking space status change alarm information directly to the parking space management system or providing parking space status change alarm information for use in the parking lot management apparatus, Is used for at least one of a speaker installed in a parking lot management apparatus, a notification light informer, an LED electric sign board and a GUI monitor, or a combination thereof, or used as an empty parking space guidance information through an app of a driver's vehicle or a driver's mobile terminal.

Preferably, the pre-shape learning preparation unit displays on the GUI using at least one of snapshot picture data and moving picture data, which are CCTV parking lot image data input to the input module, so that the parking space area recognized by the manager is one of the polygons A parking space manually setting GUI mechanism for designating an area and generating the designated area as learning shape data; And a parking space automatic setting GUI mechanism for automatically recognizing a location and an area of parking spaces in the CCTV parking lot image data input to the input module and generating learning image data, And an environment function capable of manually correcting the learning shape data created by the automatic setting GUI mechanism, wherein the control module includes a graphic processing unit for supporting the parking space manually setting GUI mechanism and the parking space automatic setting GUI mechanism .

Preferably, the learning complex data set of the pre-environment learning preparation unit includes an open CCTV image as various known open data regarding the parking status of the plurality of parking lots already photographed, and the open CCTV image includes at least the environment change, Wherein the weather change includes at least a change of clear, cloudy, rain, snow, and wind, and the ambient object change includes at least a change of a moving or fixed object in a parking lot or around a parking space, Based change includes at least a minus sign, a color, and a seasonal change of the surrounding environment over time, and the parking space learning unit receives the learning complex data set including the open CCTV image and learns the actual shape of the specific parking space Negative learning (n learning) which learns the deformed shape of the parking space according to the environment change wherein the parking space image object recognizing device carries out machine learning to improve the perfection of car park object recognition through eg learning, and the parking lot car image object recognizing device includes data, a data set, and information generated from the input module, the AI module, And a comprehensive database for storing and providing each component of the module, a specific parking management device or an external source as necessary.

Preferably, the artificial intelligence unit updates the learned neural network artificial intelligence based on the open data from the external source and the real-time image information provided from the CCTV system of the specific parking lot, thereby determining whether or not the parking space in the specific parking lot is parked Increases learning ability and provides increased learning ability as an external source for learning neural network artificial intelligence upgrade of other parking box video object recognition devices.

According to another embodiment of the present invention, there is provided a real-time parking space image object recognition system using CCTV image information used in parking lot parking information, A CCTV system device for accumulating the video signal; And an open data source capable of providing an open data including at least image information related to the general parking lot in advance in a preliminarily provided form, as well as at least connecting to at least a real parking area video object recognition device installed in a plurality of general parking windows With an external source; A parking lot management device installed in a specific parking window; And a parking space image object recognition device for recognizing whether a parking space of a specific parking space is parked based on the CCTV image information displayed from the CCTV system, An external source data input unit for inputting the open data from an external source that provides an open data including at least information in advance and a real time data input unit for inputting real time image data from a CCTV system of a specific parking lot An input module; The learned neural network artificial intelligence which can judge whether or not the vehicle is parked with respect to a random parking space by learning the expected shape of the parking space and the parking space environment using the open data inputted from the external source data input part of the input module An artificial intelligence module to generate; A parking lot image provided in real time from a CCTV system of a specific parking lot among the public parking lots is applied to the learned neural network artificial intelligence to judge in real time whether or not the parking lot is parked in the specific parking lot, And a parking space state determiner for constantly providing parking space state information indicating whether parking spaces in the specific parking space are parked in real time at all times.

Preferably, in the system, the artificial intelligence unit uses, as an open data from an external source data input unit of the input module, a CCTV image related to a parking lot status of a public parking lot as a still image or a moving image, A shape learning pre-preparation unit for determining a shape of a box and a periphery of a parking space, and generating and storing a plurality of learning shape data based on the determined shape; And a time-based change using a plurality of images provided from at least one of the learning shape data generated from the shape learning preparation unit and the still image or moving image captured from the CCTV, An environment learning preparation unit for generating and accumulating a learning complex data set reflecting environmental changes; Based on the learning shape data from the preparation unit for pre-shape learning and the compound learning data set from the pre-environment learning preparation unit, determines whether there is an empty parking space or a cold parking space in the parking spaces of the common parking lots Wherein the learning shape data and the learning complex data set are data processed so as to be used as an input for a learning method in a parking space learning unit Wherein the parking space learning unit and the parking space state determination unit learn or determine whether the parking space is an empty parking space or a cold parking space using machine learning using at least one of autonomous learning and map learning, Includes at least one of Deep Learning and Shallow learning.

Preferably, in the system, the control module is configured to display an alarm, a message, a color, and the like reflecting the change of empty parking spaces and cold parking spaces in the specific parking lot, based on the real- And a parking space status change notification unit for directly providing the parking space change status, the parking space status change notification, or the parking space status change notification, The status change notification information may be used for at least one of a speaker installed in the parking lot management apparatus, a notification light informer, an LED electric signboard, and a GUI monitor, or a combination thereof, or may be used for a parking lot .

Preferably, in the system, the pre-shape learning preparation unit displays on the GUI using at least one of snapshot photograph data and moving picture data, which are CCTV parking lot image data input to the input module, A parking space manually setting GUI area for specifying an area as one of the polygons and generating the designated area as learning shape data; And a parking space automatic setting GUI mechanism for automatically recognizing a location and an area of parking spaces in the CCTV parking lot image data input to the input module and generating learning image data, And an environment function capable of manually correcting the learning shape data created by the automatic setting GUI mechanism, wherein the control module includes a graphic processing unit for supporting the parking space manually setting GUI mechanism and the parking space automatic setting GUI mechanism .

Preferably, in the system, the learning complex data set of pre-environment learning preparations includes a CCTV image as various known open data relating to the parking status of a plurality of parking lots already photographed, and the open CCTV image Wherein the weather change includes at least a change of fine, cloudy, rain, snow, and wind, and the ambient object change includes at least a change of a moving or fixed object in a parking lot or around a parking space Wherein the time-based change includes at least a minus sign, a color, and a seasonal change of the surrounding environment over time, and the parking space learning unit receives the learning complex data set including the open CCTV image, Positive learning that learns shape and positive learning that learns the deformed shape of the parking space according to the environment change. Wherein the parking space recognizing device performs machine learning for enhancing the perfection of car park object recognition through negative learning, wherein the parking lot car image object recognizing device includes data, a data set, And a comprehensive database for storing information and providing it to each component of the module, a specific parking management device, or an external source as needed.

Preferably, in the system, the artificial intelligence unit updates the learned neural network artificial intelligence based on open data from the external source and real-time image information provided from a CCTV system of a specific parking lot, And the increased learning ability is provided as an external source for the learned neural network artificial intelligence upgrade of other parking box video object recognition devices.

According to another embodiment of the present invention, there is provided a real-time parking space image object recognition system using a real-time parking space image object recognition system using CCTV image information used for parking lot parking information, A CCTV system for photographing in real time and accumulating the image information; And an open data source capable of providing an open data including at least image information related to the general parking lot in advance in a preliminarily provided form, as well as at least connecting to at least a real parking area video object recognition device installed in a plurality of general parking windows With an external source; A parking lot management device installed in a specific parking window; And a parking space image object recognition device for recognizing whether a parking space of a specific parking space is parked based on the CCTV image information displayed from the CCTV system, An open data input step of inputting the open data from an external source provided in advance with an open data including at least image information related to the general parking lot; The artificial intelligence module of the video object recognizing device learns the shape of the expected parking space and the parking space environment by using the open data to determine whether or not the vehicle is parked in the random parking space, An artificial intelligence learning step of generating artificial intelligence; A real time data input step in which the input module inputs real time image data from a CCTV system of a specific parking lot; Wherein the parking space status determination unit of the video object recognition apparatus applies a parking space image provided in real time from the CCTV system of the specific parking lot to the learned neural network artificial intelligence to check whether or not parking of the parking spaces in the specific parking space is performed in real time And a parking lot state determination step of providing parking lot state information indicating whether or not the parking spaces in the specific parking lot are parked at all times in real time based on the determination result.

Preferably, the artificial intelligence learning step is a step in which the artificial intelligence module uses, as an open data from an open data input unit of the input module, a CCTV image related to a parking lot status of a general parking lot as a still image or a moving image, A shape learning preliminary step of determining a possible shape of a parking space and a shape of a surrounding area of a parking space, and generating and storing a plurality of learning shape data based on the determined shape; Wherein the artificial intelligence module uses a plurality of images provided from at least one of the learning shape data generated from the shape learning preparation unit and the still image or moving image captured from the CCTV, An environmental learning preparation step of generating and accumulating a learning complex data set reflecting an environmental change including at least one of the learning complex data sets; Wherein the artificial intelligence module learns through the learning based on the learning shape data in the pre-shape learning step and the complex learning data set from the pre-environment learning preparation step, Wherein the learning shape data and the learning complex data set are input for a learning method in a parking space learning unit, and wherein the learning shape data and the learning complex data set are input Wherein in the learning of the parking space and the determination of the parking space state, it is determined whether or not the parking space is an empty parking space or a cold parking space by using machine learning using at least one of autonomous learning and map learning. And the machine learning is performed by learning at least one of Deep Learning and Shallow learning Including me.

Preferably, in the method, after the parking lot state determination step, based on the real-time parking space state information provided from the parking space state determination unit, the parking space state change notification unit of the parking space image object recognition apparatus determines In the parking lot, there is a blank parking lot. A parking lot status change notification step, which provides an alarm, a message, a color change, and a polygonal image reflecting the change of the parking lot, or provides the parking lot status change alarm information for use in the parking lot management apparatus Wherein the parking lot status change notification information provided by the parking lot status change notification unit is used for at least one of a speaker installed in the parking lot management apparatus, a notification light transmitter, an LED electric sign board, and a GUI monitor or a combination thereof Or is used as empty parking space guidance information through the app of the driver's vehicle or the driver's mobile terminal.

Preferably, in the above-described method, the preparatory step for pre-shape learning is a step of manually setting a parking space of the artificial intelligence module using at least one of snapshot picture data and moving picture data, which are CCTV parking picture image data input to the input module A parking space setting GUI step of designating an area of a parking space area recognized by an administrator as one of polygons by displaying on a GUI and generating the designated area as learning shape data; Or automatically setting the parking space of the artificial intelligence module automatically generates the learning image data by automatically recognizing the location and area of the parking spaces in the CCTV parking space image data input to the input module Wherein the step of manually setting the parking space manually comprises the step of providing a correction function by which an administrator can manually correct the learning shape data created by the parking space automatic setting GUI mechanism, A manual manual setting GUI mechanism and a graphics processing unit supporting the parking space automatic setting GUI mechanism.

Preferably, in the method, the learning complex data set in the pre-environment learning preparation step includes an open CCTV image as various known open data related to the parking status of the plurality of parking lots already photographed, Wherein the weather change includes at least a change of clear, cloudy, rain, snow, and wind, and the ambient object change includes a change in a moving or fixed object in a parking lot or around a parking space Wherein the time-based change includes at least a minus sign, a color, and a seasonal change of the surrounding environment over time, the parking space learning unit receives a learning complex data set including the open CCTV image, Positive learning that learns the actual shape of the box and learning the deformed shape of the parking space according to the environment change Key performs machine learning to improve the perfection of car park object recognition through negative learning, and a comprehensive database of the car park image object recognition device is generated from the input module, the AI module, and the control module Data set, and information to be provided to each component of the module, a specific parking management device, or an external source as needed.

Preferably, the artificial intelligence unit updates the learned neural network artificial intelligence based on the open data from the external source and the real-time image information provided from the CCTV system of the specific parking lot to determine whether or not the parking space in the specific parking lot is parked Further comprising the step of increasing the functional learning ability and providing the increased learning capability to an external source for the learned neural network artificial intelligence upgrade of the other parking box image object recognition device.

According to the above configuration, the present invention can obtain the following effects.

First, it is not necessary to dispose a unit sensing device for detecting whether the parking space is empty parking lot or cold parking space for each parking space or 3-4 parking space small groups, and only the CCTV system existing in the parking lot is used It is possible to detect whether the parking space is parked, so that the unit sensing device can not be arranged for every parking space or parking space small group. The environmental burdens such as the installation cost, the management cost, and the change of the parking lot facility due to the installation can be solved at all.

Secondly, since the sensor or the camera sensitive to the environment or theft is used as the conventional unit sensing device, there are a lot of costs and restrictions in applying the parking space recognition system to the environmentally unfriendly area or the outdoor parking lot, especially the huge outdoor parking lot. The system of the present invention can be easily applied to any CCTV installation area in any environmental condition because only the conventional CCTV system installed in the parking lot is used as the sensor without using any environment-sensitive sensor or camera at all .

 Third, since the present invention basically uses a technique of not installing a unit sensing device in each parking lot in a parking lot, it is possible to eliminate a management burden problem such as frequent facility replacement due to a high failure rate generated for each unit sensing device It is possible to eliminate all the problems such as difficulty in guidance of the parking space generated due to failure of the unit sensing device, distrust of the parking space guidance system, inconvenience of the manager, and driver.

Fourthly, it is possible to greatly reduce the problem of drivers wandering in a large parking lot or in a large parking lot where indoor and outdoor are combined together.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a schematic block diagram showing a configuration of a parking space image object recognition system according to an embodiment of the present invention; FIG.
2 is an explanatory view showing a learning process of an artificial intelligence module using CCTV image information in a parking box image object recognition system according to an embodiment of the present invention.
3 is an explanatory view showing a process of determining whether a parking space is parked in real time by applying a learning result of an artificial intelligence module in a parking space image object recognition system to a specific parking space according to an embodiment of the present invention.
4 is a flowchart illustrating an operation procedure of a parking space image object recognition system according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a schematic block diagram showing a configuration of a parking space image object recognition system according to an embodiment of the present invention; FIG. Referring to the drawings, a parking space image object recognition system (hereinafter, referred to as a "recognition system") mixes input images in real time from a CCTV 1100 of a CCTV system 1000 of a parking window to determine whether the parking space is empty parking space A parking space image object recognition device 4000 for determining whether the parking space is a parking space, a CCTV system 1000 installed in a specific parking space, and a parking lot management device 2000 installed in a specific parking space.

In addition, the parking lot image object recognition system may further include an external source 5000. The external source 5000 may be a public or open form (hereinafter, "open material") of a parking-related image, a pre-recorded image or a moving image of a CCTV image outside the specific parking window, "Open Video", "Open CCTV Video"), and may include or be connected to general public media, library materials, CCTV system of general parking windows, etc.

Referring to FIG. 1, a parking space image object recognition apparatus 4000 according to an embodiment of the present invention includes an input module 4100, an artificial intelligence module 4200, a control module 4300, a comprehensive database 4400).

First, the input module 4100 has a function of inputting a CCTV image provided from an external source 5000 or a CCTV system installed in a specific parking lot or a public parking lot. More specifically, the input module 4100 includes an external source input unit 4110 for inputting known data, that is, open data, from an external source 5000 providing a photographed CCTV image or moving image, And a real-time data input unit 4120 for receiving the current parking situation from the CCTV 1100 of the CCTV system 1000 installed in the parking window into the CCTV system in real time. Open data or real time data including at least CCTV image data may be stored in the input module 4100 or may be stored in the comprehensive database 4400 so that each component may be searched and used when necessary.

The AI module (4200) develops artificial intelligence learning ability to determine whether or not the parking space is parked considering various shapes, temporal and physical surroundings related to the parking spaces of the public parking lot, It is possible to provide a general purpose artificial intelligence learned neural network artificial intelligence. Specifically, the artificial intelligence module 4200 models and estimates possible parking space shapes and surrounding shapes in a parking lot using CCTVE image data input from an external source data input unit, that is, open data, Learning preparation preparation unit 4210 for generating and accumulating a plurality of learning shape data based on the open learning data, which is also a CCTV video, on the learning shape data prepared by the shape learning preparation unit 4210, An environment learning prepara- tion unit 4220 for modeling environment changes such as weather change, surrounding object change, color based on time, shade change, etc. to generate and accumulate a learning complex data set; And the learning complex data set from the environment pre-preparation preparation unit 4220, It may include an empty parking space or by providing the trained neural network build artificial intelligence parking space learning unit (4230) to determine whether the cold parking space at.

The shape learning preparation preparation unit 4210 may also include a shape manual setting GUI mechanism 4211 and a shape setting automatic setting GUI mechanism 4212 as a graphic user interface (GUI) for determining a shape. The operation of this portion will be described in detail later with reference to Figs. 2 and 3.

The learning shape data from the shape learning preparation preparation unit 4210 and the learning complex data set from the environment learning preparation preparation unit 4220 are prepared so as to fit the input data of the parking space learning unit 4230. [ The shape learning preparation unit 4210, the environment preparation preparation unit 4220, and the parking space learning unit 4230 store the generated learning shape data, the learning complex data set, and the universal parking determination function learning information In another embodiment, the data may be stored in the comprehensive database 4400. In this case, the necessary data may be searched and used from the comprehensive database 4400 whenever necessary. The process of learning operation of the artificial intelligent parking spaces by the shape learning pre-preparation unit 4210, the environment learning preparation unit 4220, and the parking space learning unit 4230 of the artificial intelligence module 4200 will be described later with reference to FIGS. 3 will be described in more detail.

The control module 4300 basically controls the entire system. Specifically, the control module 4300 determines whether or not the present parking lot is present based on the general-purpose parking availability determination function learning information provided from the parking space learning section 4230 of the intelligent module 4200 And a parking space state determination unit 4310 for determining the parking space state of the parking space of the vehicle. That is, the parking space state determination unit 4310 receives the neural network artificial intelligence learned from the comprehensive parking space learning unit 4230 of the comprehensive DB 4400 or the artificial intelligence module 4200 and receives the neural network artificial intelligence learned from the CCTV system 1000 1100) in real time from the real-time data input unit 4120 and utilizes the function of determining whether or not the parking space has already been learned, It provides the parking space status information that specifies in real time whether or not the parking spaces of the parking window are cold parking spaces, that is, where the parking space is the cold parking space and the empty parking space.

On the other hand, based on the real-time parking space state information provided from the parking space state determination unit 4310, the control module 4300 displays an empty parking space in the specific parking space, an alarm reflecting the change of the empty parking space, Or may be provided as status change alarm information to the monitor unit 2200 and the alarm unit 2300 provided in the parking lot management apparatus 2000. [

On the other hand, the control module 4300 controls the operation of the shape setting manual setting GUI mechanism 4211 and the shape setting automatic setting GUI mechanism 4211 of the control unit 4330 having the control program of the system together with the processor and the shape learning preparation unit 4211 And a graphics processing unit 4340 having a GPU supporting the GPU.

The CCTV system 1000 includes a VMS (Vedio Management System) 1200 for processing the operation of the CCTV camera 1100 and the CCTV camera 1100 and a corresponding CCTV image, a video image DB 1300, and the like, and may be configured to be permanently stored in the comprehensive DB 4400 as needed.

The parking lot management apparatus 2000 receives the parking lot change status of the parking lot from the parking lot status determination unit 4310 and the parking lot status change notification unit 4320 A monitor unit 2200 for providing a driver or a manager with an image, a change status of a parking space of a parking lot from a parking space status determination unit 4310 and a parking space status change notification unit 4320, (Not shown).

Next, with reference to FIG. 1 to FIG. 3, a real-time parking space image object recognition system according to an embodiment of the present invention will be described focusing on an artificial intelligent learning process and a parking space state information process.

The present invention can receive the CCTV image related to the parking lot already photographed as open data from the CCTV of the public parking lot to the external source data input part 4110 of the input module 4100 through the wired / wireless communication network. In addition, video data accumulated in advance in the video DB 1300 of the CCTV apparatus 1000 or the comprehensive DB 4400 of the recognition apparatus 4000 may be used as open data. The shape learning pre-preparation unit 4210 of the artificial intelligence module 400 uses the open material to display a snapshot picture of the moving picture or moving picture in the graphic processing unit 4340 on the GUI mechanism 4211 (4212) (A square, a trapezoidal shape, or a rhombic shape) manually or automatically through the GUI mechanism 4211 (4212) to position and area of the parking spaces, And stores the generated shape data D100 in the comprehensive DB 4400. At this time, the position information of the polygon of the learning shape data may be directly displayed on the snapshot photograph, but the coordinate information may be stored and managed in a separate file (text, xlml, binary).

2 (a), the shape learning preparation unit 4210 can display a position and an area of parking spaces in which a car is parked, either manually or automatically through GUI devices 4211 and 4212 (D100a, D100b) to generate the shape learning data D100, or one of the parking spaces in the moving picture or snapshot picture (D100c) of the CCTV image full screen including a plurality of parking spaces The shape learning data D100 is generated by changing one of the line color, line thickness, and line shape of the displayed polygon to distinguish the empty parking space from the cold parking space, thereby learning the parking space.

Referring to FIG. 2B, the preprogramming unit 4220 prepares the environment for learning by using the open data (for example, the parking lot image data set) and the parking lot of the clear weather, cloudy, rain, (Environment) learning data or complex (environment) data sets by collecting environment-related data of clear, cloudy, rain, or snowy weather conditions collected from open data as long-term parking CCTV image data. The complex learning data set required for learning can enhance the learning effect of the parking space learning unit 4230 when there are almost the same number of video images for empty parking lots and cold parking spaces. In addition, the complex learning data should be able to cover a car in which the angle and size of the CCTV camera are expressed differently in various weather conditions through various image data. The open data (parking lot image data set) includes a plurality of parking lot image images (D111a to D111c) and parking spaces divided from one or more parking lots. Also, a parking space area information in one video image D111a can be distinguished from a parking space area included in the video image by including one xml file D111b with the same name.

The parking space learning unit 4230 uses shape learning data D100 according to a change in weather using a Deep Learning or a shallow learning technique and a parking lot image data set as an open data (Data set in (b) of FIG. 5B), and performs the parking lot learning so as to have the ability to grasp the state of the parking space object by inputting the complex learning data set generated by the environment pre-preparation preparation unit 4300 . That is, referring to FIG. 2, empty parking spaces are learned by inputting a blank parking space image D100a or a video image D100c in which empty parking spaces are displayed in a green polygon, and in the shape learning data D100, An image D100b or a video image D100c in which a cold parking space is displayed as a red polygon is learned, and the parking lot learning information capable of distinguishing the parking space object from the parking space image data is learned And stores it as a learned neural network artificial intelligence or as a backup in a comprehensive DB 4400. In this case, the color of the empty parking space and the cold parking space is described as an example of the use of the parking space learning section 4230 as information for distinguishing the state of whether the parking space is an empty parking space or a cold parking space in the input image , The area information of the empty parking space and the cold parking space may be stored in a separate xml file D111b in each video image D111a so that the environment pre-preparation preparation unit 4220 may form a complex learning data set.

In this case, the parking space learning unit 4230 uses a supervised learning method such as ANN (Artificial Neural Network), Boosting, and a random forest, or clustering, , An ICA (Independent Component Analysis), PCA (Principle Component Analysis), and SVD (Singular Value Decomposition).

The parking lot state determination unit 4310 determines whether the image data P100 of the CCTV 1100 of the parking lot that is actually operating based on the neural network artificial intelligence learned by the parking space learning unit 4230 is input to the input module 4120 The control unit 4120 receives the input from the real-time data input unit 4120 and distinguishes the parking space objects included in the image data by using Deep Learning or Shallow Learning techniques, And transmits the information of the parking spaces predicted and determined by the parking spaces to the parking space state change notification unit 4230.

The parking lot state change notification unit 4230 detects in real time the change in the information predicted and determined as the empty parking lot and the free parking space by the parking lot object according to the parking lot information received from the parking lot state determination unit 4310, In the monitor section 2200 screen or the alarm section 2300 of the main window management apparatus 2000 functioning as a GUI of the manager software, so that the manager can easily confirm the use of at least one of a message, It informs.

1, the shape learning prepara- tion section 4210 includes a parking section learning section 4230, a parking section learning section 4230, a parking section learning section 4230, A box manual setting GUI mechanism 4211 and a parking box automatic setting GUI mechanism 4212 for automatically creating input data. The parking mechanism setting manual GUI setting unit 4211 sets at least one of snapshot picture data and moving picture data, which are CCTV parking picture image data input from the external source 5000 to the input module 4100, through the graphic processing unit 4340, And the administrator views the image displayed on the GUI mechanism to create the learning shape data D100 by designating the area of the empty parking space and the free parking space as polygons. Also, the parking block automatic setting GUI mechanism 4210 displays at least one of snapshot photograph data and moving picture data, which are CCTV parking lot image data input to the input module 4100, to the GUI mechanism through the graphic processor 4340, And displays the polygonal area of the parking space, thereby generating the learning shape data D100. At this time, if the area of the polygon displayed on the GUI differs from the area of the parking space recognized by the manager, the polygon displaying the position and area of the parking space is manually corrected using the GUI of the parking space setting, .

As shown in FIG. 3A, the parking space learning unit 4400 uses one or more of the learning shape data D100 and the open data D110 (parking lot image data set) The learning of the parking lot object is performed by using the deep learning or the shallow learning technique with the environmental learning data set generated by the preparation preparation unit 4220 as input. The parking space learning unit 4230 stores the filter F100 composed of K × K pixels as the learning shape data D100 or the image data D100a to D100b included in the complex learning data set D110, D100c and D111a to D111c from the upper left corner to the lower right corner and obtains a dot product by a filter composed of pixels of the image images D100a to D100c and D111a to D111c and K × K pixels, A convolution layer D210a for generating feature maps is applied. At this time, in order to distinguish the various features of the video image included in the image of the parking space provided by the learning shape data D100, the convolution layer D210a includes a plurality of multiplex filters F100 instead of one filter F100 To generate a feature map.

The quantitative value indicated in the feature map in the ReLU (Rectified Linear Unit) layer is input into the feature map generated through the filter (F100) as a non-linear value capable of determining the characteristics of the empty parking space and the empty parking space And generates an activation map (Activation map) to which the ReLu function, which is an activating function, is applied.

픽셀로 구성된 맥스풀링 필터(F200)를 stride 2로하여 2칸씩 피쳐맵의 좌측상단부터 우측하단까지 순차적으로 스캐닝하며

픽셀에서 최대값을 뽑아내어 적용한 샘플링맵(sampling map)을 생성한다.In the Maxpool Layer,

Pixels are scanned sequentially from the upper left corner of the feature map to the lower right corner of the feature map by two spaces with a maximum pulling filter F200 as stride 2

Extracts the maximum value from the pixel and creates the applied sampling map.

A plurality of convolution layers D210a and D210b, a ReLu layer and a MaxPool layer D220a and D220b are repeatedly combined (corresponding to D200 in FIG. 2C) A final sampling map in which features are extracted can be generated.

(D400a) by applying the Softmax function (D300c), which is an activation function, to a so-called "pulley connected layer " (D300a) in which each pixel value of the finally outputted sampling map is connected. A neural network D300 that can obtain the probability for the cell D400b in the range of 0 to 1.0 is constructed and a neuron of a neural network is constructed using a so-called "dropout layer (D300b) (D300c) is applied to the learning shape data D100 so as to prevent the over-fitting phenomenon that is too biased from the learning shape data D100. When the result value is changed, the sum of all the results of the transformed result is 1. Therefore, the probabilitities can be used as the convolution layer, the ReLu layer, the MaxPool layer, and the Fully connected layer. Resource-intensive matrix operation is used. Therefore, a graphics processor 4340 (see FIG. 1) including a graphics processing unit (GPU) can be used to improve the performance of the matrix operation. The number of repetitions, execution locations, and option values of each layer can be used according to Deep Learning or Shallow Learning methods such as LeNet, AlexNet, ZFnet, GoogleNet, VGGNet, Res Net .

3, image data P100 of the CCTV 1000 of the specific parking lot in actual operation is input from the real-time data input unit 4120 of the image module 4100, The method of predicting and judging the car / cold car park will be explained in more detail. The parking lot state determination unit 4310 determines the parking lot state of the CCTV 1000 of the parking lot that is actually operating based on the neural network D300 (see FIG. 2C) learned by the parking space learning unit 4230 A convolution layer, a ReLu layer, and a MaxPool layer are repeatedly combined (corresponding to D200 ') to the image data P100, and a last sampling map And distinguishes the parking space objects contained in the input image data according to the probability values generated by applying the fully-connected layer and the Softmax function (D300c). If the parking space object is an empty parking space, The parking lot state change notification unit 4320 notifies the parking lot state change notification unit 4320 of the information on the monitor unit 2200 of the parking lot management apparatus 2000 having the GUI function of the manager software The state of the parking space (P400) Change alarms, and messages. That is, the parking lot status change notification information P400 provided in the monitor unit 2200 shows the real-time parking status of the specific parking lot, the red line portion indicates a cold parking space, the green line portion indicates an empty parking space . In the left side of the screen, it can be confirmed that whether or not the parking space is parked according to the position of the parking spaces is expressed in data form. Accordingly, it is possible to provide information on whether or not the specific parking space of the specific parking lot is parked, through the status change notification information, to the parking lot manager or the driver in an alarm, display, and sound manner. In particular, when notifying the driver or informing the parking attendant, the application may transmit a change of parking status or parking status through a specific app in the mobile phone or an app in the car.

Next, referring to the flowchart of FIG. 4, the operation of the system described above will be summarized and the overall operation flow of the real-time parking box image object recognition system using the CCTV image information according to the embodiment of the present invention and the real- A method of recognizing a parking space video object will be described.

Step S1: The external source data input unit 4110 of the input module 4100 of the parking lot video object recognition apparatus 4000 inputs the open data from the external source 5000. In this case, the control unit 4330 of the control module 4300 may instruct the input module 4100 to store the open data in the comprehensive DB 4400. Basically, the comprehensive DB 4400 can be generated in the parking space state determination unit 4310 and the parking space state change notification unit 4320 of the input module 4100, the artificial intelligence module 4200, the control module 4300 itself It is preferable to store all the data or information in the backup form and configure each component so that the backup data or information can be searched and used through the comprehensive DB 4400 as needed.

Step S2: Based on the open data inputted from the external source data input part 4110 of the input module 4100, the parking image data is displayed on the GUI image 4211 using the shape manual setting GUI mechanism 4211 of the shape learning preparation preparation part 4210 And the administrator recognizes the area to designate the area of the parking space area with a polygon (for example, rectangular, trapezoidal, rhombic, or the like) to generate learning shape data D100 (S2-1).

The position and area of the parking spaces in the parking lot image data, which is the open data inputted to the input module 4100, are stored in the shape automatic setting GUI mechanism 4212 (S2-2). In this case, the learning image data may be generated by displaying the area of the parking space with a polygon (e.g., one of square, trapezoidal, rhombus, etc.).

On the other hand, if the parking space displayed on the GUI is automatically recognized by the shape automatic setting GUI mechanism 4212 in step S2-2 and is different from the parking space recognized by the manager, the manual setting GUI tool 4211 is used to manually To generate the learning shape data D100 (S2-S3).

Step S3: The environmental pre-preparation unit 4220 collects open data (parking lot image data set) for various long periods to apply a weather, cloudy, rain or snowy weather condition to the learning shape data D100, Various complex learning data sets are generated by applying environmental changes such as a weather change, an object change, a time-based shade, and a color change around a parking space, and are arranged and accumulated so that they can be used as inputs for machine learning of the parking space learning unit 4230 . As described above, the data set thus prepared may be stored in the preparation preparation unit 4220 by itself, or may be stored in the comprehensive DB 4400 for backup in parallel or without self-storage.

 Step S4: The parking space learning section 4230 discriminates and recognizes the parking space objects using the deep learning or the shallow learning technique using at least one of the generated learning shape data D100 and the complex learning data set as input, It learns whether the car object is parked or not.

More specifically, the learning unit 4230 inputs one or more of the convolution layer (Convolution Layer), the ReLu layer, and the MaxPool layer among the techniques of deep learning or shallow learning by inputting the learning shape data D100 Repeatedly applying it generates a sampling map.

Subsequently, the parking space learning unit 4230 applies one or more of the deep connected or shallow running techniques to the sampling map as a Fully Connected Layer, a Drop Out Layer, and a Softmax function. By learning the kan-chan parking spaces, it generates and accumulates learned neural network artificial intelligence that can be applied universally to the general parking lot.

Step S5: The parking lot state determination unit 4310 of the control module 4300 determines the parking lot state of the parking lot 4100 from the CCTV 1100 of the CCTV system 1000 of the parking lot actually operating in the specific parking lot, ), The real-time parking lot image data is applied to the learned neural network artificial intelligence by using the deep learning or shallow running technique to judge the parking state of the parking space object for each individual parking space, Predicts and judges the change of the box / cold parking space, and provides the parking space status information at all times in real time.

Step S6: The parking space status change notification unit 4320 of the control module 4300 uses the at least one of alarm, message, change of cats, polygon based on the parking space status information, And provides it to the monitor unit 2200 of the parking lot management apparatus 2000 having the GUI function of the software in the form of a screen or guides the manager or the driver through sound and display through the alarm unit 2300. [ In this case, the management unit 2100 of the parking lot management apparatus 2000 detects an unknown vehicle entering the parking lot and automatically detects an empty parking space through the app installed in the vehicle or the mobile phone app of the vehicle driver through the alarm unit 2300 The location of the location. The controller 4330 of the control module 4300 in the parking lot image object recognition apparatus 4000 receives guidance on vehicle entry from the management section 2100 of the parking lot management apparatus 2000 and informs the entry vehicle of the empty parking lot information It is also possible to guide the user through the direct parking status change notification unit 4320.

Step S7: accumulation of real-time image information of the specific parking window in the comprehensive DB 4000, accumulation of additional open data from the external source 5000, or new image information of another external general parking window through the external source 5000 And accumulates them in the comprehensive DB 4000 via the input unit 4110. [

Thereafter, the artificial intelligence module 4200 is updated while proceeding from step S2 to step S4, and the real-time parking space guidance function is performed through steps S5 and S6 with the function of the newly updated artificial intelligence module 4200 . However, updating through steps S2 to S4 is preferably performed when open data is accumulated in the comprehensive DB 4340 for a certain period of time.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

1000: CCTV system device 1100: CCTV (camera)
1200: VMS 1300: Image DB
2000: parking lot management device 2100:
2200: Monitor section 2300: Alarm section
4000: parking lot image object recognition device 4100: input module
4110: External source data input unit 4120: Real time data input unit
4200 Artificial Intelligence Module 4210: Preparations for shape learning
4211: Shape manual setting GUI mechanism 4212: Shape automatic setting GUI mechanism
4220: Preparing before environment learning 4230: Parking space learning part
4300: control module 4310:
4320: parking lot state change alarm part 4330:
4340: graphic processing unit 4400: comprehensive DB

Claims (18)

A real-time parking area image object recognition apparatus using CCTV image information,
An external source data input unit for inputting the open data from an external source providing previously known open data including at least image data related to the general parking lot in advance and a real time image data input unit An input module including a real-time data input unit;
A learned neural network having a function capable of determining whether or not the vehicle is parked with respect to an arbitrary parking space by learning the shape of the expected parking space and the parking space environment using the open data input from the external source data input unit of the input module, An artificial intelligence module for generating artificial intelligence;
A parking lot image provided in real time from a CCTV system of a specific parking lot among the public parking lots is applied to the learned neural network artificial intelligence to judge in real time whether or not the parking lot is parked in the specific parking lot, And a parking space state determiner for providing parking space state information indicating whether the parking spaces in the specific parking space are parked in real time at all times in real time;
Wherein the real-time parking area image object recognition device uses the CCTV image information.
The method according to claim 1,
The artificial intelligence unit
Determining a shape of a parking space and a shape of a parking space available in a plurality of public parking lots by using a CCTV image related to the parking lot state of the public parking space as a still image or a moving image as an open data from an external source data input unit of the input module, A shape learning pre-preparation unit for generating and accumulating a plurality of learning shape data based on the shape;
And a time-based change using a plurality of images provided from at least one of the learning shape data generated from the shape learning preparation unit and the still image or moving image captured from the CCTV, An environment learning preparation unit for generating and accumulating a learning complex data set reflecting environmental changes;
Based on the learning shape data from the preparation unit for pre-shape learning and the compound learning data set from the pre-environment learning preparation unit, determines whether there is an empty parking space or a cold parking space in the parking spaces of the common parking lots A parking space learning unit for generating the learned artificial intelligence having the function;
Lt; / RTI >
Wherein the learning shape data and the learning complex data set are data processed so as to be used as an input for a learning method in a parking space learning unit, and the parking space learning unit and the parking space state determination unit Learning or determining whether a parking space is an empty parking space or a cold parking space using machine learning using one,
Wherein the machine learning includes at least one of Deep Learning and Shallow learning. ≪ RTI ID = 0.0 > 15. < / RTI >
The method according to claim 1,
The control module
Message, color change, and polygon image reflecting the change of empty parking spaces and cold parking spaces in the specific parking lot, based on the real-time parking space state information provided from the parking space state determination unit, A parking space status change notification unit for providing parking space status change alarm information so that the parking space status change alarm can be used;
Further comprising:
The parking lot state change notifying information provided by the parking lot state change notifying unit may be used for at least one of a speaker, a notification light emitter, an LED electric signboard, and a GUI monitor installed in the parking lot management apparatus or a combination thereof, And the parking lot information is used as empty parking space guide information through the app of the driver's mobile terminal.
The method according to claim 1,
The pre-
The CCTV parking lot image data input to the input module is displayed on the GUI using at least one of snapshot photograph data and moving picture data to designate a parking lot area recognized by the manager as one of the polygons, Manually set parking spaces created by data GUI mechanism; And
A parking space automatic setting GUI mechanism for automatically recognizing the location and area of parking spaces in the CCTV parking lot image data input to the input module and generating learning image data;
Lt; / RTI >
Wherein the parking space manually setting GUI mechanism includes an environment function enabling an administrator to manually correct the learning shape data generated by the parking space automatic setting GUI mechanism,
Wherein the control module further comprises a graphic processing unit for supporting the manual parking setting GUI mechanism and the parking space automatic setting GUI mechanism.
5. The method according to any one of claims 1 to 4,
The learning complex data set of the pre-environment learning preparation unit includes an open CCTV image as various known open data related to the parking condition of the plurality of parking lots already photographed, the open CCTV image including at least the environment change, Wherein the weather change includes at least a change of clear, cloudy, rain, snow, wind, and the ambient object change includes at least a change of a moving or fixed object in a parking lot or around a parking space, Color and seasonal changes of the surrounding environment according to progress,
The parking space learning unit receives positive learning data for learning the actual shape of a specific parking space by inputting a learning complex data set including the open CCTV image and learning the shape of the parking space deformed according to the environment change And to perform machine learning which improves the perfection of car park object recognition through negative learning,
The parking space video object recognition device
And a comprehensive database for storing data, data sets, and information generated from the input module, the AI module, and the control module, and providing the data, the specific parking lot management device, or an external source to each component of the module, Wherein the real-time parking area image object recognition device uses CCTV image information.
6. The method of claim 5,
The artificial intelligence unit updates the learned neural network artificial intelligence based on the open data from the external source and the real-time image information provided from the CCTV system of the specific parking lot, thereby increasing the learning ability of determining whether the parking space is parked in a specific parking lot And provides the increased learning ability as an external source for upgrading the learned neural network artificial intelligence of the other parking box video object recognition device.
A real-time parking space image object recognition system using CCTV image information, the system comprising: a CCTV system device for photographing real-time parking status of a parking lot and accumulating image information thereof;
And an open data source capable of providing an open data including at least image information related to the general parking lot in advance in a preliminarily provided form, as well as at least connecting to at least a real parking area video object recognition device installed in a plurality of general parking windows With an external source;
A parking lot management device installed in a specific parking window;
A parking space image object recognition device for recognizing whether a parking space of a specific parking space is parked based on CCTV image information displayed from the CCTV system;
Wherein the parking space video object recognizing device comprises:
An external source data input unit for inputting the open data from an external source provided in advance with an open data including at least video data related to the general parking lot; An input module including a real-time data input unit;
The learned neural network artificial intelligence which can judge whether or not the vehicle is parked with respect to a random parking space by learning the expected shape of the parking space and the parking space environment using the open data inputted from the external source data input part of the input module An artificial intelligence module to generate;
A parking lot image provided in real time from a CCTV system of a specific parking lot among the public parking lots is applied to the learned neural network artificial intelligence to judge in real time whether or not the parking lot is parked in the specific parking lot, And a parking space state determiner for providing parking space state information indicating whether the parking spaces in the specific parking space are parked in real time at all times in real time;
Wherein the real-time parking area image object recognition system uses CCTV image information.
8. The method of claim 7,
The artificial intelligence unit
Determining a shape of a parking space and a shape of a parking space available in a plurality of public parking lots by using a CCTV image related to the parking lot state of the public parking space as a still image or a moving image as an open data from an external source data input unit of the input module, A shape learning pre-preparation unit for generating and accumulating a plurality of learning shape data based on the shape;
And a time-based change using a plurality of images provided from at least one of the learning shape data generated from the shape learning preparation unit and the still image or moving image captured from the CCTV, An environment learning preparation unit for generating and accumulating a learning complex data set reflecting environmental changes;
Based on the learning shape data from the preparation unit for pre-shape learning and the compound learning data set from the pre-environment learning preparation unit, determines whether there is an empty parking space or a cold parking space in the parking spaces of the common parking lots A parking space learning section that provides the learned neural network artificial intelligence;
Lt; / RTI >
Wherein the learning shape data and the learning complex data set are data processed so as to be used as an input for a learning method in a parking space learning unit, and the parking space learning unit and the parking space state determination unit Learning or determining whether a parking space is an empty parking space or a cold parking space using machine learning using one,
Wherein the machine learning includes at least one of Deep Learning and Shallow learning. ≪ RTI ID = 0.0 > [10] < / RTI >
8. The method of claim 7,
The control module
Message, color change, and polygon image reflecting the change of empty parking spaces and cold parking spaces in the specific parking lot, based on the real-time parking space state information provided from the parking space state determination unit, A parking space status change notification unit for providing parking space status change alarm information so that the parking space status change alarm can be used;
Further comprising:
The parking lot state change notifying information provided by the parking lot state change notifying unit may be used for at least one of a speaker, a notification light emitter, an LED electric signboard, and a GUI monitor installed in the parking lot management apparatus or a combination thereof, The parking lot information is used as empty parking space guide information through the application of the driver's mobile terminal.
8. The method of claim 7,
The pre-
The CCTV parking lot image data input to the input module is displayed on the GUI using at least one of snapshot photograph data and moving picture data to designate a parking lot area recognized by the manager as one of the polygons, Manually set parking spaces created by data GUI mechanism; And
A parking space automatic setting GUI mechanism for automatically recognizing the location and area of parking spaces in the CCTV parking lot image data input to the input module and generating learning image data;
Lt; / RTI >
Wherein the parking space manually setting GUI mechanism includes an environment function enabling an administrator to manually correct the learning shape data generated by the parking space automatic setting GUI mechanism,
Wherein the control module further comprises a graphic processing unit for supporting the manual parking setting GUI mechanism and the parking space automatic setting GUI mechanism.
11. The method according to any one of claims 7 to 10,
The learning complex data set of the pre-environment learning preparation unit includes an open CCTV image as various known open data related to the parking condition of the plurality of parking lots already photographed, the open CCTV image including at least the environment change, Wherein the weather change includes at least a change of clear, cloudy, rain, snow, wind, and the ambient object change includes at least a change of a moving or fixed object in a parking lot or around a parking space, Color and seasonal changes of the surrounding environment according to progress,
The parking space learning unit receives positive learning data for learning the actual shape of a specific parking space by inputting a learning complex data set including the open CCTV image and learning the shape of the parking space deformed according to the environment change And to perform machine learning which improves the perfection of car park object recognition through negative learning,
The parking space video object recognition device
And a comprehensive database for storing data, data sets, and information generated from the input module, the AI module, and the control module, and providing the data, the specific parking lot management device, or an external source to each component of the module, A real time parking box image object recognition system using CCTV image information.
12. The method of claim 11,
The artificial intelligence unit updates the learned neural network artificial intelligence based on the open data from the external source and the real-time image information provided from the CCTV system of the specific parking lot to determine whether to park the parking space in the specific parking lot And provides the increased learning ability as an external source for upgrading the learned neural network artificial intelligence of the other parking box image object recognition device.
A real-time parking spot image object recognition system using a real-time parking spot image object recognition system using CCTV image information used in parking lot parking information. The system comprises a CCTV system device for capturing real- ; And an open data source capable of providing an open data including at least image information related to the general parking lot in advance in a preliminarily provided form, as well as at least connecting to at least a real parking area video object recognition device installed in a plurality of general parking windows With an external source; A parking lot management device installed in a specific parking window; And a parking space image object recognition device for recognizing whether or not a parking space of a specific parking space is parked based on the CCTV image information displayed from the CCTV system,
An open data input step of inputting the open data from an external source which provides an open data including at least image information related to the parking lot in advance,
The artificial intelligence module of the video object recognizing device learns the shape of the expected parking space and the parking space environment by using the open data to determine whether or not the vehicle is parked in the random parking space, An artificial intelligence learning step of generating artificial intelligence;
A real time data input step in which the input module inputs real time image data from a CCTV system of a specific parking lot;
The parking space state determination unit of the video object recognition apparatus applies a parking space image provided in real time from the CCTV system of the specific parking lot to the learned neural network artificial intelligence to determine whether or not the parking space for the parking spaces in the specific parking space is parked A parking lot state determination step of providing parking lot state information indicating whether or not parking spaces in the specific parking lot are parked at all times in real time based on the determination result;
The method comprising the steps of: (a) inputting the CCTV image information;
14. The method of claim 13,
The artificial intelligence learning step
The artificial intelligence module uses the CCTV image related to the parking lot status of the public parking lot as a still image or a moving image as an open data from the open data input unit of the input module so that the shape of a parking space existing in a plurality of public parking lots, A shape learning preliminary step of generating and accumulating a plurality of learning shape data based on the determined shape;
Wherein the artificial intelligence module uses a plurality of images provided from at least one of the learning shape data generated from the shape learning preparation unit and the still image or moving image captured from the CCTV, An environmental learning preparation step of generating and accumulating a learning complex data set reflecting an environmental change including at least one of the learning complex data sets;
Wherein the artificial intelligence module learns through the learning based on the learning shape data in the pre-shape learning step and the complex learning data set from the pre-environment learning preparation step, A learning step of learning a neural network artificial intelligence function for determining whether or not a field is a field;
Lt; / RTI >
Wherein the learning shape data and the learning complex data set are data processed so as to be used as input for a learning method in a parking space learning unit,
Learning or determining whether the parking space is an empty parking space or a cold parking space by using machine learning using at least one of autonomous learning and map learning in the parking block learning step and the parking block state determining step,
Wherein the machine learning includes at least one of Deep Learning and Shallow learning. ≪ RTI ID = 0.0 > 18. < / RTI >
14. The method of claim 13,
After the parking lot state determination step,
A parking space state change notification unit of the parking space image object recognition apparatus generates an alarm message that reflects a change of an empty parking space and a cold parking space in the specific parking space based on the real time parking space state information provided from the parking space state determination unit A color change state, a polygon image directly, or a parking lot state change notification step for providing parking lot state change alarm information for use in the parking lot management apparatus;
Further comprising:
The parking lot state change notifying information provided by the parking lot state change notifying unit may be used for at least one of a speaker, a notification light emitter, an LED electric signboard, and a GUI monitor installed in the parking lot management apparatus or a combination thereof, Wherein the CCTV image information is used as empty parking space guide information through an application of the driver's mobile terminal.
14. The method of claim 13,
In the preparation step before the shape learning,
The parking space manual setting GUI of the artificial intelligence module is displayed on the GUI using at least one of the snapshot picture data and the moving picture data which are the CCTV parking lot image data input to the input module, A parking space manually setting GUI for designating one area and generating the designated area as learning shape data; or
A parking space automatic setting GUI of the parking space automatic setting GUI of the artificial intelligence module automatically recognizes the location and area of parking spaces in the CCTV parking space image data input to the input module and generates learning image data;
Further comprising:
The parking section manual setting GUI step includes a step of providing a correction function by which an administrator can manually correct the learning shape data created by the parking area automatic setting GUI mechanism,
Wherein the video object recognition apparatus further comprises a graphics processing unit for supporting the GUI setting mechanism for the parking space manually and the GUI setting unit for automatically setting the parking space.
17. The method according to any one of claims 13 to 16,
The learning complex data set in the pre-environment learning preparation step includes an open CCTV image as various known open data related to the parking status of the plurality of parking lots already photographed, and the open CCTV image includes at least the environment change, Wherein the weather change includes at least a change of sunny, cloudy, rain, snow, wind, and the ambient object change includes at least a change of a moving or fixed object in a parking lot or around a parking space, Color and seasonal changes of the surrounding environment over time,
The parking space learning unit receives positive learning data for learning the actual shape of a specific parking space by inputting a learning complex data set including the open CCTV image and learning the shape of the parking space deformed according to the environment change And to perform machine learning which improves the perfection of car park object recognition through negative learning,
Wherein the integrated database of the parking space image object recognition device stores data, a data set, and information generated from the input module, the AI module, and the control module, Wherein the step of providing the CCTV image information comprises the steps of:
18. The method of claim 17,
The artificial intelligence unit updates the learned neural network artificial intelligence based on the open data from the external source and the real-time image information provided from the CCTV system of the specific parking lot to determine whether to park the parking space in the specific parking lot. And providing the increased learning capability to an external source for upgrading the learned neural network artificial intelligence of another parking box image object recognition device.

Images