WO2022145594A1

WO2022145594A1 - Unmanned parking control system and method based on deep learning object tracking

Info

Publication number: WO2022145594A1
Application number: PCT/KR2021/006827
Authority: WO
Inventors: 최성춘
Original assignee: 주식회사 오픈시스넷
Priority date: 2020-12-28
Filing date: 2021-06-01
Publication date: 2022-07-07
Also published as: KR102493273B1; KR20220094004A

Abstract

Provided is an intelligent unmanned parking control system comprising: an image-capturing device for generating parking-zone-specific video information; and a server, which detects an entering vehicle from the parking-zone-specific video information by using a pre-stored machine learning model, extracts a plurality of pieces of still image information including the entering vehicle from the parking-zone-specific video information by using the pre-stored machine learning model, extracts number plate information about the entering vehicle from each of the plurality of pieces of still image information including the entering vehicle by using the pre-stored machine learning model, and compares the number plate information about the entering vehicle, having been extracted from each of the plurality of pieces of still image information including the entering vehicle, so as to generate number information about the entering vehicle.

Description

Unmanned parking control system and method through deep learning object tracking

The present invention relates to an intelligent unmanned parking control system and method using 360-degree omnidirectional CCTV, and in particular, from the moment the vehicle is detected through deep learning object tracking, during parking, multiple results through recognition of the same license plate at various angles and locations It relates to a deep learning-based intelligent unmanned parking control system and method that can increase license plate recognition accuracy with

Currently, the parking control system is being replaced by an intelligent parking control system from the unmanned entrance and exit and billing system.

In the case of the existing intelligent parking control, since the service is provided through sensor and coil-type hardware, there is a problem of increased facility construction and maintenance costs.

In addition, since real-time monitoring is performed through a one-way CCTV in the prior art, a large number of CCTVs are required, and there is a problem in that real-time monitoring is impossible due to the occurrence of a shaded area.

Accordingly, technology that can eliminate shadow areas and provide intelligent parking control through 360-degree omnidirectional CCTV is required. is required

The technical problem to be achieved by the present invention is to solve the license plate misrecognition that occurs in 360-degree omnidirectional CCTV image recognition, from the moment the vehicle is detected to the number of results through the recognition of the same license plate at various angles and locations while parking is performed. Accuracy of license plate recognition It is to provide a deep learning-based intelligent unmanned parking control system and method that can increase

According to one embodiment, an intelligent unmanned parking control system is provided. The intelligent unmanned parking control system detects a vehicle entering from the video information for each parking area using an image photographing device that generates video information for each parking area, and a pre-stored machine learning model, and uses the pre-stored machine learning model to Extracting a plurality of still image information including the entering vehicle from the video information for each parking area, and using a pre-stored machine learning model, license plate information of the entering vehicle from each of the plurality of still image information including the entering vehicle and a server for generating number information of the entering vehicle by comparing the number plate information of the entering vehicle extracted from each of a plurality of still image information including the entering vehicle.

The server includes a communication unit for receiving the video information for each parking area from the image capturing device, an artificial intelligence analysis unit for generating number information of the vehicle entering from the video information for each parking area using a pre-stored machine learning model, and It may include a payment management unit that generates information on the vehicle to be charged a parking fee based on the number information of the entering vehicle.

The artificial intelligence analysis unit, among the number plate information of the entering vehicle extracted from each of a plurality of still screen image information including the entering vehicle, converts a letter or number having a number of overlaps greater than the reference value for each digit as the number information of the entering vehicle You can choose.

According to one embodiment, an unmanned parking control method of an intelligent unmanned parking control system is provided. The intelligent unmanned parking control method includes the steps of: an image capturing device generating video information for each parking area; detecting a vehicle entering from the video information for each parking area using a machine learning model stored in advance by an artificial intelligence analysis unit; the artificial intelligence extracting a plurality of still screen image information including the vehicle entering from the video information for each parking area by using a machine learning model stored in advance by the analysis unit, and the vehicle entering the vehicle using the machine learning model stored in advance by the artificial intelligence analysis unit extracting license plate information of the entering vehicle from each of a plurality of still screen image information including, and generating number information of the entering vehicle by comparing them.

In the step of generating the number information of the entering vehicle, the AI analysis unit has a number of duplicates greater than the reference value for each digit among the number plate information of the entering vehicle extracted from each of a plurality of still image information including the entering vehicle. A letter or number may be selected as the number information of the entering vehicle.

From the moment the vehicle is detected to during parking, it is possible to increase the license plate recognition accuracy as a result of multiple results through the same license plate recognition at various angles and locations.

Costs can be reduced by minimizing the number of cameras.

1 is a block diagram of an intelligent unmanned parking control system according to an embodiment.

2 is a view for explaining video information for each parking area according to an embodiment.

3 is a block diagram of a server of an intelligent unmanned parking control system according to an embodiment.

4 and 5 are diagrams for explaining the operation of the artificial intelligence analysis unit according to an embodiment.

6 is a flowchart of an intelligent unmanned parking control method according to an embodiment.

Hereinafter, with reference to the accompanying drawings, the embodiments of the present invention will be described in detail so that those of ordinary skill in the art to which the present invention pertains can easily implement them. However, the present invention may be embodied in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part "includes" a certain element, it means that other elements may be further included, rather than excluding other elements, unless otherwise stated.

Terms used in the embodiments of the present invention are selected as currently widely used general terms as possible while considering the functions in the present invention, which may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, etc. . In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding embodiments. Therefore, the terms used in the present embodiments should be defined based on the meaning of the term and the overall contents of the present embodiments, rather than the simple name of the term.

In an embodiment of the present invention, terms including an ordinal number such as first, second, etc. may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

Also, in an embodiment of the present invention, the singular expression includes the plural expression unless the context clearly dictates otherwise.

In addition, in the embodiments of the present invention, terms such as "comprises" or "have" are intended to designate that the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification exist, but one It should be understood that it does not preclude the possibility of the presence or addition of or more other features or numbers, steps, operations, components, parts, or combinations thereof.

In addition, in an embodiment of the present invention, a 'module' or 'unit' performs at least one function or operation, and may be implemented as hardware or software, or a combination of hardware and software. In addition, a plurality of 'modules' or a plurality of 'units' may be integrated into at least one module and implemented by at least one processor, except for 'modules' or 'units' that need to be implemented with specific hardware.

1 is a block diagram of an intelligent unmanned parking control system according to an embodiment. 2 is a view for explaining video information for each parking area according to an embodiment. 3 is a block diagram of a server of an intelligent unmanned parking control system according to an embodiment. 4 and 5 are diagrams for explaining the operation of the artificial intelligence analysis unit according to an embodiment.

Referring to FIG. 1 , the intelligent unmanned parking control system according to an embodiment includes an entrance detection device 100 , an image photographing device 200 , a server 300 , a vehicle exit detection device 400 , and a user terminal 500 . do.

The entrance detection device 100 detects the entrance and exit of a vehicle entering the parking lot. As an embodiment, the vehicle entrance detection device 100 may detect a beacon signal of a vehicle or user terminal 500 , and when the corresponding beacon signal is detected, it is determined that the vehicle has entered the vehicle, and the breaker is opened and the vehicle is provided to the server 300 . You can send a message announcing the entry of

As an embodiment, the image capturing apparatus 200 may generate video information for each parking area when an entrance of a vehicle is detected.

The image photographing apparatus 200 may be a fixed 360-degree CCTV as an embodiment, may be disposed in the center of the ceiling of the parking lot, and may include a plurality of lenses.

As an embodiment, the image capturing apparatus 200 may determine whether a vehicle parked in a pre-divided parking line is parked from video information for each parking area using a pre-stored machine learning model.

Referring to FIG. 2 , as an embodiment, the image photographing apparatus 200 may photograph 12 to 16 surfaces (a plurality of preset parking areas) using a plurality of lenses, and may generate video information for each parking area. have. Through this, the number of CCTVs can be reduced and costs can be reduced. In addition, it is possible to improve the accuracy by analyzing the video information for each parking area and recognizing the license plate.

The image capturing apparatus 200 may include a communication module that transmits and receives data to and from the server 300 using wired/wireless communication as an embodiment.

The server 300 extracts the number information of the vehicle parked in the parking line partitioned in advance from the video information for each parking area by using the machine learning model stored in advance, and information on the vehicle to be charged for parking based on the number information of the entering vehicle can create

Referring to FIG. 3 , the server 300 according to an embodiment may include a communication unit 310 , an artificial intelligence analysis unit 320 , and a payment management unit 33 .

As an embodiment, the communication unit 310 may receive video information for each parking area from the image capturing apparatus 200 .

The artificial intelligence analysis unit 320 may generate a machine learning model using a pre-stored algorithm based on vehicle shape data, vehicle license plate data, and parking vehicle data parked in a parking line as an embodiment. The pre-stored algorithm may be a machine learning (Machine Learning) learning algorithm or a deep learning (Deep Learning) learning algorithm as an embodiment.

As an embodiment, the artificial intelligence analysis unit 320 may detect a vehicle entering from video information for each parking area using a pre-stored machine learning model. As an embodiment, the artificial intelligence analysis unit 320 may detect a vehicle entering from video information of a parking lot entrance area among video information for each parking area by using a pre-stored machine learning model.

Referring to FIG. 4 , the artificial intelligence analysis unit 320 extracts a plurality of still image information (captured images) including a vehicle entering from the video information for each parking area by using a pre-stored machine learning model as an embodiment. can do. The still screen image information may be information including a front or rear shape of the vehicle, as an embodiment.

Referring to FIG. 5 , the artificial intelligence analysis unit 320 may extract license plate information of the entering vehicle from each of a plurality of still screen image information including the entering vehicle by using a pre-stored machine learning model as an embodiment. , it is possible to generate the number information of the entering vehicle by comparing the number plate information of the entering vehicle extracted from each of a plurality of still screen image information including the entering vehicle.

Specifically, the artificial intelligence analysis unit 320 converts a letter or number having a number of overlaps greater than the reference value for each digit among the license plate information of the entering vehicle extracted from each of a plurality of still image information including the vehicle entering the number of the vehicle entering the vehicle. information can be selected. The reference value is an example, and may be two or more values, and may vary according to a user's setting.

For example, when the license plate information extracted through the four still image information is 'ABC-1234', 'ABG-1284', 'NBC-1734', and 'APC-7231', artificial intelligence analysis unit 320 can select the letter or number ('ABC-1234') with the highest number of duplicates in each digit as the number information of the vehicle entering the vehicle. Through this, the accuracy of number recognition can be improved, and the accuracy of determining whether parking is complete can be improved.

The artificial intelligence analysis unit 320, as an embodiment, analyzes video information for each parking area generated through the image capturing device 200 to extract the number information of the vehicle entering, and when the number information of the vehicle entering the vehicle is extracted, parking is completed. can be judged as

As an embodiment, the server 300 may store video information from a point in time when a vehicle is detected to a point in time when it is determined that the parking of the vehicle is complete among video information for each parking area received from the video photographing device 200 .

As an embodiment, the payment management unit 33 may generate information on the vehicle to be charged for parking based on the number information of the vehicle entering the vehicle. As an embodiment, the payment management unit 33 may generate parking fee charge target vehicle information based on the number information of the corresponding entering vehicle when the artificial intelligence analysis unit 320 determines that the parking is complete. In this case, the communication unit 310 may transmit, as an embodiment, a message informing the user terminal 500 of parking location information and parking fee charging through an app or a web.

As an embodiment, the server 300 may detect the beacon signal of the vehicle or the user terminal 500 to determine the location of the vehicle in real time, and when the vehicle enters the parking lot, the user terminal 500 through the app or the web ) can be provided with a map guiding them to a parking area.

As an embodiment, the artificial intelligence analysis unit 320 may determine whether an abnormal situation occurs by analyzing the image information including the vehicle entering the vehicle generated after the payment information is generated, and the communication unit 310 when the abnormal situation occurs A notification message may be transmitted to the user terminal 500 . The abnormal situation is an example, and may be a fire occurrence situation or a situation in which a person is located near a vehicle entering the vehicle for a predetermined time or longer.

The artificial intelligence analysis unit 320, as an embodiment, may generate a machine learning model based on learning data about an abnormal situation, and determine whether an abnormal situation occurs from image information using a pre-stored machine learning model. can

The un-parking detecting apparatus 400 may detect the un-parking of the entering vehicle. As an embodiment, the vehicle taking-out detection apparatus 400 may detect a beacon signal of the vehicle or the user terminal 500 . When the vehicle or user terminal 500 detects a beacon signal after detecting the entrance of the vehicle or the user terminal 500 , the vehicle exit detection device 400 determines that the vehicle is exiting the vehicle, opens the breaker, and sends the vehicle to the server 300 . You can send a message announcing the departure of

When the exit of the vehicle is detected, the payment management unit 330 may generate payment information based on the number information of the vehicle, the entry and exit time of the vehicle, and the exit time of the vehicle. In this case, the communication unit 310 may transmit payment information to the user terminal 500 .

The user terminal 500, as an embodiment, may be connected to the server 300 through an app or web, and information on the number of the vehicle entering, the entry and exit time of the vehicle entering, a map guiding the parking area, and parking location information And a message informing of charging a parking fee, abnormal situation occurrence information, and payment information may be output on the screen.

The user terminal 500 is an embodiment, and includes a memory means such as a mobile communication terminal, a desktop computer, a notebook computer, a workstation, a palmtop computer, a personal digital assistant (PDA), a web pad, etc. And it may be a digital device equipped with a microprocessor and equipped with computing power.

Referring to FIG. 6 , in the intelligent unmanned parking control method according to an embodiment, the image capturing device 200 generates video information for each parking area (S100), and the artificial intelligence analysis unit 320 is a pre-stored machine learning model. Detecting the vehicle entering from the video information for each parking area using (S200), the artificial intelligence analysis unit 320 using a pre-stored machine learning model, a plurality of still screens including the vehicle entering from the video information for each parking area Step of extracting image information (S300), the artificial intelligence analysis unit 320 extracting license plate information of the entering vehicle from each of a plurality of still screen image information including the entering vehicle using a machine learning model stored in advance (S400) ), and the artificial intelligence analysis unit 320 compares the number plate information of the entering vehicle extracted from each of a plurality of still image information including the entering vehicle to generate number information of the entering vehicle (S500).

In the step (S500) of generating the number information of the entering vehicle, the artificial intelligence analysis unit 320 duplicates the number plate information of the entering vehicle extracted from each of a plurality of still image information including the entering vehicle for each digit greater than the reference value A letter or number having the number of times can be selected as the number information of the entering vehicle.

Since steps S100 to S500 are the same as the operation contents of the intelligent unmanned parking control system described above, a detailed description thereof will be omitted.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto. is within the scope of the right.

A person of ordinary skill in the art related to this embodiment will understand that it may be implemented in a modified form within a range that does not deviate from the essential characteristics of the above description. Therefore, the disclosed methods should be considered in an illustrative and not a restrictive sense. The scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

Claims

A video recording device for generating video information for each parking area, and

Detecting a vehicle entering from the video information for each parking area using a pre-stored machine learning model, and extracting a plurality of still image information including the vehicle entering from the video information for each parking area using a pre-stored machine learning model and extracting license plate information of the entering vehicle from each of a plurality of still image information including the entering vehicle by using a pre-stored machine learning model, and extracting from each of the plurality of still image information including the entering vehicle A server for generating number information of the entering vehicle by comparing the number plate information of the entering vehicle

An intelligent unmanned parking control system that includes
In claim 1,

The server is

A communication unit for receiving the video information for each parking area from the image photographing device,

An artificial intelligence analysis unit for generating number information of the entering vehicle from the video information for each parking area using a pre-stored machine learning model, and

An intelligent unmanned parking control system comprising a payment management unit generating information on a vehicle to be charged a parking fee based on the number information of the vehicle entering the vehicle.
In claim 2,

The artificial intelligence analysis unit,

Intelligent unmanned parking that selects, as the number information of the entering vehicle, a letter or number having a duplicate number greater than a reference value for each digit among the license plate information of the entering vehicle extracted from each of a plurality of still screen image information including the entering vehicle control system.
As an unmanned parking control method of an intelligent unmanned parking control system,

The video recording device generating video information for each parking area,

Detecting the vehicle entering from the video information for each parking area by using the machine learning model stored in advance by the artificial intelligence analysis unit;

extracting a plurality of still image information including the vehicle entering from the video information for each parking area by using the machine learning model stored in advance by the artificial intelligence analysis unit;

extracting the license plate information of the entering vehicle from each of a plurality of still screen image information including the entering vehicle by using the machine learning model stored in advance by the artificial intelligence analysis unit, and

generating number information of the entering vehicle by comparing the number plate information of the entering vehicle extracted from each of a plurality of still screen image information including the entering vehicle by the artificial intelligence analysis unit

An intelligent unmanned parking control method comprising a.
In claim 4,

The step of generating the number information of the entering vehicle,

The artificial intelligence analysis unit selects a letter or number having a number of overlaps greater than the reference value for each digit among the license plate information of the entering vehicle extracted from each of a plurality of still screen image information including the entering vehicle as the number information of the entering vehicle An intelligent unmanned parking control method.