KR102418476B1 - Method of operating parking control system that precisely performs reading of license plates and readings of counterfeiting and contamination by using two-dimensional image data and three-dimensional depth data - Google Patents

Abstract

Provided is an operating method of a parking control system capable of precisely performing both of reading of license plates and determination of forgery and contamination by using both of two-dimensional image data and three-dimensional depth data based on Internet-of-things (IoT) and artificial intelligence (AI). The operating method of a parking control system for precisely reading license plates and determining forgery and contamination comprises the steps of: sensing, by an image sensor, a license plate to generate image data and transmitting the image data to a main server; sensing, by a depth sensor, the license plate to generate depth data and transmitting the depth data to the main server; and reading, by the main server, characters on the license plate by using the image data and the depth data, and determining whether the license plate is forged or contaminated.

Description

The operation method of a parking control system that precisely performs the reading of license plates and the determination of forgery and contamination using two-dimensional image data and three-dimensional depth data OF COUNTERFEITING AND CONTAMINATION BY USING TWO-DIMENSIONAL IMAGE DATA AND THREE-DIMENSIONAL DEPTH DATA}

The present invention relates to an operating method of a parking control system that precisely performs the reading of a license plate and the determination of forgery and contamination by using two-dimensional image data and three-dimensional depth data.

The parking control system refers to a system that automatically (unmanned) or semi-automatically (manned) vehicles entering and exiting a parking lot. In detail, the parking control system identifies and stores the vehicle by recognizing letters and numbers by tracking the license plate of the vehicle entering the parking lot, and when the vehicle leaves, calculates the parking time, etc. and issues an invoice. Furthermore, the parking control system may perform a function of providing such information to a manager or residents by tracking the location where a specific vehicle is parked.

That is, the parking control system must accurately recognize the license plate of the vehicle, and the conventional parking control system uses the Internet of Things (IoT), etc. to capture the license plate of an entering vehicle with an image sensor (camera module). The license plate was read by analyzing one image data with a character recognition program.

However, since the conventional parking control system reads the license plate using only two-dimensional image data, 1) there was a problem in that the license plate could not be accurately read, and 2) it was not possible to read the forgery and contamination of the license plate at all. There was a problem.

On the other hand, artificial intelligence (AI) is a field of computer engineering and information technology that realizes human learning ability, reasoning ability, perceptual ability, and natural language understanding ability through computer programs. In particular, supervised learning, which is used in the most fields of artificial intelligence, is a method of accurately predicting an actual value using training data including a correct answer (Label).

If a depth sensor (for example, a non-contact three-dimensional sensor such as lidar) is additionally used in the conventional parking control system, three-dimensional depth data (contour and bending angle) can be acquired together, and letters and numbers are recognized If a program for recognizing contours and bending angles is created by artificial intelligence learning, it will be possible to accurately read license plates and determine whether counterfeiting or contamination is present.

The problem to be solved by the present invention is to use both two-dimensional image data and three-dimensional depth data based on Internet of Things (IoT) and artificial intelligence (AI) to detect and counterfeit vehicle license plates. It is to provide a method of operating a parking control system that can precisely perform all of the determination of whether or not pollution is present.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

The operating method of the parking control system that precisely performs the reading of the license plate of the present invention for solving the above problems and the determination of whether forgery and contamination is an image sensor generates image data by sensing the license plate and transmits it to the main server to do; The depth sensor senses the license plate, generates depth data, and transmits it to the main server; The main server using the image data and the depth data, it may be characterized in that it comprises the step of reading the characters of the license plate and determining whether counterfeiting or contamination.

The main server matches each of one or more characters on the license plate with random characters using a program for recognizing characters in image data to match the similarity rate value for the character having the highest similarity value among the random characters to the first similarity Set as a rate value, match each of one or more characters on the license plate with arbitrary characters using a program that recognizes characters from depth data, and set the similarity value for the character having the highest similarity value as the second similarity value, , each of the one or more characters on the license plate may be read as a character set with a higher similarity value among a character set with a first similarity value and a character set with a second similarity value.

The main server sets an error rate value for each of one or more characters of the license plate by using a program for recognizing contours in the depth data, and when the average value of the error rate values set for each of the one or more characters is greater than or equal to the error rate reference value, a forged license plate It can be characterized by judging by .

The main server determines that the high similarity value among the first similarity value and the second similarity value set for a specific character on the license plate is less than the similarity rate reference value, and when the error rate value set for a specific character on the license plate is less than the error rate reference value, the corresponding character is It can be characterized as judged to be contaminated.

The main server may be characterized in that it generates a program for recognizing a character and a program for recognizing an outline by artificial intelligence learning.

In the operating method of the parking control system that precisely performs the reading of the license plate and the determination of forgery and contamination of the license plate of the present invention, the reading and counterfeiting of the license plate is a program based on IoT technology and artificial intelligence using both an image sensor and a depth sensor. And it has the advantage of being able to accurately perform the determination of contamination without errors.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a perspective view showing a place where the operating method of the parking control system that precisely performs the reading of the license plate of the present invention and the determination of forgery and contamination is performed.
2 is a flow chart showing the sequence in which the operating method of the parking control system that precisely performs the reading of the license plate of the present invention and the determination of whether forgery or contamination is performed.
3 is a schematic diagram showing a system in which the operating method of the parking control system for precisely performing the reading of the license plate of the present invention and the determination of whether forgery or contamination is performed.
4 is a conceptual diagram illustrating the reading of the license plate in the operating method of the parking control system that precisely performs the reading of the license plate of the present invention and the determination of whether counterfeiting and contamination are present.
5 is a conceptual diagram illustrating the determination of whether or not the license plate is forged in the method of operating a parking control system that precisely performs the reading of the license plate of the present invention and the determination of forgery and contamination.
6 is a conceptual diagram illustrating the determination of whether or not the license plate is contaminated in the operating method of the parking control system that precisely performs the reading of the license plate of the present invention, and the determination of whether or not forgery and contamination are present.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the present embodiments allow the disclosure of the present invention to be complete, and those of ordinary skill in the art to which the present invention pertains. It is provided to fully understand the scope of the present invention to those skilled in the art, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

Hereinafter, "character" as used herein is a visual sign system used to write human language, and may be defined as letters including numbers, unlike dictionary meanings. That is, as used herein, the term “letter” is interpreted to include not only letters but also numbers.

Hereinafter, with reference to the drawings, a method 1000 of operating a parking control system for precisely performing the reading of the license plate of the present invention and the determination of whether forgery or contamination is described will be described. 1 is a perspective view showing a place where the operating method of a parking control system that precisely performs the reading of the license plate of the present invention and the determination of whether counterfeiting and contamination is performed, and FIG. 2 is the reading and counterfeiting of the license plate of the present invention. It is a flowchart showing the order in which the operating method of the parking control system that precisely performs the determination of whether or not is contaminated is performed. It is a schematic diagram showing a system in which the operating method is performed, and FIG. 4 is a conceptual diagram showing the reading of the license plate in the operating method of the parking control system that precisely performs the reading of the license plate of the present invention and determination of forgery and contamination, 5 is a conceptual diagram illustrating determining whether a vehicle license plate is forged or not in the operating method of a parking control system that precisely performs the reading of the license plate of the present invention and determination of forgery and contamination, and FIG. 6 is the vehicle license plate of the present invention It is a conceptual diagram showing the determination of whether or not the license plate is contaminated in the operation method of the parking control system that precisely performs the reading of and counterfeiting and the determination of whether the vehicle is contaminated.

The operating method 1000 of the parking control system for precisely performing the reading of the license plate of the present invention and the determination of forgery and contamination is an image sensor 10 , a depth sensor 20 , a network 30 , and a main server 40 . ) can be performed in a system containing

The image sensor 10 is a sensor capable of generating two-dimensional image data by capturing a license plate, for example, a camera module, etc. may be used, but is not limited thereto.

The depth sensor 20 is a sensor capable of generating three-dimensional depth data by scanning a license plate. For example, a non-contact three-dimensional sensor such as a lidar may be used, but is not limited thereto.

The network 30 means a connection structure capable of exchanging information between each node, such as the image sensor 10, the depth sensor 20, and the main server 40, and an example of such a network includes RF, 3GPP ( 3rd Generation Partnership Project network, LTE (Long Term Evolution) network, 5GPP (5th Generation Partnership Project) network, WIMAX (World Interoperability for Microwave Access) network, Internet, LAN (Local Area Network), Wireless LAN (Wireless) Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), Bluetooth (Bluetooth) network, NFC network, satellite broadcasting network, analog broadcasting network, Digital Multimedia Broadcasting (DMB) network, etc. are included, but are not limited thereto. does not

The main server 40 is a server that provides a web page, an app page, a program or an application related to the operating method 1000 of the parking control system that precisely performs the reading of the license plate of the present invention and the determination of forgery and contamination. can

The operating method 1000 of the parking control system that precisely performs the reading of the license plate of the present invention and the determination of forgery and contamination of the present invention is that the image sensor 10 senses the license plate to generate image data, and the main server sending (100) to (40); The depth sensor 20 senses the license plate, generates depth data, and transmits it to the main server 40 (200); The main server 40 using the image data and the depth data, reading the characters on the license plate and determining whether counterfeiting or contamination (300); may include.

In the step 100 of the image sensor 10 sensing the vehicle license plate to generate image data and transmitting to the main server 40, the image sensor 10 captures the license plate of the vehicle entering the parking lot control system. can In this case, the image data may be a dynamic image or a static image photographed by the image sensor 10 having each pixel unit, and may be expressed as two-dimensional information.

In the step 200 of the depth sensor 20 sensing the vehicle license plate to generate depth data and transmitting it to the main server 40 , scanning the license plate of the vehicle entering the parking lot control system by the depth sensor 20 is a step can In this case, the depth data may be a scanned image having each polygon unit scanned by the depth sensor 20 , and may be expressed as three-dimensional information.

In the step 300 of the main server 40 using the image data and the depth data to read the characters of the license plate and determine whether or not counterfeiting and contamination are present , the vehicle license plate captured by the image sensor 10 in the main server 40 By processing the two-dimensional information of and the three-dimensional information of the license plate captured by the depth sensor 20, it may be a step of reading the characters of the license plate and at the same time determining whether counterfeiting or contamination is present.

First, it will be described that the main server 40 reads the characters of the license plate using the image data and the depth data.

The main server 40 matches each of one or more characters on the license plate with random characters using a program that recognizes characters in image data to provide a similarity rate value for a character having the highest similarity value among random characters One similarity value is set, and each of one or more characters on the license plate is matched with arbitrary characters using a program that recognizes characters from depth data. Each of the one or more characters on the license plate may be read as a character set with a higher similarity value among the character set with the first similarity value and the character set with the second similarity value.

In detail, the main server 40 classifies one or more characters of the license plate into unit characters in the image data and the DApps data using a character recognition program, and sets the unit characters to arbitrary characters preset and stored in the database. can be matched with

The similarity rate value may be an index indicating the degree of similarity to the unit characters extracted from the image data and the DApps data and arbitrary characters preset and stored in the database as a percentage (that is, the similarity rate value means the similarity).

As an example, in the case of a unit character extracted from image data (eg, the actual number “8”), a similarity rate value to “1” among predetermined and stored arbitrary characters is 12%, and similarity to “6” is The rate value may be 50%, and the similarity rate value with “8” may be 92%. In this case, in relation to the unit character “8” extracted from the image data, arbitrary characters “1”, “6”, “ A character having the highest similarity value among "8" may be "8", and the first similarity value may be 92%.

In addition, in the case of a unit character extracted from the depth data (eg, the actual number “8”), a similarity rate value with “1” is 15% among predetermined and stored arbitrary characters, and a similarity rate value with “6” is 47%, and a similarity value with “8” may be 88%. In this case, in relation to the unit character “8” extracted from the depth data, arbitrary characters “1”, “6”, “8” A character having the highest similarity value among "8" may be "8", and the second similarity value may be 88%.

In this case, finally, the number "8" of the actual license plate is the letter "8" that sets the higher similarity value among the letter "8" that sets the first similarity value and the letter "8" that sets the second similarity value. 1 The character that sets the similarity value)" can be read.

In the above case, the character with the first similarity value and the character with the second similarity value are the same, but if the character with the first similarity value and the character with the second similarity value are different, the actual vehicle The characters on the license plate can be read as characters with a higher similarity value set. Furthermore, there is a problem when the first similarity value and the second similarity value are the same. In this case, it may be preferable to read the characters of the actual license plate as characters in which the first similarity value is set (because reading simple characters is a two-dimensional Because image data is considered to be more accurate than 3D image data).

On the other hand, as described above, one or more characters of the license plate can be independently read, it is possible to complete the license plate by combining the independently read characters.

Hereinafter, it will be described that the main server 40 determines whether the vehicle license plate is forged by using the depth data. Forgery of the license plate is made by attaching a counterfeit sticker on the license plate. In this case, there is a difference in the shape of the outline between the embossed characters on the actual license plate and the characters formed by the counterfeit sticker. can judge On the other hand, the shape of the outline causes a large difference in the shape and angle of the character protruding from the plane of the license plate plate (angle of bending; the angle between the plate and the boundary of the embossed character).

The main server 40 sets an error rate value for each of one or more characters of the license plate using a program for recognizing contours in the depth data, and if the average value of the error rate values set for each of the one or more characters is greater than or equal to the error rate reference value, it is forged It can be judged by the license plate of the vehicle.

In detail, in the main server 40, sample contour data for each character may be stored in a database, and contour data of a unit character extracted from the depth data and read as a specific character by the above-described reading process and corresponding thereto By comparing the sample contour data for the characters to be used, the error rate value can be set by expressing the difference between the shape of the protruding unit character and the angle of the unit character boundary protruding from the plate of the license plate as a percentage (%) as the degree of error (i.e., the error rate value can be set. The error rate value means the degree to which the error is large).

In this case, when the average value (eg, 62%) of one or more error rate values for one or more characters extracted from the depth data is greater than or equal to the error rate reference value (eg, 50%) preset in the database of the main server 40 , it may be determined that the license plate is forged.

On the other hand, as described above, whether or not the forgery of one or more characters of the license plate is determined as a whole, and it can be concluded that the license plate is forgery or not (for example, the reading of the license plate of the present invention and whether it is forged or contaminated In the operation method of the parking control system that precisely performs the judgment of the vehicle, only a specific part of the license plate cannot be judged to be a forgery).

Finally, a description will be given of determining whether the vehicle license plate is contaminated by the main server 40 using the image data and the depth data. The license plate may be contaminated by corpses of insects and muddy water, and the license plate may be determined to be contaminated when it shows a low degree of similarity despite a low probability of being forged.

The main server 40 determines that the high similarity value among the first similarity value and the second similarity value set for a specific character on the license plate is less than the similarity rate reference value, and the error rate value set for a specific character on the vehicle license plate is less than the error rate reference value. It can be determined that the text is contaminated.

In detail, the main server 40 may store a similarity rate reference value (for example, 80%), and a high similarity among a first similarity value and a second similarity value set for a specific character among one or more characters of a vehicle license plate When the rate value (set to the same value when the first similarity value and the second similarity value are the same) is less than the similarity rate reference value (for example, the higher similarity value between the first similarity value and the second similarity value of a specific character is 72%) there may be

In addition, an error rate reference value (for example, 50%) may be stored in the main server 40, and when the error rate value set for a specific character among one or more characters of the vehicle license plate is less than the error rate reference value (for example, of a specific character) There may be an error rate value of 30%).

In the above case, the similarity of a specific character is low, but the probability of forgery is also low, so that the surrounding area may be determined to be contaminated.

On the other hand, as described above, one or more characters on the vehicle license plate can independently determine whether or not they are contaminated, and accordingly, it is possible to set the vicinity of a specific contaminated character as a contaminated area.

Meanwhile, the main server 40 may generate a program for recognizing a character and a program for recognizing an outline by artificial intelligence learning.

Artificial intelligence (AI) is a field of computer engineering and information technology that realizes human learning ability, reasoning ability, perceptual ability, and natural language understanding ability through computer programs. In particular, supervised learning, which is used in the most fields of artificial intelligence, is a method of predicting a future value using training data including a correct answer (Label).

For reference, machine learning can be defined as a methodology that gives a computer the ability to learn on its own without being explicitly programmed, and this is a method that allows a program to learn patterns from data on its own based on data. Machine learning is divided into supervised learning and unsupervised learning depending on whether the correct answer is specified in the data required for learning. It is divided into regression, which maps to values, clustering, which groups similar data, and dimensional reduction, which maps multidimensional data to a representative lower dimension.

Deep learning, one of the machine learning techniques, has dramatically improved the performance of machine learning, which has been stagnant for a while. , as a deep learning structure, a deep neural network (DNN) in which a plurality of hidden layers exist between an input layer and an output layer, and a filter required for factor extraction in front of the hidden layer Convolutional Neural Network (CNN) that learns filters together with Here, the high performance of the deep learning model is explained in two ways. First, the neural network is a universal approximator that can approximate all kinds of functions by superimposing the weighted sum of functions in each layer. Data can be simulated with high accuracy. Second, in order to classify data well, it is important to properly extract the factors representing the data, and it is possible to extract the optimal factors through filter learning using a spiral neural network. In addition, deep learning is a developed form of a model in the field of artificial intelligence called a neural network. Recent deep learning models have many inner layers, so the number of weights (meaning the strength of connection) connecting nodes can be up to several billions.

Accordingly, the main server 40 sets a plurality of image data captured in the past as feature values, and sets the similarity rate value obtained by matching it with arbitrary characters preset and stored in the database as a label value to learn It is possible to create an artificial intelligence prediction program. As a result, when the current image data is input, a program for recognizing characters capable of outputting a similarity rate value predicted by artificial intelligence can be generated.

In addition, in the main server 40, a plurality of depth data captured in the past is set as a feature value, and an error rate value obtained by matching this with the sample contour data preset and stored in the database is set as a label value, and artificially through learning. An intelligent prediction program can be created. As a result, when current depth data is input, a program for recognizing an outline capable of outputting an error rate value predicted by artificial intelligence may be generated.

The operating method 1000 of the parking control system that precisely performs the reading of the license plate of the present invention described above and the determination of forgery and contamination is implemented as a program (or application) to be executed in combination with a server, which is hardware. may be stored in the medium.

The above-described program is C, C++, JAVA, machine language, etc. that a processor (CPU) of the computer can read through a device interface of the computer in order for the computer to read the program and execute the methods implemented as a program It may include code (Code) coded in the computer language of Such code may include functional code related to a function defining functions necessary for executing the methods, etc., and includes an execution procedure related control code necessary for the processor of the computer to execute the functions according to a predetermined procedure. can do. In addition, this code may further include additional information necessary for the processor of the computer to execute the functions or code related to memory reference for which location (address address) in the internal or external memory of the computer should be referenced. have. In addition, when the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the functions, the code uses the communication module of the computer to determine how to communicate with any other computer or server remotely. It may further include a communication-related code for whether to communicate and what information or media to transmit and receive during communication.

The storage medium is not a medium that stores data for a short moment, such as a register, a cache, a memory, etc., but a medium that stores data semi-permanently and can be read by a device. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and an optical data storage device. That is, the program may be stored in various recording media on various servers accessible by the computer or in various recording media on the computer of the user. In addition, the medium may be distributed in a computer system connected to a network, and a computer-readable code may be stored in a distributed manner.

The steps of a method or algorithm described in relation to an embodiment of the present invention may be implemented directly in hardware, as a software module executed by hardware, or by a combination thereof. A software module may contain random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present invention pertains.

As mentioned above, although embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains know that the present invention may be embodied in other specific forms without changing the technical spirit or essential features thereof. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

As mentioned above, although embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains know that the present invention may be embodied in other specific forms without changing the technical spirit or essential features thereof. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

Claims (5)

generating image data by the image sensor sensing the license plate and transmitting the image data to the main server;
generating depth data by sensing the license plate by the lidar and transmitting it to the main server; and
The main server using the image data and the depth data, comprising the step of reading the characters of the license plate and determining whether counterfeiting or contamination,
The main server matches each of one or more characters on the license plate with random characters using a program for recognizing characters in image data to match the similarity rate value for the character having the highest similarity value among the random characters to the first similarity Set as a rate value, match each of one or more characters on the license plate with arbitrary characters using a program that recognizes characters in depth data, and set the similarity value for the character with the highest similarity value as the second similarity value, ,
Each of the one or more characters on the license plate is read as a character set with a higher similarity value among the character set with the first similarity value and the character set with the second similarity value. A method of operating a parking control system that precisely performs the judgment of
According to claim 1,
The image data is a dynamic image or a static image that is captured by the image sensor and has each pixel unit,
Depth data is scanned by the lidar and is a scanning image having each polygon unit. A method of operating a parking control system to precisely read a license plate and determine whether forgery or contamination is present.
The method of claim 1,
The main server sets an error rate value for each of one or more characters of a license plate using a program for recognizing contours in depth data,
When the average value of the error rate values set for each of one or more characters is greater than or equal to the error rate reference value, it is determined that the license plate is forged.
4. The method of claim 3,
The main server determines that the high similarity value among the first similarity value and the second similarity value set for a specific character on the license plate is less than the similarity rate reference value, and when the error rate value set for a specific character on the license plate is less than the error rate reference value, the corresponding character is An operating method of a parking control system that precisely performs the reading of a license plate and the determination of forgery and contamination, characterized in that it is determined to be contaminated.
5. The method of claim 4,
The main server is an operating method of a parking control system that precisely performs the determination of whether the vehicle license plate is read and forged and contaminated, characterized in that it generates a program for recognizing characters and a program for recognizing contours by artificial intelligence learning.

Images