KR20220071771A - System, apparatus and method for recognizing license plate of vehicle and appratus for managing the same - Google Patents

Abstract

The present invention relates to a license plate identification system, apparatus, and method to identify a license plate clearly and a license plate identification management apparatus. According to the present invention, the license plate identification system comprises a license plate identification apparatus constructing an application pipeline; determining an appropriate learning model for analysis for the application pipeline; receiving the learning model for analysis by being communicatively connected to a license plate identification management apparatus; acquiring at least one image data to be analyzed by performing photographing; and applying the learning model for analysis to the at least one image data to be analyzed to detect and acquire a license plate number from the at least one license plate to be analyzed from the at least one image data to be analyzed.

Description

SYSTEM, APPARATUS AND METHOD FOR RECOGNIZING LICENSE PLATE OF VEHICLE AND APPRATUS FOR MANAGING THE SAME

It relates to a license plate recognition system, method, apparatus, and a license plate recognition management device.

In the past, when a vehicle enters a parking lot, the driver of the vehicle obtains a parking card or the like, or a parking management agent records the vehicle entering the parking lot and approves the vehicle entry, and when the vehicle exits, parking management The agent checked the vehicle number or received a parking card and used it to settle the parking fee, etc. or to control the blocking bar. Recently, a separately provided parking management system automatically determines whether the vehicle enters or leaves the vehicle, checks whether payment is made, etc., and then controls the blocking bar in response thereto. However, these systems often have errors in their operation at night or in extreme conditions (heavy rain, heavy snow or fog, etc.). A lot of money was invested in hardware for accurate entry and exit management, which caused an increase in installation, maintenance, and management costs for the system.

It is a task to solve the problem to provide a license plate recognition system, method, apparatus and license plate recognition management device that can clearly and clearly recognize the license plate of the vehicle and accurately acquire information therefrom.

In order to solve the above problems, a vehicle license plate recognition system, a terminal device for vehicle license plate recognition, a vehicle license plate recognition management device for vehicle license plate recognition, and a vehicle license plate recognition method are provided.

The license plate recognition system builds an application pipeline, determines a learning model for analysis appropriate for the application pipeline, and is communicatively connected with the license plate recognition management device and the license plate recognition management device to obtain the learning model for analysis At least one target vehicle license plate of the at least one analysis target image data by receiving, performing photographing to obtain at least one analysis target image data, and applying the learning model for analysis to the at least one analysis target image data It may include a vehicle license plate recognition device for detecting and obtaining the vehicle number from.

The license plate recognition management apparatus may determine a learning model for analysis by determining a learning model suitable for the at least one application pipeline from at least one candidate learning model.

The license plate recognition management apparatus may acquire at least one image data including at least one license plate, and train the candidate learning model by using the acquired at least one image data.

The license plate recognition management apparatus may further inspect the performance of the application pipeline, and determine the learning model for analysis according to the inspection result.

The vehicle license plate recognition apparatus detects the at least one target vehicle license plate from the at least one analysis target image data, and image quality improvement processing for the at least one analysis target image data from which the at least one target vehicle license plate is detected and obtain a vehicle number from the at least one target vehicle license plate detected from the at least one analysis target image data that has been subjected to image quality improvement processing using the learning model for analysis.

The vehicle license plate recognition system may further include a terminal device that is communicatively connected to the vehicle license plate recognition device and receives the vehicle number obtained from the at least one target vehicle license plate from the vehicle license plate recognition device.

The license plate recognition device may belong to any one recognition device group among a plurality of recognition device groups.

The vehicle license plate recognition method includes the steps of, by the license plate recognition management device, determining a learning model for analysis for the application pipeline, at least one license plate recognition device communicatively connected to the license plate recognition management device, the learning model for analysis Receiving the step, the at least one license plate recognition device performing the photographing to obtain at least one analysis target image data, and the at least one license plate recognition device analyzing the at least one analysis target image data It may include applying a learning model for detecting and obtaining a vehicle number from at least one target vehicle license plate of the at least one analysis target image data.

The step of determining, by the license plate recognition management device, a learning model for analysis suitable for an application pipeline, determines the learning model for analysis by determining a learning model suitable for the at least one application pipeline from at least one candidate learning model may include the step of

The vehicle license plate recognition method includes: acquiring, by the license plate recognition management apparatus, at least one image data including at least one license plate; and training the candidate learning model using the acquired at least one image data may further include.

The vehicle license plate recognition method may further include the step of inspecting, by the license plate recognition management apparatus, the performance of the application pipeline.

Step by which the at least one license plate recognition device detects and obtains a vehicle number from at least one target license plate of the at least one analysis target image data by applying the learning model for analysis to the at least one analysis target image data is, detecting the at least one target vehicle license plate from the at least one analysis target image data, performing image quality improvement processing on the at least one analysis target image data from which the at least one target vehicle license plate is detected and obtaining a vehicle number from the at least one target vehicle license plate detected from at least one analysis target image data that has been processed to improve image quality by using the learning model for analysis.

The vehicle license plate recognition method may further include the step of receiving, by the terminal device communicatively connected to the vehicle license plate recognition device, the vehicle number obtained from the at least one target vehicle license plate from the vehicle license plate recognition device.

The license plate recognition management device is configured to build an application pipeline, and use at least one candidate learning model to determine a learning model for analysis suitable for the application pipeline, and at least one vehicle corresponding to the learning model for analysis. It may include a communication unit for transmitting to the license plate recognition device.

The processor may determine the learning model for analysis by determining a candidate learning model suitable for the at least one application pipeline from among at least one candidate learning model.

The vehicle license plate recognition device includes a communication unit for receiving at least one learning model for analysis, a photographing unit for obtaining at least one analysis target image data by performing shooting, and a learning model for analysis in the at least one analysis target image data It may include a processor for detecting and obtaining at least one of letters and numbers from at least one target vehicle license plate of the at least one analysis target image data by application.

The processor detects the at least one target vehicle license plate from the at least one analysis target image data, and performs image quality improvement processing on the at least one analysis target image data from which the at least one target vehicle license plate is detected, , it is possible to obtain a vehicle number from the at least one target vehicle license plate detected from the at least one analysis target image data that has been processed for image quality improvement by using the learning model for analysis.

According to the above-described vehicle license plate recognition system, method, apparatus and license plate recognition management device, the license plate of a vehicle entering a parking lot, etc. can be recognized more clearly and clearly and information such as a license plate can be accurately obtained from the license plate of the recognized vehicle. there will be

According to the vehicle license plate recognition system, method, apparatus and license plate recognition management device described above, even when the license plate is photographed using a relatively inexpensive camera instead of a high-performance camera device, the image quality of the license plate can be improved. This makes it possible to recognize the license plate more clearly.

According to the above-described vehicle license plate recognition system, method, apparatus and license plate recognition management device, there is also an advantage of being able to more clearly recognize the license plate of the vehicle even in extreme conditions such as night or bad weather.

According to the vehicle license plate recognition system, method, apparatus and license plate recognition management device described above, it is possible to provide, manage, or update a learning model to be used individually or unified for each of a plurality of license plate recognition devices, so that a plurality of license plates There is also an effect of enabling the integrated management of the recognition device.

According to the vehicle license plate recognition system, method, apparatus and license plate recognition management device described above, it is possible to efficiently and effectively manage and improve the device for license plate recognition, as well as to build a system using a relatively inexpensive camera. As a result, the overall cost of construction, maintenance and management can be reduced, and the machine learning model used for recognizing license plates can be more easily updated, so that the ease of management can be obtained. have.

1 is a schematic diagram of an embodiment of a license plate recognition system.
2 is a block diagram of an embodiment of a vehicle license plate recognition device.
3 is a view showing an example in which the license plate is recognized by the vehicle license plate recognition device.
Figure 4 is a block diagram of an embodiment of the license plate recognition management apparatus.
5 is a flowchart of an embodiment of a vehicle license plate recognition method.

In the following specification, the same reference numerals refer to the same components unless otherwise specified. A term to which a 'unit' is added used below may be implemented in software or hardware, and according to an embodiment, one 'unit' may be implemented as one physical or logical part, or a plurality of 'units' may be implemented as one It is also possible to be implemented with physical or logical parts, or one 'unit' may be implemented with a plurality of physical or logical parts. Throughout the specification, when it is said that a certain part is connected to another part, it may mean a physical connection or electrically connected according to a certain part and another part. In addition, when it is said that a part includes another part, this does not exclude another part other than the other part unless otherwise stated, and it means that another part may be further included according to the designer's choice do. Terms such as first and second are used to distinguish one part from another, and unless otherwise specified, they do not mean sequential expressions. Also, the singular expression may include the plural expression unless the context clearly dictates otherwise.

Hereinafter, various embodiments of a vehicle license plate recognition system, a vehicle license plate recognition device, and a vehicle license plate recognition management device will be described with reference to FIGS. 1 to 4 .

1 is a schematic diagram of an embodiment of a license plate recognition system.

1, the license plate recognition system 1, in one embodiment, at least one license plate recognition device (100: 100-1 to 100-K, K is a natural number greater than or equal to 1) and at least It may include a license plate recognition management device 200 provided to be able to communicate with one license plate recognition device 100, and if necessary, at least one license plate recognition device 100: 100-1 to 100-K, and It may include one or more terminal devices 3 capable of performing communication with at least one of the license plate recognition management apparatus 200 . In this case, at least one license plate recognition device (100: 100-1 to 100-K) and the license plate recognition management device 200 are mutually using the wired/wireless communication network 2 to use an application (app, software or program, etc.) It can be referred to as ) and is provided to transmit and receive data. The wired/wireless communication network 2 may be constructed based on a wired communication network, a wireless communication network, or a combination thereof. Here, the wired communication network may be provided using a cable. The wireless communication network may include at least one of a short-range communication network and a telecommunication network. The short-range communication network is, for example, Wi-Fi, Wi-Fi Direct, Bluetooth, Bluetooth Low Energy, Zigbee communication, RFID: Radio -Frequency IDentification) and/or a network implemented based on a short-range communication technology such as Near Field Communication (NFC) may be included. The telecommunication network may include, for example, a mobile communication network implemented based on a mobile communication standard such as 3GPP, 3GPP2, WiBro or WiMAX series. At least one terminal device 3 is also through such a wired/wireless communication network 2, the license plate recognition device (100: 100-1 to 100-K) and at least one of the license plate recognition management device 200 and data such as Transmission and reception can be performed.

At least one license plate recognition device (100: 100-1 to 100-K) acquires at least one image data (10 in FIG. 3, hereinafter analysis target image data) for the front or side of the vehicle (not shown) And, it is possible to obtain a vehicle number by recognizing the vehicle license plate from the at least one analysis target image data (10). At least one license plate recognition device (100: 100-1 to 100-K) may be implemented using a camera device such as a closed circuit camera (CCTV) or a license plate recognition system (LPR: License Plate Recognition system), etc. Alternatively, it may be implemented using an electronic device specially manufactured for recognizing a license plate. In addition, the vehicle license plate recognition device (100: 100-1 to 100-K) may be implemented using a smart phone, a tablet PC, a navigation device and/or a black box device. According to an embodiment, the license plate recognition system 1 may include a plurality of license plate recognition devices 100: 100-1 to 100-K always. Here, at least two of the plurality of license plate recognition devices (100: 100-1 to 100-K) are arranged in different remote locations and may be geographically dispersed. A detailed operation of the license plate recognition device 100: 100-1 to 100-K will be described later.

According to one embodiment, the vehicle license plate recognition device (100: 100-1 to 100-K) belongs to one or more recognition device groups (R-1, R-2 to R-K) of any one recognition device group. it could be Here, the recognition device group (R-1, R-2 to R-K) may be a group including one or more license plate recognition devices 100-1 to 100-K, respectively. For example, as shown in FIG. 1 , the first recognition device group R-1 includes the first license plate recognition device 100-1, and the second recognition device group R-2 includes the second recognition device group R-2. 2 to the fourth license plate recognition device 100-2 to 100-4, and the K-th recognition device group (R-K) is the (K-3) to K-th license plate recognition device 100-(K-3). to 100-K). However, this is exemplary and may include the same or different number of license plate recognition devices 100-1 to 100-K for each recognition device group R-1, R-2 to R-K. Also, here, the license plate recognition devices 100-1 to 100-K included in each of the recognition device groups R-1, R-2 to R-K may be implemented with all the same or different devices, or some may be implemented using devices that are homogeneous and some are heterogeneous. According to an embodiment, each of the recognition device groups R-1, R-2 to R-K may be classified according to the region in which the license plate recognition devices 100-1 to 100-K are disposed. That is, the license plate recognition devices 100-1 to 100-K in the same recognition device group R-1, R-2 to R-K are located in proximity to each other, and different recognition device groups R-1, R-2 to R-K) and the license plate recognition devices 100-1 to 100-K may be located at a remote location from each other. More specifically, for example, the vehicle license plate recognition apparatus 100-1 of the first recognition device group R-1, the license plate recognition apparatus 100-2 to the license plate recognition apparatus 100-2 of the other second recognition device group R-2 100-4) and may be located in a different city or country. Of course, according to the embodiment, it is also possible for a plurality of license plate recognition devices 100-1 to 100-K disposed relatively far from each other to be bundled into one recognition device group R-1, R-2 to R-K. According to the embodiment, all the license plate recognition devices 100-2 to 100-4 in one recognition device group, for example, the second recognition device group R-2, have the same learning model (98 in FIG. 4, below). Learning model for analysis) or may be provided to perform a vehicle license plate recognition process using an application, in this case, the license plate recognition management device 200 is all the license plate recognition devices in the same group (R-2) (100- 2 to 100-4) by transmitting the same learning model 98 or application for analysis, the license plate recognition device 100 in the same group R-2 recognizes the vehicle through the same learning model 98 or application for analysis may be made to do.

The license plate recognition management device 200 determines the learning model 98 for each operation of the license plate recognition device 100: 100-1 to 100-K, that is, the learning model for analysis, and the determined analysis learning The model may be transmitted simultaneously or sequentially to at least one of the license plate recognition devices 100: 100-1 to 100-K. The license plate recognition management device 200 can be implemented using one or more information processing devices, for example, a server computer device, a desktop computer, a laptop computer, a smart phone, a tablet PC, a smart watch, a head mounted display ( Head Mounted Display (HMD) devices, navigation devices, portable game machines, personal digital assistants (PDA), digital televisions, set-top boxes, home appliances (such as refrigerators and robot vacuums), artificial intelligence sound reproduction devices (artificial intelligence speakers) ), a vehicle, a manned vehicle, an unmanned aerial vehicle, a robot and/or an industrial machine, and may be implemented using various types of information processing devices capable of processing and transmitting/receiving data in addition to the illustrated ones.

The terminal device 3 is connected to at least one of the license plate recognition devices 100: 100-1 to 100-K, and a terminal from at least one of the connected license plate recognition devices 100: 100-1 to 100-K. The user of the device 3 may obtain desired information and provide it to the user. Here, the information desired by the user is, for example, a vehicle number obtained according to the vehicle license plate recognition result or a processing result for a vehicle corresponding to the obtained vehicle number (for example, whether access permission or payment is made), etc. may include The terminal device 3 is a vehicle license plate recognition device 100: 100-1 to 100-K according to the operation of the external connection management unit (123 in FIG. 2) provided in the vehicle license plate recognition device 100: 100-1 to 100-K ) may receive the acquired information. In this case, the terminal device 3 may acquire these information by using an application programming interface (API) provided by the external connection management unit 123 . The terminal device 3 is, according to the embodiment, a desktop computer, a laptop computer, a smart phone, a tablet PC, a smart watch, a head mounted display device, a navigation device, a portable game console, a digital television, an electronic billboard, a set top box, a home appliance, It may include, but is not limited to, an artificial intelligence sound reproducing device, a vehicle, an air vehicle, or an industrial machine.

Hereinafter, with reference to Figure 2, the above-described vehicle license plate recognition device (100: 100-1 to 100-K) will be described.

2 is a block diagram of an embodiment of a vehicle license plate recognition device.

As shown in FIG. 2 , the vehicle license plate recognition device 100 includes a photographing unit 111 , a sensing unit 113 , a communication unit 115 , an input/output unit 117 , a storage unit 119 and a processor 120 . may be included, and at least one of them 111 to 119 may be omitted according to the designer's arbitrary selection.

The photographing unit 111 may acquire image data for a photographing direction (ie, analysis target image data) using visible light or infrared rays, and transmit it to the processor 120 . According to an embodiment, when a signal for vehicle detection is transmitted from the sensor 113 or the processor 120 , the photographing unit 111 may perform photographing in response to acquire the analysis target image data 10 . The image data 10 to be analyzed obtained by the photographing unit 111 may include image data of relatively low quality. For example, when the photographing unit 111 is implemented using a camera device manufactured inexpensively using a relatively low-resolution image pickup device, the obtained analysis target image data 10 may be low-resolution image data. . In addition, the image data 10 to be analyzed may be out of focus, shake, overall dark, or have noise above a certain level depending on the quality of the employed camera or changes in the surrounding environment according to weather or time.

The sensing unit 113 may detect the movement or presence of the vehicle, output a signal corresponding to the sensing, and transmit it to the photographing unit 111 or the processor 120 . Accordingly, the photographing unit 111 is able to photograph the vehicle. The sensing unit 113 is implemented using an optical sensor that detects infrared rays or lasers, a magnetic field sensor that detects a magnetic field, or a pressure-sensitive sensor (piezoelectric sensor, etc.) that detects pressure applied by the vehicle, according to an embodiment. It is possible.

The communication unit 115 is provided so that the vehicle license plate recognition device 100 communicates with at least one of the license plate recognition management device 200 and the terminal 3 by wire or by connecting to a wireless communication network. For example, the communication unit 115 may receive a learning model or application for analysis from the license plate recognition management device 200 , and an analysis target image obtained by the photographing unit 111 or the processor 120 obtained by the The license plate analysis result (ie, the license plate number of the corresponding vehicle) may be transmitted to at least one of the license plate recognition management apparatus 200 and the terminal device 3 . The communication unit 115 may include a wired cable terminal and a LAN card according to an embodiment, and may include a communication chip, a circuit board on which the communication chip is mounted, an antenna, and the like.

The input/output unit 117 receives a command or data input from a user (which may include a designer or manager of the vehicle license plate recognition device 100 and/or a driver or passenger of a vehicle, etc.) and/or necessary for the user Information (eg, the image data to be analyzed 10, the recognized vehicle number, the type of vehicle, whether entry is approved, entry time, exit time, interval between entry and exit times (eg, parking time, etc.) and / or fees, etc.) may be provided visually and/or audibly. Accordingly, the user inputs desired data (eg, information on a designated vehicle, etc.) into the vehicle license plate recognition device 100 and/or whether the vehicle number is recognized from the vehicle license plate recognition device 100 or the vehicle number recognition It is possible to easily grasp the accuracy of the vehicle number and the processing result corresponding to vehicle number recognition (eg, fare, etc.). The input/output unit 117 may include at least one of an input unit and an output unit, wherein the input unit includes, for example, a physical button, a keyboard, a keypad, a card receiver, a touch screen, a touch pad, a scanner, a microphone, It may include a mouse, tablet input device, trackball, trackpad, and/or data input/output terminal (such as a universal serial bus terminal), and the output unit may include a display, a printer device, a speaker device, an image output terminal and/or a data input/output terminal, etc. may include, but is not limited to the above-described examples.

The storage unit 119 may temporarily or non-temporarily store various information or applications required for the operation of the vehicle license plate recognition device 100 . For example, the storage unit 119 may store the received learning model or application for analysis, and provide the learning model or application for analysis to the processor 120 according to the call of the processor 120 . Also, the storage unit 119 stores the analysis target image data 10 obtained by the photographing unit 111, or a vehicle license plate (hereinafter, target vehicle license plate) or target vehicle license plate detected by the processor 120 from the analysis target image data 10 It is also possible to store the vehicle number (which may consist of letters and/or numbers) of the corresponding vehicle obtained from the . The storage unit 119 may include, for example, at least one of a main memory device (ROM or RAM) and an auxiliary memory device (a flash memory device, a solid state drive, a hard disk drive, etc.).

The processor 120 may recognize a vehicle license plate (ie, a target vehicle license plate) of a photographed vehicle using the analysis target image data 10 , and as a result obtain a vehicle number for the photographed vehicle. To this end, the processor 120 may use the application stored in the storage unit 119 to perform a license plate recognition and license plate acquisition process. In this case, all or part of the application may be transmitted from the license plate recognition management apparatus 200 . When a part of the application is transmitted from the license plate recognition management device 200 , according to an embodiment, the learning model 98 for analysis included in or used in the application may be transmitted from the license plate recognition management device 200 . In addition, the processor 110 obtains a new application for vehicle recognition and analysis by modifying and updating the existing application using the learning model 98 for analysis received from the license plate recognition management device 200, and the acquired The new application can also be used to recognize the target license plate and obtain the license plate number. In addition, the processor 120 may control the overall operation of the vehicle license plate recognition apparatus 100 . For example, the processor 120 may receive the detection signal output by the detection unit 113 , generate a photographing command in response to the received detection signal, and then transmit the photographing command to the photographing unit 111 . According to one embodiment, the processor 120 is, for example, a central processing unit (CPU, Central Processing Unit), a micro controller unit (MCU, Micro Controller Unit), a microcomputer (Micom, Micro Processor), an application processor (AP). , Application Processor), an electronic control unit (ECU, Electronic Controlling Unit), and/or other electronic devices capable of processing various calculations and generating control signals, and the like. These may be implemented using one or two or more semiconductor chips.

According to an embodiment, the processor 120 may include an image processing unit 130 , an application processing unit 121 , and an external connection management unit 123 as shown in FIG. 2 . At least one of the application processing unit 121 and the external connection management unit 123 may be omitted. The image processing unit 130 , the application processing unit 121 , and the external connection management unit 123 may be logically or physically separated, and when physically separated, the image processing unit 130 and the application processing unit 121 . ) and the external connection management unit 123 may be implemented using different semiconductor chips.

3 is a view showing an example in which the license plate is recognized by the vehicle license plate recognition device.

The image processing unit 130 may analyze the analysis target image data 10 to obtain a vehicle license plate, and further acquire a vehicle number if necessary. Specifically, for example, the image processing unit, as shown in FIG. 2 , may include a license plate detection unit 131 , an image quality improving unit 133 , a character recognition unit 135 , and a result output unit 137 .

The license plate detection unit 131 may receive the analysis target image data 10 obtained by the photographing unit 111 as shown in FIG. 3 . Here, the analysis target image data 10 may be an image of low quality as described above. The license plate detection unit 131 may analyze the received analysis target image data 10 to determine whether the vehicle license plate 9 is present in the analysis target image data 10 . Here, the detection unit 131 may check the existence of the vehicle license plate 9 by using the existence of a shape identical to or similar to the shape of the vehicle license plate 9 externally in the image data 10 to be analyzed, and if necessary Accordingly, the vehicle license plate 9 may be detected from the analysis target image data 10 with reference to the position of the license plate 9 with respect to the overall appearance of the vehicle. According to an embodiment, the license plate detection unit 131 may use at least one learning model to detect the license plate 9 from the analysis target image data 10 . For example, the detection unit 131 detects a shape corresponding to the license plate 9, or at least one learning model for identifying the overall appearance of the vehicle and identifying the location of the license plate 9 with respect to the overall appearance of the vehicle. It is also possible to use At least one learning model may be transmitted from the license plate recognition management apparatus 200 . Here, at least one learning model is, according to an embodiment, a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a long-term memory (LSTM: At least one available for image analysis, such as long short term memory), Generative Adversarial Network (GAN), Conditional GAN (cGAN), and/or Deep Reinforcement Learning. of the learning model.

When the position identification and detection of the vehicle license plate 9 is completed, the image quality improvement unit 133 performs an improvement process on the image quality of at least one analysis target image data 10 to improve the image quality of at least one analysis target image Data 11 can be obtained. The image quality improvement process may be performed by adjusting the contrast or brightness of the image data 10 to be analyzed, removing noise, or clearly correcting an unclear part. The image quality enhancement unit 133 may also use at least one learning model for image quality improvement, and the learning model used here may be the same as or different from the learning model used by the license plate detection unit 131 . At least one learning model used by the image quality improvement unit 133 may also be transmitted from the license plate recognition management apparatus 200 . The image quality enhancing unit 133 may be omitted if necessary.

The character recognition unit 135 receives the license plate 9 from the image data 10 in which the license plate detection unit 131 detects the license plate 9 or the image data 11 on which the image quality improvement process is performed by the image quality improvement unit 133 . ), and by recognizing letters or numbers 9A from the extracted license plate 9, it is possible to obtain a vehicle number for the vehicle. The character recognition unit 135 may recognize letters or numbers from the license plate 9 using at least one predetermined character recognition algorithm, and the predetermined character recognition algorithm may be built based on at least one learning model. have. Here, at least one learning model used in the character recognition algorithm may also be transmitted from the license plate recognition management apparatus 200 . In addition, the learning model used by the character recognition unit 135 may be the same as or different from the learning model used by the license plate detection unit 131 or the image quality improving unit 133 .

The result output unit 137 may output the character recognition result of the character recognition unit 135 in the form of an electrical signal and transmit it to any one of the communication unit 115 , the input/output unit 117 , and the storage unit 119 .

When the application processing unit 121 transmits at least one learning model 98, the license plate recognition management device 200 adds the learning model 98 to an existing application or at least one learning model 98 received ) by replacing at least one learning model of the existing application corresponding to the application may be modified. The modified application may be delivered to the image processing unit 130, and the image processing unit 130 uses the received application to detect the license plate 131, improve the image quality (133, can be omitted), character recognition 135, and output the result. The operation of (137) can be performed.

The external connection management unit 123 may analyze the request transmitted from the terminal device 3 according to the connection and request of the terminal device 3 , and cause the vehicle license plate recognition device 100 to operate according to the request. For example, the external connection management unit 123, the information requested to the terminal device 3 according to the information transfer request transmitted from the terminal device 3 (for example, license plate recognition result or charging processing operation result according to this) I, the result of operation control of a circuit breaker, etc.) is determined and obtained, and the communication unit 115 is controlled to transmit the requested information to the terminal device 3 . According to an embodiment, the external access management unit 123 may authenticate the terminal device 3 or the user of the terminal device 3 who requested the connection, and determine whether to process the transmitted request according to the authentication result. The external connection management unit 123 may be implemented based on, for example, an application program programming interface.

In addition to the above, the processor 120 may perform various operations according to embodiments. For example, when the vehicle number is confirmed, the processor 120 controls the input/output unit 117 to visually or aurally output the confirmed vehicle number to the outside, or based on the confirmed vehicle number, the entry and exit time. A predetermined operation, such as determining a fee (parking fee, etc.) based on the determined entry and exit time, or determining whether to enter or exit the vehicle and control the operation of the blocking bar corresponding to the determination can be performed. Such an operation may be different depending on the installation location of the license plate recognition device 100 (eg, a general road or a parking lot access road, etc.). In addition, the processor 120 may further perform various operations according to the designer's selection.

Hereinafter, an embodiment of the vehicle license plate recognition management apparatus 200 will be described with reference to FIG. 4 .

Figure 4 is a block diagram of an embodiment of the license plate recognition management apparatus.

Referring to FIG. 4 , the license plate recognition management apparatus 200, in one embodiment, includes a communication unit 211 , an input/output unit 213 , which may be omitted), a storage unit 215 and a processor 220 . may include

The communication unit 211 may connect to a wired/wireless communication network and communicate with at least one license plate recognition device 100 to transmit and receive data, for example, the first license plate recognition device 100-1. , may be communicatively connected to at least one of the second vehicle license plate recognition apparatus 100-2 to the K-th vehicle license plate recognition apparatus 100-K to transmit and receive data. In this case, the communication unit 211 may deliver the learning model 98 obtained by the processor 220 or an application including the learning model to at least one license plate recognition device 100: 100-1 to 100-K. . Here, the learning model 98 or application transmitted by the communication unit 211 may be the same or different for each license plate recognition device 100 . The transmission of the learning model 98 or application to the license plate recognition device 100: 100-1 to 100-K may be performed every time the learning model 98 is acquired or modified for each license plate recognition device 100 have. According to the embodiment, when a plurality of license plate recognition devices (eg, 100-2 to 100-4) exist in any one recognition device group (R-1 to R-K), the learning model 98 or the application It may be transmitted sequentially or simultaneously to each of the license plate recognition devices (eg 100-2 to 100-4) belonging to the recognition device group (eg R-2). The communication unit 211 may include a wired cable terminal and a LAN card, and the like, according to the same embodiment as the communication unit 115 of the vehicle license plate recognition device 100, and a communication chip, a circuit board on which the communication chip is mounted, and an antenna. and the like.

The input/output unit 213 may receive data or commands for the vehicle license plate recognition management device 200 and/or output data or applications acquired or stored by the vehicle license plate recognition management device 200 to the outside. have. The input/output unit 213 may include at least one of an input unit and an output unit, similarly to the input/output unit 117 of the vehicle license plate recognition device 100, and the input unit may include a keyboard, a mouse device, or a data input/output terminal. and the output unit may include a display or a data input/output terminal.

The storage unit 215 may temporarily or non-temporarily store at least one data or application required for the operation of the license plate recognition management apparatus 200 . For example, the storage unit 215 is at least one initial learning model received from the outside or defined by the user, the learning model 98 for analysis generated by the processor 220, or the learning model 98 for analysis ) or image data (97, at least one license plate recognition device 100: 100-1 to 100-K) received from the application or each license plate recognition device (100: 100-1 to 100-K), including may include the obtained analysis target image data 10) or an application pipeline 99 for generating the learning model 98 for analysis, and the like. Also, the storage unit 215 may store at least one application (which may include a command, a library, and/or a variable, etc.) provided for the operation of the processor 220 . The storage unit 215 may provide the stored data 97 and 98 or the application pipeline 98 to the processor 220 according to the call of the processor 220 . The storage unit 215 may be implemented using at least one of a main memory device and an auxiliary memory device.

The processor 220 may generate the learning model 98 for analysis to be provided to the at least one license plate recognition device 100: 100-1 to 100-K. According to an embodiment, the processor 220 may include a learning model processing unit 221 , a learning model distribution management unit 223 , and a performance checking unit 225 as shown in FIG. 4 . Among them, the learning model distribution management unit 223 and/or the performance inspection unit 225 may be omitted. Each of the learning model processing unit 221 , the learning model distribution management unit 223 , and the performance inspection unit 225 may be logically and/or physically separated according to an embodiment.

The learning model processing unit 221 may acquire the learning model for analysis 98 , and transmit the acquired learning model for analysis 98 to at least one of the communication unit 211 and the storage unit 215 . Specifically, the learning model processing unit 221 acquires the image data 97 separately input by a designer or manager and/or transmitted from at least one license plate recognition device 100: 100-1 to 100-K, By using the acquired image data 97 to train at least one pre-stored learning model (hereinafter, referred to as a candidate learning model) or an acquired learning model for analysis 98, a new or modified analysis target learning model 98 is obtained. can be obtained The learning model processing unit 211 is, according to an embodiment, at least one candidate differently depending on the at least one license plate recognition device 100 to which the analysis target learning model 98 is to be distributed (100: 100-1 to 100-K) By training the learning model or the previously acquired learning model 98 for analysis, at least one license plate recognition device (100: 100-1 to 100-K) corresponding to each of the analysis target learning model 98 is newly or modified. may be obtained. According to another embodiment, the learning model processing unit 211 trains at least one candidate learning model or the previously acquired learning model 98 for analysis differently for each of the recognition device groups R-1 to R-K. A new or modified analysis target learning model 98 corresponding to the recognition device group R-1 to R-K may be obtained. In this case, the analysis target learning model 98 obtained corresponding to each of the recognition device groups R-1 to R-K may be distributed to each of the recognition device groups R-1 to R-K, and accordingly, the same recognition device group At least one learning model recognition apparatus 100-1, 100-2 to 100-4, 100-(K-3) to 100-K belonging to (R-1 to R-K) is the same learning model 98 for analysis can be received and used. Here, the candidate learning model trained for the acquisition of the learning model for analysis 98 is, for example, a deep neural network, a convolutional neural network, a recurrent neural network, a long-term memory, a generative adversarial neural network, a conditional generative adversarial neural network and/or a Alternatively, it may include, but is not limited to, deep reinforcement learning.

According to an embodiment, the learning model processing unit 221 may acquire the appropriate learning model 98 for analysis by using the application pipeline 99 . Specifically, the learning model processing unit 221 may acquire at least one candidate learning model or at least one existing learning model for analysis 98 , and build an application pipeline at the same time, preceding or following it. The application pipeline to be built may be a vehicle license plate recognition device (100: 100-1 to 100-K) is provided in response to the application used for the recognition of the license plate (9). For example, the application pipeline 99, the license plate detection process 131, the image quality improvement process (133, may be omitted depending on the embodiment), the character recognition process 135 and the result output process (137, according to the embodiment) may be omitted), and may further include image data 97 to be input as necessary. The application pipeline 99 may be different or the same for each license plate recognition device 100: 100-1 to 100-K for which the learning model 98 for analysis is to be obtained. When the application pipeline 99 is built, the learning model processing unit 221 may, in response, apply at least one candidate learning model or a pre-obtained learning model for analysis 98 to the application pipeline 99 . . The learning model processing unit 221 sequentially acquires at least one candidate learning model optimal for the application pipeline through training using the image data 97 and analyzes the obtained at least one candidate learning model as a learning model 98 for analysis. By determining as ), the vehicle license plate recognition device (100: 100-1 to 100-K) acquires at least one learning model for analysis 98 to be used, or by using the image data 97 for an existing analysis learning model By retraining (98), it is also possible to obtain a learning model (98) for analysis that is optimal for the application pipeline.

The learning model processing unit 221 may perform training on the existing candidate learning model or the learning model for analysis 98 according to a predefined setting or according to an operation of a user, an administrator, a designer, or the like. If necessary, a user or an administrator may specify and input various values such as various setting values for learning (model training configuration, for example, setting values for model optimization or configuration). The training process of the learning model processing unit 221 may be performed periodically or aperiodically.

The learning model distribution management unit 223 may manage and process distribution of the acquired or modified at least one learning model for analysis 98 for analysis. Specifically, the learning model distribution management unit 223 may control the communication unit 211 to transmit the learning model 98 for analysis to the corresponding license plate recognition device 100: 100-1 to 100-K. For example, the learning model distribution management unit 223 transmits the image data 97 used for training the candidate learning model to obtain the learning model 98 for analysis (100: 100-1) to 100-K) or to a group of recognition devices (R-1 to R-K), it is possible to control the transmission of the learning model 98 for the corresponding analysis. Here, the delivered learning model 98 for analysis may include a candidate learning model optimal for the application pipeline 99 as described above. The learning model distribution management unit 223 may distribute the learning model 98 for analysis automatically according to a predefined setting or manually according to an operation of a user or the like.

The performance checker 225 may check the performance of the application pipeline 99 to which the learning model optimal for the application pipeline 99, that is, the learning model for analysis 98 is applied. The performance check unit 225 is a process included in each of the application pipeline 99, for example, the license plate detection process 131, the image quality improvement process 133, the character recognition process 135 and the result output process 137 for each You can check the performance. In this case, the performance test unit 225 may perform a performance test by inputting arbitrary image data 97 into each process 131 to 137 , checking the result, and determining a score corresponding to the result. The performance check of the performance checker 225 may be performed periodically or aperiodically, depending on an embodiment, or may be performed according to an operation of a user or an administrator. The test result of the performance tester 225 may be transmitted to the learning model processor 221 , and the learning model processor 221 sends the candidate learning model or the learning model 98 for analysis according to the test result of the performance tester 225 . training can be performed. For example, the learning model processing unit 221 adds a predetermined weight and/or applies a different candidate learning model to reinforce the processes 131 to 137 that have a relatively low score or do not meet the criteria for analysis learning. A model 98 may be obtained.

As described above, the learning model processing unit 221 of the processor 220 uses or does not use the performance inspection result of the performance inspection unit 225, and a candidate learning model or a previously obtained analysis in the application pipeline 99, etc. By applying the learning model 98 for, acquiring an optimal candidate learning model through training, or improving the previously acquired learning model 98 for analysis, at least one license plate recognition device 100: 100-1 to 100 -K) to obtain one or more learning models 98 for analysis. The communication unit 211 may transmit the learning model 98 for analysis obtained under the control of the learning model distribution management unit 223 of the processor 220 to the corresponding license plate recognition device 100: 100-1 to 100-K). There, the vehicle license plate recognition apparatus 100: 100-1 to 100-K recognizes the vehicle license plate from the analysis target image data 10 more appropriately using the received analysis learning model 98, and the vehicle license plate It is possible to obtain a vehicle number from Therefore, even when the license plate recognition device 100: 100-1 to 100-K can acquire only low-quality image data 10, the image data 10 is analyzed using the optimal learning model 99 and Based on this, it is possible to perform license plate recognition and license plate acquisition more efficiently and effectively. In addition, a plurality of license plate recognition devices (100: 100-1 to 100-K) exist, and each license plate recognition device (100: 100-1 to 100-K) is image data 10 of different quality from each other Even in the case of obtaining , the learning model 98 for analysis optimal for each license plate recognition device (100: 100-1 to 100-K) for each license plate recognition device (100: 100-1 to 100-K) ) can be generated, transmitted and used, so that the operating efficiency of each vehicle license plate recognition device 100: 100-1 to 100-K is improved.

Hereinafter, various embodiments of a vehicle license plate recognition method will be described with reference to FIG. 5 .

5 is a flowchart for an embodiment of a vehicle license plate recognition method.

According to an embodiment of the vehicle license plate recognition method shown in FIG. 5 , the vehicle license plate recognition management device that can be built using a computer device for a server, etc. may acquire at least one candidate learning model ( 400 ). Depending on the situation or embodiment, the license plate recognition management device, it is also possible to obtain the at least one previously generated learning model for analysis in place of the candidate learning model, or by substituting for the candidate learning model. Here, the at least one candidate learning model and/or the at least one learning model for analysis may be obtained in response to at least one license plate recognition device or at least one group of recognition devices to obtain the learning model for analysis. .

Following, preceding, or simultaneously with the acquisition of the candidate learning model or the learning model for analysis, the license plate recognition management apparatus may build an application pipeline ( 402 ). The application pipeline may correspond to processes to be processed by the vehicle license plate recognition device, and may include, for example, a license plate detection process, an image quality improvement process, a character recognition process, and a result output process. Here, the image quality improvement process or the result output process may be omitted. The construction of the application pipeline may be different or the same according to at least one license plate recognition device or at least one recognition device group to obtain a learning model for analysis.

When the application pipeline is built, the license plate recognition management device applies at least one candidate learning model to the application pipeline, and uses the existing or newly acquired image data to obtain and analyze the candidate learning model optimal for the application pipeline A learning model may be obtained ( 404 ). Here, the image data may be received from at least one license plate recognition device or at least one recognition device group that is a transmission target of the learning model for analysis. Accordingly, the optimal candidate learning model may be one obtained in response to at least one license plate recognition device or at least one group of recognition devices that is a transmission target of the learning model for analysis. That is, the training may be performed differently for each license plate recognition device or group of recognition devices. Of course, according to circumstances, the training performance of the learning model and the result may be the same between different license plate recognition devices or different groups of recognition devices. According to an embodiment, the license plate recognition management apparatus applies at least one learning model for analysis to the application pipeline, and re-trains the existing learning model for analysis using the existing or newly acquired image data to the application pipeline. An optimal candidate learning model may be obtained.

The learning model for analysis may be transmitted to the at least one license plate recognition device, and may be transmitted to the corresponding at least one license plate recognition device (408). Here, the corresponding at least one license plate recognition device may be at least one license plate recognition device that transmits image data used for training. In addition, at least one license plate recognition device may belong to one or two or more recognition device groups, in this case, the learning model for analysis may be transmitted to all license plate recognition devices in the same recognition device group. In other words, license plate recognition device(s) belonging to the same group of recognition devices may receive the same learning model for analysis. When the learning model for analysis is obtained, the license plate recognition device may perform application update by modifying the learning model for analysis of the existing application for license plate recognition using the learning model for analysis. If necessary, it is also possible that an application including a learning model for analysis is transferred from the license plate recognition management device to the license plate recognition device.

Vehicle license plate recognition device after receiving the learning model for analysis, before the reception of the learning model for analysis or at the same time as the reception of the learning model for analysis, by performing a photographing image data including the vehicle license plate (ie, analysis target image data) ) can be obtained (410).

When the analysis target image data is obtained, the vehicle license plate recognition device may perform image analysis using the acquired learning model for analysis. Specifically, the vehicle license plate recognition apparatus may detect the vehicle license plate from the obtained analysis target image data and determine whether there is a vehicle license plate (412). For example, it may be determined whether a vehicle license plate exists in the analysis target image data by using whether a shape identical to or similar to the shape of the vehicle license plate is externally present in the analysis target image data. If necessary, for this purpose, the position of the license plate with respect to the overall appearance of the vehicle may also be used. In addition, a learning model for analysis may be used for detection of the vehicle license plate. The learning model for analysis is available for detection of a shape corresponding to the license plate, identification of the overall appearance of the vehicle, or location of the license plate with respect to the overall appearance of the vehicle.

According to an embodiment, image data quality improvement processing may be sequentially performed ( 414 ). For example, an operation such as adjustment of contrast or brightness of image data or removal of noise may be performed. The image quality improvement operation may also be performed using the received learning model for analysis. The image quality improvement process 414 may be omitted in some embodiments.

Subsequently, when image data for analysis or image data for analysis on which image quality improvement processing is performed is obtained, a vehicle number including at least one letter and/or at least one number is extracted from the image for the license plate included in the image data. become (416). The extraction of the license plate number may be performed based on a predetermined character recognition algorithm based on at least one learning model.

When the vehicle number is extracted, an operation according to the extraction of the vehicle number may be performed ( 418 ). For example, a visual or auditory output operation of the license plate number, determination of whether entry or exit is possible according to the license plate number, control of the operation of a blocking bar, and/or a fee settlement processing based on entry and exit times, etc. At least one operation may be performed. This operation may be performed by the license plate recognition device and/or another device (eg, a circuit breaker) physically or electrically connected to the license plate recognition device. In addition, the extracted vehicle number according to the embodiment may be transmitted to at least one terminal device communicatively connected to the vehicle license plate recognition device.

The vehicle license plate recognition method according to the above-described embodiment may be implemented in the form of a program that can be driven by a computer device. Here, the program may include program instructions, data files, and data structures alone or in combination. The program may be designed and manufactured using machine code or high-level language code. The program may be specially designed to implement the above-described method, or may be implemented using various functions or definitions that are known and available to those skilled in the art of computer software. Also, here, the computer device may be implemented including a processor or memory that enables the function of the program to be realized, and may further include a communication device if necessary.

A program for implementing the above-described vehicle license plate recognition method may be recorded in a computer-readable recording medium. The computer-readable recording medium is, for example, a semiconductor storage device such as a solid state drive (SSD), ROM, RAM or flash memory, a magnetic disk storage medium such as a hard disk or floppy disk, a compact disk or DVD, etc. It may include at least one type of physical device capable of storing a specific program executed in response to a call of a computer, such as an optical recording medium, a magneto-optical recording medium such as a floppy disk, and a magnetic tape.

Although various embodiments of the vehicle license plate recognition system, the vehicle license plate recognition method, the vehicle license plate recognition device and the vehicle license plate recognition management device for the vehicle license plate recognition device have been described above, the vehicle license plate recognition system, the vehicle license plate recognition method, the vehicle license plate recognition device And the license plate recognition management device for the license plate recognition device is not limited only to the above-described embodiment. Various systems, devices or methods that can be implemented by modifying and transforming based on the above-described embodiments by those of ordinary skill in the art are also the above-described vehicle license plate recognition system, vehicle license plate recognition method, vehicle license plate recognition device and vehicle license plate recognition It may be an example of a license plate recognition management device for the device. For example, the described techniques are performed in an order different from the described method, and/or the described components of the system, structure, apparatus, circuit, etc., are combined or combined in a different form than the described method, or other components or Even if it is replaced or substituted by an equivalent, it may be an embodiment of the vehicle license plate recognition system, the vehicle license plate recognition method, the vehicle license plate recognition device and the vehicle license plate recognition management device for the vehicle license plate recognition device.

1: License plate recognition system 3: Terminal device
100: vehicle license plate recognition device 111: photographing unit
113: detection unit 115: communication unit
117: input/output unit 119: storage unit
120: processor 130: image processing unit
131: license plate detection unit 133: image quality improvement unit
135: character recognition unit 137: result output unit
200: vehicle license plate recognition management device 211: communication unit
213: input/output unit 215: storage unit
220: processor 221: learning model processing unit
223: training model distribution management unit 225: performance inspection unit

Claims (17)

Vehicle license plate recognition management device for building an application pipeline, and determining a learning model for analysis suitable for the application pipeline; and
It is communicatively connected with the license plate recognition management device to receive the learning model for analysis, perform photographing to obtain at least one analysis target image data, and add the at least one analysis target image data to the analysis learning model A vehicle license plate recognition system comprising a; by applying a vehicle license plate recognition device for detecting and obtaining a vehicle number from at least one target vehicle number plate of the at least one analysis target image data. According to claim 1,
The license plate recognition management apparatus, by determining a learning model suitable for the at least one application pipeline from at least one candidate learning model, the license plate recognition system for determining the learning model for analysis. 3. The method of claim 2,
The license plate recognition management apparatus acquires at least one image data including at least one license plate, and uses the acquired at least one image data to train the candidate learning model. According to claim 1,
The license plate recognition management device further examines the performance of the application pipeline, and the license plate recognition system for determining the learning model for analysis according to the inspection result. According to claim 1,
The vehicle license plate recognition apparatus detects the at least one target vehicle license plate from the at least one analysis target image data, and image quality improvement processing for the at least one analysis target image data from which the at least one target vehicle license plate is detected A vehicle license plate recognition system for performing and acquiring a vehicle number from the at least one target vehicle number plate detected from at least one analysis target image data that has been processed for image quality improvement by using the learning model for analysis. According to claim 1,
The vehicle license plate recognition system further comprising a; a terminal device communicatively connected to the vehicle license plate recognition device and receiving the vehicle number obtained from the at least one target vehicle license plate from the vehicle license plate recognition device. According to claim 1,
The license plate recognition device is a vehicle license plate recognition system belonging to any one recognition device group among a plurality of recognition device groups. determining, by the license plate recognition management device, a learning model for analysis for the application pipeline;
receiving the learning model for analysis by at least one license plate recognition device communicatively connected to the license plate recognition management device;
acquiring, by the at least one vehicle license plate recognition device, at least one analysis target image data by photographing;
Step by which the at least one license plate recognition device detects and obtains a vehicle number from at least one target license plate of the at least one analysis target image data by applying the learning model for analysis to the at least one analysis target image data ; Vehicle license plate recognition method including. 9. The method of claim 8,
The step of the license plate recognition management device determining a learning model for analysis appropriate for the application pipeline,
determining a learning model for analysis by determining a learning model suitable for the at least one application pipeline from at least one candidate learning model; 10. The method of claim 9,
acquiring, by the license plate recognition management device, at least one image data including at least one license plate; and
Using the obtained at least one image data to train the candidate learning model; Vehicle license plate recognition method further comprising a. 9. The method of claim 8,
The vehicle license plate recognition management device inspecting the performance of the application pipeline; Vehicle license plate recognition method further comprising. 9. The method of claim 8,
Step by which the at least one license plate recognition device detects and obtains a vehicle number from at least one target license plate of the at least one analysis target image data by applying the learning model for analysis to the at least one analysis target image data Is,
detecting the at least one target vehicle license plate from the at least one analysis target image data;
performing image quality improvement processing on the at least one analysis target image data in which the at least one target vehicle license plate is detected; and
Obtaining a vehicle number from the at least one target vehicle license plate detected from the at least one analysis target image data that has been image quality improved by using the learning model for analysis; Vehicle license plate recognition method comprising a. 9. The method of claim 8,
Receiving the vehicle number obtained from the at least one target vehicle license plate from the vehicle license plate recognition device by a terminal device communicatively connected to the vehicle license plate recognition device; Vehicle license plate recognition method further comprising. a processor for constructing an application pipeline and determining a learning model for analysis suitable for the application pipeline by using at least one candidate learning model; and
Vehicle license plate recognition management device comprising a; communication unit for transmitting the learning model for analysis to the corresponding at least one license plate recognition device. 15. The method of claim 14,
The processor, license plate recognition management device for determining the learning model for analysis by determining a candidate learning model suitable for the at least one application pipeline from among at least one candidate learning model. a communication unit for receiving at least one learning model for analysis;
a photographing unit to obtain at least one analysis target image data by photographing; and
A processor for detecting and obtaining at least one of letters and numbers from at least one target vehicle license plate of the at least one analysis target image data by applying the learning model for analysis to the at least one analysis target image data; license plate recognition device. 17. The method of claim 16,
The processor detects the at least one target vehicle license plate from the at least one analysis target image data, and performs image quality improvement processing on the at least one analysis target image data from which the at least one target vehicle license plate is detected, , A vehicle license plate recognition device for obtaining a vehicle number from the at least one target vehicle license plate detected from the at least one analysis target image data that has been processed for image quality improvement by using the learning model for analysis.

Images