KR102161949B1 - Integrated parking management method and system using vehicle number plate recognition technology using neural network - Google Patents

Abstract

Disclosed is an integrated parking management method using vehicle number plate recognition technology using neural network. The integrated parking management method using vehicle number plate recognition technology using neural network, which is carried out by a vehicle number recognizer including a processor, a memory, a camera, and a communication module to recognize characters on a vehicle′s license plate, includes: receiving, by the processor, an image including a classified text area; performing, by the processor, a first convolution operation on the received image to output a first output feature map; outputting, by the processor, a second output feature map by applying sampling to the outputted first output feature map; performing, by the processor, a second convolution operation on the second output feature map to output a third output feature map; applying, by the processor, sampling to the outputted third output feature map to output a fourth output feature map; and recognizing, by the processor, a character in a classified character area by using a fully connected layer configured to fully connect preset feature maps defined as a numerical expression of the outputted fourth output feature map and the characteristics of the character, and having the number of pixels of characters included in each area after the character area is divided into certain areas, having the sum of white pixels or black pixels located on the horizontal lines drawn in several along the height of the character, or defined as the sum of white pixels or black pixels located on vertical lines drawn several times according to the width.

Description

Integrated parking management method and system using vehicle number plate recognition technology using neural network}

The present invention relates to an integrated parking management method and system to which vehicle license plate recognition technology using a neural network is applied, and in detail, an integrated parking management method and system to which vehicle license plate recognition technology using a neural network that can increase the performance of vehicle license plate character recognition is applied. It is about.

Vehicle license plate recognition is used for various purposes such as speed violation, illegal parking, and parking control.

The method of recognizing the license plate of a moving vehicle is as follows. First, an image of a moving vehicle is acquired. The license plate area is extracted from the second acquired image. Finally, the license plate of the vehicle is recognized through character recognition in the extracted license plate area.

However, when an image of a moving vehicle is acquired, the license plate attached to the vehicle may be tilted or damaged, or an image distortion of the vehicle may occur due to the influence of lighting such as sunlight. In addition, there are cases in which letters and numbers among overseas license plates are made of plastic. When the letters and numbers are made of a plastic material, a height difference between the letters and numbers occurs from the license plate, and a problem of recognizing numbers and characters according to the shooting angle of the camera may occur due to the difference in height.

There is a need for a new license plate recognition method to improve the above problems.

Korean Patent Publication No. 10-1999985 (2019.07.09.)

The technical problem to be achieved by the present invention is to provide an integrated parking management method and system using a vehicle license plate recognition technology that applies feature maps extracted from a character region recognized in a vehicle license plate region to a Convolutional Neural Network (CNN) algorithm. Is to do.

The integrated parking management method to which the vehicle license plate recognition technology is applied using a neural network performed by a vehicle number recognizer including a processor, a memory, a camera, and a communication module in order to recognize characters on the license plate of a vehicle according to an embodiment of the present invention is described above. When the entrance of the vehicle is recognized by the roof coil, the processor receives an image including a license plate area of the vehicle photographed by the camera from the camera, the processor includes a character area classified in the license plate area of the vehicle Receiving an image to perform, the processor performing a first convolution operation on the received image to output a first output feature map, the processor applying sampling to the output first output feature map Outputting a second output feature map, wherein the processor performs a second convolution operation on the second output feature map to output a third output feature map, wherein the processor performs a second output feature map Applying sampling to the map and outputting a fourth output feature map, the processor is defined as a numerical expression of the outputted fourth output feature map and the character characteristics, and the character region as predetermined regions The number of pixels of the character included in each area after being divided, white pixels located on horizontal lines drawn in multiple lines along the height of the character, or the sum of black pixels, or white pixels located in vertical lines drawn in multiple lines depending on the width Recognizing a character in the classified character area using a fully connected layer that completely connects a preset feature map that can be defined as a sum of fields or black pixels, and the processor recognizes the character And transmitting the text to the parking control server. The parking control server calculates a parking fee according to the recognized character so that the driver of the vehicle can pay the parking fee of the vehicle, and transmits the calculated parking fee of the vehicle to an exit unmanned fee calculator.

The processor is a second pulley for completely connecting a fifth output feature map output by performing convolution operations and sampling operations among CNN algorithms for data that may correspond to the character region and a second preset feature map. Classifying characters that may be confusing with each other by comparing the difference between the derived result values and a threshold value with respect to the result values derived by applying the connected layer, and the processor classifying the second characters according to the classified characters. Resetting a preset feature map to define a third preset feature map, and the processor includes a sixth output feature map output by applying the CNN algorithm to any one of the confusing characters, and the And recognizing a character by applying a third pulley connected layer that completely connects the third preset feature map.

The number of the fifth output feature map input to the second pulley connected layer and the number of the sixth output feature map input to the third pulley connected layer are different from each other, and input to the second pulley connected layer The number of second preset feature maps to be set and the number of third preset feature maps input to the third pulley connected layer are different from each other.

The fully connected layer includes an input layer, a hidden layer, and an output layer.

The hidden layer includes a plurality of hidden neurons.

)은 다음의 수학식으로 표현된다. When the number of features of the preset feature map is m (m is a natural number of 2 or more), and the number of features of the output fourth output feature map is n (n is a natural number of 2 or more), the plurality of hidden neurons Among them, the input value of the hidden neuron (j is a natural number) (

) Is expressed by the following equation.

[Equation]

)은 다음의 수학식으로 표현된다. The output value of the j-th (j is a natural number) hidden neuron among the plurality of hidden neurons (

) Is expressed by the following equation.

[Equation]

는 상기 a번째 특징에 대한 j번째 가중치를, 상기

는 상기 미리 설정된 피처 맵의 특징들 중 a번째 특징을, 상기

는 상기 c번째 결과 값에 대한 j번째 가중치를, 상기

는 상기 출력된 제4출력 피처 맵의 특징들 중 c번째 결과 값을, 상기

는 j번째 오프셋(offset)을, 상기 함수 f()는 시그모이드(sigmoid) 함수를 나타낸다. remind

Is the j-th weight for the a-th feature,

Is the a-th feature among the features of the preset feature map,

Is the j-th weight for the c-th result value,

Is the c-th result value among the features of the outputted fourth output feature map,

Denotes the j-th offset, and the function f() denotes a sigmoid function.

The output layer includes a plurality of output neurons.

)은 다음의 수학식으로 표현된다. The input value of the k-th (k is a natural number) output neuron among the plurality of output neurons (

) Is expressed by the following equation.

[Equation]

)은 다음의 수학식으로 표현된다. The output value of the k-th (k is a natural number) output neuron among the plurality of output neurons (

) Is expressed by the following equation.

[Equation]

는 상기 j번째 은닉 뉴런의 출력 값에 대한 k번째 가중치를, 상기

는 상기 j번째 은닉 뉴런의 출력 값을, 상기

는 k번째 오프셋을, 상기 함수 f()는 시그모이드(sigmoid) 함수를 나타낸다.remind

Is the k-th weight for the output value of the j-th hidden neuron,

Is the output value of the j-th hidden neuron,

Denotes the k-th offset, and the function f() denotes a sigmoid function.

는 다음의 수학식으로 표현된다. The error of the j-th weight with respect to the output value of the i-th input neuron

Is expressed by the following equation.

[Equation]

는 상수로, 기존의 가중치를 줄이기 위해 가중치 감소 비율을, 상기

는 가중치를 어느 정도로 업데이트할 것인지 결정하는 학습율을 나타낸다. remind

Is a constant, the weight reduction ratio to reduce the existing weight,

Denotes a learning rate that determines how much to update the weights.

는 다음의 수학식으로 표현된다. The error of the k-th weight with respect to the output value of the j-th hidden neuron

Is expressed by the following equation.

[Equation]

는 상수로, 기존의 가중치를 줄이기 위해 가중치 감소 비율을, 상기

는 가중치를 어느 정도로 업데이트할 것인지 결정하는 학습율을, 상기

는 복수의 출력 뉴런들 중 k번째 뉴런의 에러를, 상기 f'(z_k)는

를 나타낸다.remind

Is a constant, the weight reduction ratio to reduce the existing weight,

Is the learning rate that determines how much to update the weights,

Is the error of the k-th neuron among the plurality of output neurons, and f'(z _k ) is

Represents.

An exit unmanned settlement system according to an embodiment of the present invention includes a roof coil for recognizing the entry of the vehicle, and a vehicle number recognizer for photographing the vehicle license plate and recognizing the vehicle license plate from the captured vehicle license plate image.

The vehicle number recognizer includes a camera for photographing the license plate of the vehicle, a communication module for transmitting the recognized vehicle number to a parking control server, and a memory for storing instructions related to vehicle license plate recognition technology using a neural network, and the commands. Includes a running processor.

The commands receive an image including a classified character area processed in a license plate photographed by the camera, perform a first convolution operation on the received image, and output a first output feature map, and the output Sampling is applied to the first output feature map to output a second output feature map, a second convolution operation is performed on the second output feature map to output a third output feature map, and the output 3) Sampling is applied to the output feature map to output a fourth output feature map, and is defined as a numerical expression of the output fourth output feature map and the character characteristics, and the character area is divided into certain areas. The number of pixels of a character included in each area, white pixels located on multiple horizontal lines drawn along the height of the character, or the sum of black pixels, or white pixels located on vertical lines drawn multiple along the width , Or by using a fully connected layer that fully connects a preset feature map that can be defined as a sum of black pixels, recognizes a character in the classified character area, and calculates the parking fee of the vehicle using the recognized character. It is implemented to transmit to the parking control server for. The parking control server transmits the calculated parking fee of the vehicle to an exit unmanned fee calculator for calculating the parking fee.

The pre-unmanned settlement system according to an embodiment of the present invention includes a roof coil that recognizes the entry of the vehicle, a vehicle number recognizer that photographs the vehicle license plate and recognizes the vehicle license plate from the captured vehicle license plate image, and unloads the vehicle. And an unmanned fare adjuster for the driver of the vehicle to pay the parking fee beforehand.

The vehicle number recognizer includes a camera that photographs the license plate of the vehicle, a communication module that transmits the recognized vehicle number to a parking control server, a memory that stores instructions for vehicle license plate recognition technology using a neural network, and executes the commands. It includes a processor that does.

The commands receive an image including a classified character area processed in a license plate photographed by the camera, perform a first convolution operation on the received image, and output a first output feature map, and the output Sampling is applied to the first output feature map to output a second output feature map, a second convolution operation is performed on the second output feature map to output a third output feature map, and the output 3) Sampling is applied to the output feature map to output a fourth output feature map, and is defined as a numerical expression of the output fourth output feature map and the character characteristics, and the character area is divided into certain areas. The number of pixels of a character included in each area, white pixels located on multiple horizontal lines drawn along the height of the character, or the sum of black pixels, or white pixels located on vertical lines drawn multiple along the width , Or by using a fully connected layer that fully connects a preset feature map that can be defined as a sum of black pixels, recognizes a character in the classified character area, and calculates the parking fee of the vehicle using the recognized character. It is implemented to transmit to the parking control server for. The parking control server transmits the calculated parking fee of the vehicle to an unmanned fee calculator for calculating the parking fee.

The manned settlement system according to an embodiment of the present invention includes a roof coil that recognizes the entry of the vehicle, a vehicle number recognizer that photographs the vehicle license plate and recognizes the vehicle license plate from the photographed vehicle license plate image, and the vehicle from the parking control server. Includes a parking management booth to receive the transfer of parking fees.

The vehicle number recognizer includes a camera that photographs the license plate of the vehicle, a communication module that transmits the recognized vehicle number to the parking control server, a memory for storing instructions related to vehicle license plate recognition technology using a neural network, and the commands. Includes a running processor.

The commands receive an image including a classified character area processed in a license plate photographed by the camera, perform a first convolution operation on the received image, and output a first output feature map, and the output Sampling is applied to the first output feature map to output a second output feature map, a second convolution operation is performed on the second output feature map to output a third output feature map, and the output 3) Sampling is applied to the output feature map to output a fourth output feature map, and is defined as a numerical expression of the output fourth output feature map and the character characteristics, and the character area is divided into certain areas. The number of pixels of a character included in each area, white pixels located on multiple horizontal lines drawn along the height of the character, or the sum of black pixels, or white pixels located on vertical lines drawn multiple along the width , Or by using a fully connected layer that fully connects a preset feature map that can be defined as a sum of black pixels, recognizes a character in the classified character area, and calculates the parking fee of the vehicle using the recognized character. It is implemented to transmit to the parking control server for. The parking control server transmits the calculated parking fee of the vehicle to a PC of a parking management booth capable of calculating the parking fee.

An access control system according to an embodiment of the present invention includes a loop coil for recognizing the entry of a vehicle, and a vehicle number recognizer for photographing the vehicle license plate and recognizing the vehicle license plate from the captured vehicle license plate image.

The vehicle number recognizer includes a camera that photographs the license plate of the vehicle, a communication module that transmits the recognized vehicle number to a parking control server, a memory that stores instructions for vehicle license plate recognition technology using a neural network, and executes the commands. It includes a processor that does.

The commands receive an image including a classified character area processed in a license plate photographed by the camera, perform a first convolution operation on the received image, and output a first output feature map, and the output Sampling is applied to the first output feature map to output a second output feature map, a second convolution operation is performed on the second output feature map to output a third output feature map, and the output 3) Sampling is applied to the output feature map to output a fourth output feature map, and is defined as a numerical expression of the output fourth output feature map and the character characteristics, and the character area is divided into certain areas. The number of pixels of a character included in each area, white pixels located on multiple horizontal lines drawn along the height of the character, or the sum of black pixels, or white pixels located on vertical lines drawn multiple along the width Or, it is implemented to recognize a character in the classified character area using a fully connected layer that completely connects a preset feature map that may be defined as a sum of black pixels.

The vehicle number recognizer according to an embodiment of the present invention includes a camera for photographing a license plate of a vehicle, a memory for storing instructions related to a vehicle license plate recognition technology using a neural network, and a vehicle number recognized using the vehicle license plate recognition technology using the neural network. And a communication module for transmitting to the parking control server, and a processor for executing the instructions.

The commands receive an image including a classified character area processed in a license plate photographed by the camera, perform a first convolution operation on the received image, and output a first output feature map, and the output Sampling is applied to the first output feature map to output a second output feature map, a second convolution operation is performed on the second output feature map to output a third output feature map, and the output 3) Sampling is applied to the output feature map to output a fourth output feature map, and is defined as a numerical expression of the output fourth output feature map and the character characteristics, and the character area is divided into certain areas. The number of pixels of a character included in each area, white pixels located on multiple horizontal lines drawn along the height of the character, or the sum of black pixels, or white pixels located on vertical lines drawn multiple along the width Or, it is implemented to recognize a character in the classified character area using a fully connected layer that completely connects a preset feature map that may be defined as a sum of black pixels.

An integrated parking management system according to an embodiment of the present invention includes an unattended exit settlement system including a VPN terminal, and a server system including a parking control server for communicating with the VPN terminal through a network.

The unmanned exit settlement system includes a roof coil for recognizing the entry of the vehicle, and a vehicle number recognizer for photographing the vehicle license plate and recognizing the vehicle license plate from the captured vehicle license plate image.

The vehicle number recognizer includes a camera that photographs the license plate of the vehicle, a communication module that transmits the recognized vehicle number to a parking control server, a memory that stores instructions for vehicle license plate recognition technology using a neural network, and executes the commands. It includes a processor that does.

The commands receive an image including a classified character area processed in a license plate photographed by the camera, perform a first convolution operation on the received image, and output a first output feature map, and the output Sampling is applied to the first output feature map to output a second output feature map, a second convolution operation is performed on the second output feature map to output a third output feature map, and the output 3) Sampling is applied to the output feature map to output a fourth output feature map, and is defined as a numerical expression of the output fourth output feature map and the character characteristics, and the character area is divided into certain areas. The number of pixels of a character included in each area, white pixels located on multiple horizontal lines drawn along the height of the character, or the sum of black pixels, or white pixels located on vertical lines drawn multiple along the width Or, it is implemented to recognize a character in the classified character area using a fully connected layer that completely connects a preset feature map that may be defined as a sum of black pixels.

The parking control server receives the recognized vehicle number from the vehicle number recognizer through a network.

In an embodiment of the present invention, the integrated parking management method and system using a neural network recognizes the vehicle license plate more accurately by applying the feature maps extracted from the character region recognized in the license plate region of the vehicle together with the CNN (Convolutional Neural Network) algorithm. It has the effect of increasing the efficiency of parking management by being able to recognize license plates.

A detailed description of each drawing is provided in order to more fully understand the drawings cited in the detailed description of the present invention.
1 is a block diagram of an unattended exit settlement system according to an embodiment of the present invention.
2 is a block diagram of a pre-unmanned settlement system according to another embodiment of the present invention.
3 is a block diagram of a manned settlement system according to another embodiment of the present invention.
4 is a block diagram of an access control system according to another embodiment of the present invention.
5 shows a block diagram of an integrated parking management system in which the systems shown in FIGS. 1 to 4 are integrated.
6 is a block diagram showing an interior of the vehicle number recognizer shown in any one of FIGS. 1 to 5.
7 is a flowchart illustrating an operation of a vehicle entering a parking lot in the system shown in FIG. 1.
8 is a flowchart illustrating an operation of entering a parking lot of a vehicle in the system shown in FIG. 1.
9 is a flowchart illustrating a vehicle license plate recognition operation of FIG. 7 or 8.
10 shows images processed by a processor of the vehicle number recognizer to describe the operation of the vehicle number recognizer shown in FIG. 6.
11 shows different images processed by the processor of the vehicle number recognizer to illustrate the operation of the vehicle number recognizer shown in FIG. 6.
12 shows a block diagram of a CNN algorithm processed by the processor shown in FIG. 6.
FIG. 13 is a block diagram illustrating a relationship between an input feature map and an output feature map shown in FIG. 12.
14 shows an embodiment of a block diagram of a neural network algorithm processed by the processor shown in FIG. 6.
15 shows another embodiment of a block diagram of a neural network algorithm processed by the processor shown in FIG. 6.
16 shows another embodiment of a block diagram of a neural network algorithm processed by the processor shown in FIG. 6.
17 shows a graph of overfit distortion according to learning of the neural network algorithm shown in FIG. 6.
18 shows a graph for solving the problem of overfit distortion shown in FIG. 17.
FIG. 19 shows another embodiment of a block diagram of a neural network algorithm processed by the processor shown in FIG. 6.
20 are images showing differences between numbers that may be confusing.

Specific structural or functional descriptions of the embodiments according to the concept of the present invention disclosed in the present specification are exemplified only for the purpose of describing the embodiments according to the concept of the present invention, and the embodiments according to the concept of the present invention are It may be implemented in various forms and is not limited to the embodiments described herein.

Since the embodiments according to the concept of the present invention can apply various changes and have various forms, the embodiments will be illustrated in the drawings and described in detail in the present specification. However, this is not intended to limit the embodiments according to the concept of the present invention to specific disclosed forms, and includes all changes, equivalents, or substitutes included in the spirit and scope of the present invention.

Terms such as first or second may be used to describe various elements, but the elements should not be limited by the terms. The terms are only for the purpose of distinguishing one component from other components, for example, without departing from the scope of the rights according to the concept of the present invention, the first component may be referred to as the second component, and similarly The second component may also be referred to as a first component.

When a component is referred to as being "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. Should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle. Other expressions describing the relationship between components, such as "between" and "just between" or "adjacent to" and "directly adjacent to" should be interpreted as well.

The terms used in this specification are used only to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present specification, terms such as "comprise" or "have" are intended to designate that the described feature, number, step, action, component, part, or combination thereof is present, but one or more other features or numbers It is to be understood that the possibility of addition or presence of, steps, actions, components, parts, or combinations thereof is not preliminarily excluded.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms as defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in this specification. Does not.

Hereinafter, the present invention will be described in detail by describing a preferred embodiment of the present invention with reference to the accompanying drawings.

1 is a block diagram of an unattended exit settlement system according to an embodiment of the present invention.

Referring to FIG. 1, the unmanned exit settlement system 100 is a parking management system in which a vehicle enters a parking lot for parking or is not operated by a person exiting the parking lot. The unmanned exit settlement system 100 is a system to which a number recognition method is applied in which feature maps extracted from a character area recognized in a license plate area of a vehicle are applied together with a Convolutional Neural Network (CNN) algorithm. The unmanned exit settlement system 100 can increase the license plate recognition rate of the vehicle by applying the feature maps extracted from the character region recognized in the vehicle license plate region to the Convolutional Neural Network (CNN) algorithm, thereby increasing the efficiency of parking management. Is a system.

The unmanned exit settlement system 100 includes a first circuit breaker 10 and a first vehicle number recognizer 20. According to an embodiment, the unmanned exit settlement system 100 includes a second circuit breaker 11, a second vehicle number recognizer 30, a light vehicle identification sensor 40, a four-sided photographing camera 50, and a third vehicle number recognizer 60. ), an exit unmanned fare setter 70, an exit warning light 80, or a loop coil 90 may be further included. Each of the components 10, 11, 20, 30, 40, 50, 60, 70, 80, 90 included in the unmanned exit settlement system 100 are connected to a network by wireless or wired.

The first circuit breaker 10 is used to prevent vehicles from entering the parking lot without permission. The second circuit breaker 11 is used to prevent the vehicle from leaving the parking lot without permission.

The first vehicle number recognizer 20, the second vehicle number recognizer 30, or the third vehicle number recognizer 60 photographs the license plate of the vehicle, recognizes the license plate of the vehicle from the captured vehicle license plate image, and recognizes the recognized vehicle. The number of is transmitted to the parking control server (not shown) through the network. The first vehicle number recognizer 20 photographs the vehicle license plate in front of the entering vehicle. The second vehicle number recognizer 30 photographs the license plate of the vehicle at the rear of the entering vehicle. The third vehicle number recognizer 60 photographs the vehicle license plate in front of the advancing vehicle.

A detailed configuration and operation of the first vehicle number recognizer 20, the second vehicle number recognizer 30, or the third vehicle number recognizer 60 will be described in detail in FIG. 6.

The compact vehicle determination sensor 40 determines whether the vehicle is a compact vehicle by emitting light. The small vehicle determination sensor 40 may be implemented as a laser scanner.

The four-sided photographing camera 50 may photograph four sides of the vehicle. The four-sided photographing camera 50 may further include a processor (not shown) for processing images of four sides of the captured vehicle, and a memory (not shown) for storing the processed images. According to an embodiment, the processor and the memory may be implemented as a network video network (NVR).

The exit unmanned fee calculator 70 receives the parking fee of the vehicle from the parking control server through the network.

When the vehicle leaves the vehicle, the exit warning light 80 indicates that the vehicle is leaving the vehicle.

The roof coil 90 recognizes the entry of the vehicle.

2 is a block diagram of a pre-unmanned settlement system according to another embodiment of the present invention.

Referring to FIG. 2, the pre-unmanned settlement system 110 is similar to the exit unmanned settlement system 100 shown in FIG. 1. However, in the unmanned exit settlement system 100 shown in FIG. 1, when the vehicle leaves the vehicle, the driver of the vehicle calculates the parking fee by card or cash in the exit unmanned fare calculator 70, but the advance unmanned settlement system 110 ) Differs in that the parking fee is calculated by card or cash in advance by a separate unmanned fare calculator 79 installed outside before leaving the vehicle. The pre-unmanned settlement system 110 is a system to which a number recognition method is applied in which feature maps extracted from a character area recognized in a vehicle license plate area are applied together with a Convolutional Neural Network (CNN) algorithm. The pre-unmanned settlement system 110 can increase the license plate recognition rate of the vehicle by applying the feature maps extracted from the character region recognized in the vehicle license plate region to the Convolutional Neural Network (CNN) algorithm, thereby increasing the efficiency of parking management. Is a system.

The pre-unmanned settlement system 110 includes a plurality of breakers 12 and 13, a plurality of vehicle number recognizers 21, 31, and 61, a light vehicle identification sensor 41, a four-sided photographing camera 51, and an unmanned exit It may include a fare adjustment machine 71, a car parking warning light 81, or a loop coil 91. Depending on the embodiment, the pre-unmanned settlement system 110 may not include the exit unmanned fare calculator 71, and instead may include a separate unmanned fare calculator 79 installed outside. Each of the components 12, 13, 21, 31, 41, 51, 61, 71, 81, 91 included in the pre-unmanned settlement system 110 is connected to a network by wireless or wired. In addition, each of the components (12, 13, 21, 31, 41, 51, 61, 71, 81, 91) included in the pre-unmanned settlement system 110 is the exit unmanned settlement system 100 shown in FIG. Since the operation of each of the components 10, 11, 20, 30, 40, 50, 60, 70, 80, and 90 included in is similar, detailed descriptions thereof will be omitted.

3 is a block diagram of a manned settlement system according to another embodiment of the present invention.

Referring to FIG. 3, the manned settlement system 120 is similar to the unmanned exit settlement system 100 shown in FIG. 1. However, in the unmanned exit settlement system 100 shown in FIG. 1, the parking fee is calculated by the unmanned exit fare calculator 70, but the manned settlement system 120 differs in that the parking settlement agent 75 directly receives the parking fee. There is. The manned settlement system 120 is a system to which a number recognition method is applied in which feature maps extracted from a text area recognized in a license plate area of a vehicle are applied together with a Convolutional Neural Network (CNN) algorithm. The manned settlement system 120 can increase the license plate recognition rate of the vehicle by applying the feature maps extracted from the character region recognized in the vehicle license plate region to the CNN (Convolutional Neural Network) algorithm, thereby increasing the efficiency of parking management. System.

The manned unmanned settlement system 120 includes a plurality of breakers 14, 15, a plurality of vehicle number recognizers 22, 32, and 62, a small vehicle identification sensor 42, a four-sided photographing camera 52, and a car exit attention. Lights 82, or loop coils 92. In addition, the manned unmanned settlement system 120 includes a parking management booth 72 in which the parking settlement agent 75 can enter instead of the exit unmanned fare settlement machine 70 shown in FIG. 1. The parking management booth 72 includes a PC 74 and a monitor 73 that can directly calculate the parking fee. According to an embodiment, the PC may not directly calculate the parking fee, but may receive a parking fee from a parking control server (not shown) through a network. The parking settlement agent 75 may receive a parking fee from the driver of the vehicle according to the parking fee displayed on the monitor 73. Each of the components 14, 15, 22, 32, 42, 52, 62, 72, 82, 92 included in the manned settlement system 120 is connected to a network by wireless or wired. In addition, each of the components (14, 15, 22, 32, 42, 52, 62, 72, 82, 92) included in the manned settlement system 120 is in the exit unmanned settlement system 100 shown in FIG. Since the operation of each of the included components 10, 11, 20, 30, 40, 50, 60, 70, 80, 90 is similar, a detailed description thereof will be omitted.

4 is a block diagram of an access control system according to another embodiment of the present invention.

Referring to FIG. 4, the access control system 130 is similar to the unattended exit settlement system 100 shown in FIG. 1. However, the unmanned exit settlement system 100 shown in FIG. 1 incurs a parking fee when leaving the vehicle, but the access control system 130 allows only vehicles with the proper authority to enter the parking lot instead of the parking fee. There is a difference in The access control system 130 is a system to which a number recognition method using feature maps extracted from a character area recognized in a license plate area of a vehicle together with a Convolutional Neural Network (CNN) algorithm is applied. The access control system 130 can increase the license plate recognition rate of the vehicle by applying the feature maps extracted from the character area recognized in the vehicle license plate area to the Convolutional Neural Network (CNN) algorithm, thereby increasing the efficiency of parking management. System.

The access control system 130 may include a plurality of breakers 16 and 17 and a plurality of vehicle number recognizers 23, 33, and 63. Depending on the embodiment, the access control system 130 may further include a small vehicle determination sensor 43, a four-sided photographing camera 53, a car exit warning light 83, or a roof coil 93. However, according to an embodiment, the access control system 130 may include a terminal 76 capable of determining whether a vehicle is a vehicle with a legitimate authority, instead of the unmanned exit fare calculator 70 shown in FIG. 1. When approaching the terminal 76 with a card or the like indicating that the driver of the vehicle has the proper authority, the terminal 76 can control to open the circuit breaker 16 or 17 if the card is legal.

Each of the components (16, 17, 23, 33, 43, 53, 63, 73, 83, 93) included in the access control system 130 are connected to a network by wireless or wired. In addition, each of the components (16, 17, 23, 33, 43, 53, 63, 83, 93) included in the access control system 130 are included in the unattended exit settlement system 100 shown in FIG. Since the operation of each of the components 10, 11, 20, 30, 40, 50, 60, 70, 80, 90 is similar, a detailed description thereof will be omitted.

5 shows a block diagram of an integrated parking management system in which the systems shown in FIGS. 1 to 4 are integrated.

1 to 5, the integrated parking management system 200 includes a plurality of different systems (e.g., an exit unmanned settlement system 100, a pre-unmanned settlement system 110, a manned settlement system 120), and It includes a server system 140 capable of calculating a parking fee through the number of a vehicle transmitted from the access control system 130 through the network or determining whether there is a legitimate authority. The integrated parking management system 200 includes an exit unmanned settlement system 100 shown in FIGS. 1 to 3, a pre-unmanned settlement system 110, a manned settlement system 120, an access control system 130, and a server system ( 140) may be included. According to an embodiment, the integrated parking management system 200 includes a server system 140, an exit unmanned settlement system 100 shown in FIGS. 1 to 3, a pre-unmanned settlement system 110, a manned settlement system 120, And it may include any one of the access control system 130. In FIG. 5, the integrated parking management system 200 includes an exit unmanned settlement system 100, a pre-unmanned settlement system 110, a manned settlement system 120, an access control system 130, and a server system 140. Although shown, the integrated parking management system 200 may include more systems according to an embodiment. For example, the integrated parking management system 200 may include a plurality of unmanned exit settlement systems located in different regions, a plurality of pre-unmanned settlement systems located in different regions, or may include all of them.

The unmanned exit system 100 included in the integrated parking management system 200, the pre-unmanned settlement system 110, the manned settlement system 120, the access control system 130, and the server system 140 are network 201 ) To communicate with each other. The network 201 may be a VPN (Virtual Private Network). The unmanned exit settlement system 100, the pre-unmanned settlement system 110, the manned settlement system 120, the access control system 130, and the server system 140, respectively, are VPN terminals 201, 203, 205 for communication with each other. , 207, and 209) can be used. The VPN terminal 201, 203, 205, 207, or 209 is used to connect each system to each other through a network.

The server system 140 includes a 5th VPN terminal 209, a switch 141, a parking control server 143, a database 145, a web discount server 147, a plurality of clients 151-1 to 151-N, N is a natural number).

The fifth VPN terminal 209 is a vehicle from the VPN terminal 201, 203, 205, or 207 of the exit unmanned settlement system 100, the advance unmanned settlement system 110, the manned settlement system 120, or the access control system 130. It receives information on the entrance location information, vehicle entry information, and vehicle number. The 5th VPN terminal 209 transmits the vehicle entrance location information, the vehicle entry information, and the vehicle number information to the parking control server 143 through the switch 141.

The database 145 may store entrance location information of the vehicle, entry information of the vehicle, and information on the vehicle number.

The web discount server 147 determines a discount of a parking fee according to vehicle information. For example, the web discount server 147 applies a discounted parking fee to the vehicle if it is a vehicle registered regularly.

According to an embodiment, the server system 140 may further include a non-face-to-face qualification verification server 149. The non-face-to-face qualification verification server 149 receives information on the vehicle number and determines whether the vehicle is a vehicle subject to legal discounts such as a light vehicle, an eco-friendly vehicle, a disabled person, or a person of national merit, etc. Information on the vehicle number and whether the vehicle is a legal discount target vehicle are requested from (not shown). According to the request, the non-face-to-face qualification verification server 149 receives information indicating whether the vehicle is a vehicle subject to legal discount from the administrative information joint use center system, and when the vehicle corresponds to a vehicle subject to legal discount, the Discounted parking fees apply for vehicles.

The parking control server 143 determines the parking fee of the vehicle, and transmits information on the parking fee of the vehicle to the system 100, 110, 120, or 130 through the network 201.

The plurality of clients 151-1 to 151-N are used to monitor different systems 100, 110, 120, and 130 in real time. The switch 141 is used to transmit information (eg, real-time parking screen information) output through the system 100, 110, 120, or 130 to any one of a plurality of clients 151-1 to 151-N. I can.

6 is a block diagram showing an interior of the vehicle number recognizer shown in any one of FIGS. 1 to 5.

Referring to FIG. 6, the vehicle number recognizer 300 is used to recognize a vehicle number when a vehicle enters a parking lot or when a vehicle leaves a parking lot. The structure and operation of the vehicle number recognizers shown in FIGS. 1 to 5 are the same as those of the vehicle number recognizer 300 shown in FIG. 6. Therefore, representatively, the vehicle number recognizer 300 shown in FIG. 6 will be described.

The vehicle number recognizer 300 includes a processor 310, a memory 320, a camera 330, and a communication module 340. The processor 310 executes instructions related to a method of recognizing a vehicle number stored in the memory 320. The camera 330 photographs the license plate area of the vehicle and transmits the photographed vehicle license plate image to the processor 310. The processor 310 recognizes the vehicle number by applying the feature maps extracted from the character region recognized in the license plate region of the vehicle to the CNN (Convolutional Neural Network) algorithm with respect to the vehicle license plate image. A vehicle license plate recognition technology in which the feature maps extracted from the character region recognized in the vehicle license plate region are applied to the CNN (Convolutional Neural Network) algorithm will be described in detail later.

7 is a flowchart illustrating an operation of a vehicle entering a parking lot in the system shown in FIG. 1. In FIG. 7, the unmanned exit settlement system shown in FIG. 1 has been described as an example, but the operation of entering the parking lot of the vehicle described in FIG. 7 is a pre-unmanned settlement system, a manned settlement system, and access control as shown in FIGS. The same or similar can be applied to the system.

Referring to FIGS. 1, 5 and 7, when the vehicle passes through the roof coil 90 installed on the floor of the entrance to the parking lot, the roof coil 90 recognizes the entry of the vehicle (S1001).

As the roof coil 90 recognizes the entry of the vehicle, the first vehicle number recognizer 20 and the second vehicle number recognizer 30 photograph vehicle license plates at the front and rear of the vehicle, respectively, and from the captured vehicle license plate image The license plate of the vehicle is recognized (S1002).

The vehicle number recognized by the first vehicle number recognizer 20 and the second vehicle number recognizer 30 is transmitted to the parking control server 143 through the network 201 (S1003).

The first vehicle number recognizer 20 and the second vehicle number recognizer 30 recognize a vehicle number and control the first circuit breaker 10 to open the first circuit breaker 10 (S1004). Under the control of the vehicle number recognizer 20 or 30, the first circuit breaker 10 is opened, and the vehicle enters.

8 is a flowchart illustrating an operation of entering a parking lot of a vehicle in the system shown in FIG. 1. The operation of entering the parking lot of the vehicle described in FIG. 8 may be applied in the same or similar manner to the pre-unmanned settlement system, the manned settlement system, and the access control system shown in FIGS.

1, 5, and 8, when the vehicle passes through the roof coil 90 installed on the floor of the parking lot exit, the roof coil 90 recognizes the exit of the vehicle (S2001).

As the roof coil 90 recognizes the advance of the vehicle, the third vehicle number recognizer 60 photographs the vehicle license plate in front of the vehicle, and recognizes the vehicle license plate from the photographed vehicle license plate image (S2002).

The vehicle number recognized by the third vehicle number recognizer 60 is transmitted to the parking control server 143 through the network 201 (S2003).

The parking control server 143 matches the vehicle number transmitted from the first vehicle number recognizer 20 and the second vehicle number recognizer 30 with the vehicle number transmitted from the third vehicle number recognizer 60, and the parking fee Is calculated (S2004). In this case, the web discount server 147 may determine a discount of the parking fee according to the vehicle information. The parking control server 143 may calculate the final parking fee according to the discount determined by the web discount server 147.

The parking control server 143 transmits the calculated parking fee to the exit unmanned fee calculator 70 (S2005).

The exit unmanned fare calculator 70 includes a display (not shown). The parking fee is displayed on the display, and the exit unmanned fee calculator 70 waits until the driver of the vehicle pays the parking fee by card or cash (S2006).

When the driver of the vehicle calculates the parking fee, the exit unmanned fee calculator 70 transmits an event signal indicating that the settlement has been processed to the parking control server 143 (S2007).

After the parking fee is settled, the exit unmanned fee calculator 70 controls the second circuit breaker 11 to open the second circuit breaker 11 (S2008). Under the control of the exit unmanned fare adjuster 70, the second circuit breaker 11 opens, and the vehicle enters.

9 is a flowchart illustrating a vehicle license plate recognition operation of FIG. 7 or 8. The vehicle license plate recognition operation described in FIG. 9 may be applied in the same or similar manner to the pre-unmanned settlement system, the manned settlement system, and the access control system shown in FIGS. 2 to 4.

The processor 310 shown in FIG. 6 refers to a processor that can be commonly implemented in the first vehicle number recognizer 20, the second vehicle number recognizer 30, and the third vehicle number recognizer 60. The operation of the processor 310 and the camera 330 described below will be understood as the operation of the first vehicle number recognizer 20, the second vehicle number recognizer 30, and the third vehicle number recognizer 60.

10 shows images processed by a processor of the vehicle number recognizer to describe the operation of the vehicle number recognizer shown in FIG. 6. Further, the operation of the processor refers to instructions stored in a memory included in the vehicle number recognizer shown in FIG. 6.

1, 6, 9, and 10, a camera 330 included in the vehicle number recognizer 300 photographs a license plate of a vehicle (S4001).

The processor 310 binarizes the vehicle license plate image captured by the camera 330 (S4003). A specific algorithm for binarizing the vehicle license plate image is as follows.

The processor 310 extracts horizontal boundary points of text stroke candidates by applying a DoG (Difference of Gaussian) filter to the vehicle license plate image and generates a horizontal stroke boundary map (Horizontal Stroke Boundary Map). FIG. 10B shows an image of a horizontal stroke boundary point map. The processor 310 generates a segment image by grouping pixels between text strokes through vertical accumulation of horizontal boundary point connection lines. 10C shows a segment image. The processor 310 generates a color image so that each text has a different color in the segment image. 10D shows the color image. Depending on the embodiment, the color image generation operation may be omitted. The processor 310 converts the color image into a black and white image. 10A shows a black and white image.

Referring to FIG. 9, the processor 310 performs character blob analysis. The processor 310 indexes the pixel chunks, calculates location information and size information of the pixel chunks, analyzes the indexed pixel chunks, and extracts license plate candidate characters (S4004).

The processor 310 extracts and cuts out the license plate area of the vehicle according to the extracted license plate candidate characters (S4005). The processor 310 converts the cut-out license plate area into a grayscale image.

The processor 310 estimates the sharpness of the cut-out license plate area (S4006). The processor 310 estimates the sharpness, normalizes the size of the license plate and corrects the inclination (S4007). The processor 310 calculates the sharpness, and determines whether the calculated sharpness is greater than a predetermined value (S4008). When the calculated sharpness is greater than a predetermined value, the processor 310 decreases the sharpness (S4009). When the calculated sharpness is less than a predetermined value, the processor 310 improves the image of the license plate area of the vehicle (S4010).

The processor 310 selects a license plate type from the image including the license plate area of the vehicle (S4011). The license plate type may be divided into a single line license plate or a double line license plate. In addition, the license plate type may be classified by whether or not a place name is included. The processor 310 classifies the character area from the classified license plate area (S4012).

11 shows different images processed by the processor of the vehicle number recognizer to illustrate the operation of the vehicle number recognizer shown in FIG. 6. Specifically, FIG. 11 is an image showing classification of an incorrect character area. In the classified license plate area, the classification operation of the character area is important. If the classification operation is wrong, it may be recognized as an incorrect character.

The processor 310 extracts a preset feature map from the classified character area (S4013). The preset feature map is defined as a numerical expression of character features. The preset feature map is used as an input in an ANN structured neural network algorithm. For example, the preset feature map may divide a character area into predetermined areas and be defined as the number of pixels of characters included in each area. In addition, the preset feature map may be defined as a sum of white pixels or black pixels positioned on horizontal lines drawn in multiple numbers along the height of a character. In addition, the preset feature map may be defined as a sum of white pixels or black pixels positioned on vertical lines drawn in multiple numbers according to a width.
The processor 310 outputs a CNN feature map by performing a CNN algorithm in the classified character region (S4014).

The processor 310 recognizes the vehicle license plate by applying the feature maps extracted from the character region recognized in the license plate region of the vehicle together with a Convolutional Neural Network (CNN) algorithm (S4015). That is, the processor 310 recognizes the vehicle license plate by applying the CNN algorithm as inputs to the preset feature map and the CNN feature map output through the CNN algorithm. The application of the feature maps extracted from the text region recognized in the vehicle license plate region to the CNN (Convolutional Neural Network) algorithm will be described in detail later.

The processor 310 recognizes the character string, determines whether or not it is misrecognized according to the type and arrangement rule of the characters, and outputs a final license plate character string recognition result (S4016).

12 shows a block diagram of a CNN algorithm processed by the processor shown in FIG. 6.

6, 9, and 12, a convolutional neural network (CNN) is a deep learning algorithm and is mainly used to analyze an image.

The CNN algorithm includes a first convolution layer, a first max pooling layer, a second convolution layer, a second max pooling layer, and a fully connected layer. Includes (fully connected layer).

The processor 310 performs an operation of the first convolution layer. The operation in the first convolution layer is as follows.

The processor 310 receives an image including one recognized character area. The image including the one recognized character area includes one number, one English character, or one Hangul. The image is composed of pixel values having 0 or 1. The image is expressed as (horizontal size, vertical size, number of channels). When the image is binarized, the number of channels is 1. For example, the image shown in FIG. 12 may be expressed as (28, 28, 1).

The processor 310 outputs first convolution result values by performing a convolution operation on the received image and the first filter. The received image may be referred to as an input feature map. The first convolution result values may be referred to as a first output feature map. The first filter is implemented in a matrix form and is used to detect an edge in the received image or sharpen the received image. Depending on each purpose, the integer used in the matrix may vary. The first filter may be referred to as a kernel. The first filter may be expressed as (the horizontal size of the filter, the vertical size of the filter, the number of filter channels) x (the number of filters). For example, the first filter shown in FIG. 12 may be expressed as (5, 5, 1) x 16.

The convolution operation means convolution of the received image and the first filter. In this case, a weight may be multiplied by the first filter. The first output feature map resulting from the convolution operation may be expressed as (horizontal size, vertical size, number of filters).

When the input feature map that is the received image shown in FIG. 12 is (28, 28, 1) and the first filter is (5, 5, 1) x 16, the first output feature map is (24, 24, It can be expressed as 16).

FIG. 13 is a block diagram illustrating a relationship between an input feature map and an output feature map shown in FIG. 12.

6, 9, 12, and 13, an input feature map is (l-1), an output feature map is (l), and a convolution operation using the filter bank w is as follows.

[Equation 1]

는 k번째 피처 맵의 c번째 필터의 u행, v열에 위치한 뉴런의 가중치를, 상기

는 입력 피처 맵(l-1)의 c 피처 맵의 (i+u)행, (j+v)열에 있는 뉴런의 출력을, 상기

는 l층의 k번째 피처 맵의 바이어스를, 상기

는 출력 피처 맵(l)의 k번째 출력 피처 맵에서 i행, j열에 위치한 뉴런의 출력을 의미한다. 상기 fc는 입력 피처 맵의 채널 개수, 상기 k는 필터 뱅크의 인덱스를, 상기 c는 입력 피처 맵의 인덱스를. 상기 u와 v는 필터의 가로, 세로 인덱스를 의미한다. remind

Is the weight of neurons located in row u and column v of the c-th filter of the k-th feature map,

Is the output of the neuron in the (i+u) row, (j+v) column of the c feature map of the input feature map (l-1),

Is the bias of the k-th feature map of the l layer,

Denotes the output of neurons located in row i and column j in the k-th output feature map of the output feature map (l). Wherein fc is the number of channels of the input feature map, k is the index of the filter bank, and c is the index of the input feature map. The u and v denote the horizontal and vertical indexes of the filter.

For example, the second output feature map (12, 12, 16) that is the output of the first max pooling layer is the input feature map (l-1), and the third output feature map (10, which is the output of the second convolution layer) 10, 24) may be the output feature map (l). In this case, the bank of the third filter (3, 3, 16) x 24 may be w.

Referring to FIG. 12, the processor 310 performs an operation of the first max pooling layer. The operation in the first max pooling layer refers to outputting a second output feature map by applying a second filter to a first output feature map that is a result value of the first convolution layer. The second filter may be expressed as (the horizontal size of the filter and the vertical size of the filter). For example, the second filter shown in FIG. 12 may be expressed as (2, 2).

When the first output feature map shown in FIG. 12 is (24, 24, 16) and the second filter is (2, 2), the second output feature map may be expressed as (12, 12, 16). .

The operation in the first max pooling layer may be expressed as the following equation.

[Equation 2]

Here, m and n denote the horizontal and vertical dimensions of the filter, respectively, and s and t denote the sampling area index of the input feature map.

The processor 310 performs an operation of the second convolution layer. The operation in the second convolution layer is similar to the operation in the first convolution layer. That is, the processor 310 outputs a third output feature map by convolving the second output feature map and the third filter, which are outputs of the first max pooling layer. The third filter may be a filter different from the first filter. When the second output feature map that is the received image shown in FIG. 12 is (12, 12, 16) and the third filter is (3, 3, 16) x 24, the third output feature map is (10, It can be expressed as 10, 24).

The processor 310 performs an operation of the second max pooling layer. The operation in the second max pooling layer is as follows. The operation in the second max pooling layer is similar to the operation in the first max pooling layer. That is, the processor 310 outputs the fourth output feature map by applying the fourth filter to the third output feature map, which is the output of the second convolution layer. The fourth filter may be a filter different from the second filter. When the third output feature map shown in FIG. 12 is (10, 10, 24) and the fourth filter is (2, 2), the fourth output feature map may be expressed as (5, 5, 24). .

)은 다음의 수학식과 같이 표현될 수 있다. The processor 310 performs an operation of a fully connected layer. The operation in the fully connected layer is as follows. The pulley connected layer includes a plurality of pulley connected layers. Each of the first pulley connected layer and the second pulley connected layer is expressed as (the number of input neurons and the number of output neurons). The number of input neurons in the first pulley-connected layer may be 600, and the number of output neurons in the first pulley-connected layer may be 200. The input neurons of the first pulley connected layer are derived from the fourth output feature maps 5, 5, and 24. J-th output neuron of the first pulley connected layer (

) Can be expressed as the following equation.

[Equation 3]

In addition, the number of input neurons in the second pulley-connected layer may be 200, which is the number of output neurons in the first pulley-connected layer, and 10, which is the number of output neurons in the second pulley-connected layer. An input value z and an output value of the sum of the weights of the k-th neuron of the second pulley connect layer may be expressed by the following equation.

[Equation 4]

The processor 310 receives a plurality of different images and repeats the operations of the CNN algorithm of the first convolution layer, the first max pooling layer, the second convolution layer, the second max pooling layer, and the fully connected layer. Train the CNN algorithm. According to learning, the weight of the fully connected layer and the filter weight of the convolutional layers are updated.

14 shows an embodiment of a block diagram of a neural network algorithm processed by the processor shown in FIG. 6. 15 shows another embodiment of a block diagram of a neural network algorithm processed by the processor shown in FIG. 6. 16 shows another embodiment of a block diagram of a neural network algorithm processed by the processor shown in FIG. 6.

6 and 14 to 16, the processor 310 receives an image including one recognized character area. The processor 310 outputs a first output feature map by performing a first convolution operation on the received image.

The processor 310 outputs a second output feature map by applying sampling to the first output feature map. That is, the processor 310 performs an operation of the first max pooling layer.

The processor 310 outputs a third output feature map by performing a second convolution operation on the output second output feature map.

The processor 310 outputs a fourth output feature map by applying sampling to the outputted third output feature map. That is, the processor 310 performs an operation of the second max pooling layer.

The processor 310 recognizes a character in the one recognized character area using a fully connected layer that completely connects the output fourth output feature map and a preset feature map.

The fully connected layer includes an input layer, a hidden layer, and an output layer.

The hidden layer includes a plurality of hidden neurons. The hidden layer may be expressed as (the number of input neurons of the hidden layer and the number of output neurons of the hidden layer). The output layer may be expressed as (the number of input neurons in the output layer and the number of output neurons in the output layer). For example, (840, 60) of the hidden layer shown in FIG. 15 means that the number of input neurons of the hidden layer is 840 and the number of output neurons of the hidden layer is 60. Further, (60, 10) of the output layer shown in FIG. 15 means that the number of input neurons of the output layer is 60 and the number of output neurons of the output layer is 10.

)은 다음의 수학식으로 표현된다. When the number of features of the preset feature map is m (m is a natural number of 2 or more), and the number of features of the output fourth output feature map is n (n is a natural number of 2 or more), the plurality of hidden neurons Among them, the input value of the hidden neuron (j is a natural number) (

) Is expressed by the following equation.

[Equation 5]

)은 다음의 수학식으로 표현된다. The output value of the j-th (j is a natural number) hidden neuron among the plurality of hidden neurons (

) Is expressed by the following equation.

[Equation 6]

는 상기 a번째 특징에 대한 j번째 가중치를, 상기

는 상기 미리 설정된 피처 맵의 특징들 중 a번째 특징을, 상기

는 상기 c번째 결과 값에 대한 j번째 가중치를, 상기

는 상기 출력된 제4출력 피처 맵의 특징들 중 c번째 결과 값을, 상기

는 j번째 오프셋(offset)을, 상기 함수 f()는 시그모이드(sigmoid) 함수를 나타낸다.remind

Is the j-th weight for the a-th feature,

Is the a-th feature among the features of the preset feature map,

Is the j-th weight for the c-th result value,

Is the c-th result value among the features of the outputted fourth output feature map,

Denotes the j-th offset, and the function f() denotes a sigmoid function.

Only features of the preset feature map are connected to each other, and only features of the output fourth output feature are connected to each other.

)은 다음의 수학식으로 표현된다. The output layer includes a plurality of output neurons, and an input value of the output neuron (k is a natural number) among the plurality of output neurons (

) Is expressed by the following equation.

[Equation 7]

)은 다음의 수학식으로 표현된다. The output value of the k-th (k is a natural number) output neuron among the plurality of output neurons (

) Is expressed by the following equation.

[Equation 8]

는 상기 j번째 은닉 뉴런의 출력 값에 대한 k번째 가중치를, 상기

는 상기 j번째 은닉 뉴런의 출력 값을, 상기

는 k번째 오프셋을 나타낸다. remind

Is the k-th weight for the output value of the j-th hidden neuron,

Is the output value of the j-th hidden neuron,

Represents the k-th offset.

와 10개의 뉴런들의 전체 에러 E는 다음의 수학식들로 표현된다. The error of the k-th neuron among the plurality of output neurons

The total error E of and 10 neurons is expressed by the following equations.

[Equation 9]

는 입력 샘플 이미지에 대한 기대되는 출력 값이고, 상기

는 실제 출력 값을 나타낸다. remind

Is the expected output value for the input sample image, where

Represents the actual output value.

[Equation 10]

E represents the total error of 10 neurons.

The processor 310 updates the weight W in order to reduce the total error E even in a fully connected layer that completely connects the output fourth output feature map and the preset feature map.

An error of the k-th weight with respect to the output value of the j-th hidden neuron and the error of the j-th weight with respect to the output value of the i-th input neuron may be calculated by an error backpropagation algorithm. The error inversion plate algorithm is expressed by the following equations. In the conventional CNN algorithm, the error backpropagation algorithm was not used.

[Equation 11]

[Equation 12]

는 상기 j번째 은닉 뉴런의 출력 값에 대한 k번째 가중치의 오차를, 상기

는 상기 i번째 입력 뉴런의 출력 값에 대한 j번째 가중치의 오차를 나타낸다. 상기

은 학습율을, 상기 f'(z_k)는

를 나타낸다.remind

Is the error of the k-th weight with respect to the output value of the j-th hidden neuron,

Denotes an error of the j-th weight with respect to the output value of the i-th input neuron. remind

Is the learning rate, and f'(z _k ) is

Represents.

The processor 310 calculates an error of the weights in the hidden layer and the output layer, and updates the existing weights using the following equation.

[Equation 13]

는 모멘템으로 가중치를 업데이트할 때, 기존의 업데이트 방향으로 관성이 유지되도록하는 상수이다.remind

Is a constant that keeps the inertia in the existing update direction when updating the weight with the momentum.

The integrated parking management system 200 using a neural network recognizes the vehicle license plate by applying the feature maps extracted from the text region recognized in the license plate region of the vehicle together with the CNN (Convolutional Neural Network) algorithm, thereby recognizing the vehicle license plate using only the CNN algorithm. It has the advantage of being able to accurately recognize the vehicle license plate than to recognize it.

17 shows a graph of overfit distortion according to learning of the neural network algorithm shown in FIG. 6.

Referring to FIG. 17, when a neural network algorithm is repeatedly trained to update weights, a crystal plane may be distorted as shown in FIG. 17. In particular, the ability to classify samples in the vicinity of the crystal plane may deteriorate. A phenomenon in which the classification ability is deteriorated by excessively updating the weights is called over-fitting or overfitting.

18 shows a graph for solving the problem of overfit distortion shown in FIG. 17. The X-axis of FIG. 18 indicates the number of repetitive learning, and the Y-axis indicates accuracy.

Referring to FIGS. 6 and 18, the accuracy of object classification does not increase as the number of repetitive learning increases. When the number of repetitive learning is excessively performed, a problem of over-fitting may occur. When the number of iterative learning is performed too little, a problem of under-fitting, which is less accurate, may occur.

To solve the over-fitting problem, the processor 310 applies the following equations.

[Equation 14]

는 상기 i번째 입력 뉴런의 출력 값에 대한 j번째 가중치의 오차를 , 상기

는 상수로, 기존의 가중치를 줄이기 위해 가중치 감소 비율을, 상기

는 가중치를 어느 정도로 업데이트할 것인지 결정하는 학습율을 나타낸다. 상기

는 상기 i번째 입력 뉴런의 출력 값에 대한 j번째 가중치를 나타낸다. remind

Is the error of the j-th weight with respect to the output value of the i-th input neuron,

Is a constant, the weight reduction ratio to reduce the existing weight,

Denotes a learning rate that determines how much to update the weights. remind

Represents the j-th weight for the output value of the i-th input neuron.

[Equation 15]

는 상기 j번째 은닉 뉴런의 출력 값에 대한 k번째 가중치의 오차를, 상기

는 상수로, 기존의 가중치를 줄이기 위해 가중치 감소 비율을, 상기

는 가중치를 어느 정도로 업데이트할 것인지 결정하는 학습율을, 상기

는 복수의 출력 뉴런들 중 k번째 뉴런의 에러를 나타낸다. 상기

는 상기 j번째 은닉 뉴런의 출력 값에 대한 k번째 가중치를 나타낸다. remind

Is the error of the k-th weight with respect to the output value of the j-th hidden neuron,

Is a constant, the weight reduction ratio to reduce the existing weight,

Is the learning rate that determines how much to update the weights,

Represents the error of the k-th neuron among the plurality of output neurons. remind

Represents the k-th weight for the output value of the j-th hidden neuron.

Equations 14 and 15 are different from Equations 11 and 12. When updating the weights using Equations 14 and 15, the over-fitting problem can be solved by reducing the weight to some extent for a large weight.

FIG. 19 shows another embodiment of a block diagram of a neural network algorithm processed by the processor shown in FIG. 6.

6 and 19, the processor 310 compares a difference and a threshold value between output result values using a fully connected layer that completely connects the output fourth output feature map and the preset feature map. Thus, it is determined whether there is a possibility of confusion in recognizing a character other than an expected character through the output result values. The possibility of confusion is determined by the classifier shown in FIG. 15. For example, numbers 3 and 8 can be confusing due to the influence of ambient light or the angle of the camera. In the image including the number 3, the character expected through recognition is 3, but it can be recognized as another character 8. Conversely, when the expected character is 8, it may be recognized as another character 3.

For the image including the number 3, result values derived by applying a fully connected layer that completely connects the output feature map and the preset feature map may have a probability of being number 3 being 0.7 and a probability of being number 8 being 0.6. . When the difference (0.1) between the derived result values (eg, 0.7 and 0.6) is less than the threshold value (eg, 0.2), it may be determined that there is a possibility of confusion with other characters. Accordingly, it may be determined that there is a possibility of confusion with the number 8 other than the expected number 3 through the result values derived by applying the fully connected layer.

When it is determined through the output result values that there is a possibility of confusion in recognizing a character other than the expected character, the processor 310 resets the preset feature map to define a second preset feature map. In this case, the preset feature map may be referred to as a first preset feature map.

The resetting means increasing the number of preset feature maps to clearly distinguish characters that may be confusing. Feature maps (e.g., one pixel on the left and one pixel on the right, based on the center of the character) that can indicate the characteristics of the characters (e.g., symmetry, or whether lines are connected) that can be distinguished from each other in potentially confusing characters (e.g., 3 and 8). Whether one pixel corresponds to each other, or whether black pixels are continuously connected) may be added.

The first preset feature map and the second preset feature map are used as inputs in an ANN structured neural network. For example, the first and second preset feature maps may divide a character area into predetermined areas and be defined as the number of pixels of characters included in each area. In addition, the first and second preset feature maps may be defined as the sum of white pixels or black pixels positioned on horizontal lines drawn in multiple numbers along the height of a character. In addition, it may be defined as a sum of white pixels or black pixels positioned on vertical lines drawn in multiple numbers according to the first and second preset feature map widths. However, the second preset feature map is different from the first preset feature map in that it is set to clearly distinguish characters that may be confusing.

20 are images showing differences between numbers that may be confusing.

Referring to FIG. 20, the number 8 is symmetric about the red line, and the number 3 is not symmetric about the red line. The second preset feature map may be set to determine the symmetry of these numbers.

In addition, referring to FIG. 20, black pixels are connected to the left square of the number 8, but black pixels are not connected to the left square of the number 3. According to an embodiment, the second preset feature map may be set to determine the connectivity of such pixels.

The processor 310 provides a second fully connected layer that completely connects the fifth output feature map output by re-applying the CNN algorithm to the image including the classified character region and the second preset feature map. Apply to recognize characters. A pulley connected layer that completely connects the output fourth output feature map and the first preset feature map may be referred to as a first pulley connected layer.

In order to clearly recognize the potentially confusing characters, an operation of applying the second pulley connected layer is performed. The applied CNN algorithm is the same as the weights used in the process of outputting the fourth output feature map.

The number of the fourth output feature maps input to the first pulley connected layer and the number of fifth output feature maps input to the second pulley connected layer are different from each other. For example, the number of the fourth output feature map may be greater than the number of the fifth output feature map. When the number of the fourth output feature maps is n, the number of the fifth output feature maps may be q (q is a natural number less than n).

The number of first preset feature maps input to the first pulley connected layer and the number of second preset feature maps input to the second pulley connected layer are different from each other. For example, the number of first preset feature maps may be smaller than the number of second preset feature maps. When the number of first preset feature maps is m, the number of second preset feature maps may be p (p is a natural number greater than m). By making the number of the second preset feature maps greater than the number of the first preset feature maps, characters that may be confused can be clearly identified. That is, by increasing the number of the second preset feature maps, more character features can be extracted than the first preset feature map.

The sum of the number of the fourth output feature map and the number of the first preset feature map, the number of the fifth output feature map and the number of the second preset feature map are the same.

Weights in the hidden layer and the output layer of the first pulley-connected layer and the weights in the hidden layer and the output layer of the second pulley-connected layer are the same.

Although the present invention has been described with reference to the exemplary embodiment shown in the drawings, this is only exemplary, and those of ordinary skill in the art will appreciate that various modifications and other equivalent embodiments are possible therefrom. Therefore, the true technical protection scope of the present invention should be determined by the technical idea of the attached registration claims.

100: unattended exit settlement system; 110: advance unmanned settlement system;
10: first circuit breaker; 120: manned settlement system;
11: second breaker; 130: access control system;
20: a first vehicle number recognizer; 140: server system;
30: second vehicle number recognizer; 200: integrated parking management system;
40: light vehicle discrimination sensor;
50: four-sided camera;
60: third vehicle number recognizer;
70: unmanned withdrawal fee calculator;
80: exit warning light;
90: loop coil;

Claims (9)

An integrated parking management method to which vehicle license plate recognition technology is applied using a neural network performed by a vehicle number recognizer including a processor, a memory, a camera, and a communication module in order to recognize characters on the vehicle license plate,
The processor, when the entrance of the vehicle is recognized by the roof coil, receiving an image including the license plate area of the vehicle photographed by the camera from the camera;
Receiving, by the processor, an image including a text area classified in a license plate area of the vehicle;
The processor performing a first convolution operation on the received image to output a first output feature map;
The processor outputting a second output feature map by applying sampling to the output first output feature map;
The processor performing a second convolution operation on the second output feature map to output a third output feature map;
The processor outputting a fourth output feature map by applying sampling to the outputted third output feature map;
The processor is defined as a numerical expression of the outputted fourth output feature map and characteristics of the character, and divides the character area into predetermined areas and determines the number of pixels of characters included in each area, or the height of the character. A preset feature map that can be defined as the sum of white pixels or black pixels located on multiple horizontal lines, or white pixels located on multiple vertical lines or black pixels depending on the width. Recognizing a character in the classified character area using a fully connected layer that is fully connected; And
The processor includes the step of transmitting the recognized character to the parking control server,
The parking control server uses a neural network to calculate a parking fee according to the recognized character so that the driver of the vehicle can pay the parking fee of the vehicle, and transmit the calculated parking fee of the vehicle to an exit unmanned fee calculator. Integrated parking management method with license plate recognition technology applied The method of claim 1, wherein the fully connected layer,
Includes an input layer, a hidden layer, and an output layer,
The hidden layer includes a plurality of hidden neurons (neurons),
When the number of features of the preset feature map is m (m is a natural number of 2 or more), and the number of features of the output fourth output feature map is n (n is a natural number of 2 or more),
The input value of the j-th (j is a natural number) hidden neuron among the plurality of hidden neurons (

) Is expressed by the following equation,
[Equation]

The output value of the j-th (j is a natural number) hidden neuron among the plurality of hidden neurons (

) Is expressed by the following equation,
[Equation]

remind

Is the j-th weight for the a-th feature,

Is the a-th feature among the preset features,

Is the j-th weight for the c-th result value,

Is the c-th result value among the features of the outputted fourth output feature map,

Denotes the j-th offset, and the function f() denotes a sigmoid function. The method of claim 2,
The output layer includes a plurality of output neurons,
The input value of the k-th (k is a natural number) output neuron among the plurality of output neurons (

) Is expressed by the following equation,
[Equation]

The output value of the k-th (k is a natural number) output neuron among the plurality of output neurons (

) Is expressed by the following equation,
[Equation]

remind

Is the k-th weight for the output value of the j-th hidden neuron,

Is the output value of the j-th hidden neuron,

Is a k-th offset, and the function f() is a sigmoid function. An integrated parking management method to which vehicle license plate recognition technology is applied using a neural network. The method of claim 3,
The error of the j-th weight with respect to the output value of the i-th input neuron

Is expressed by the following equation,
[Equation]

remind

Is a constant, the weight reduction ratio to reduce the existing weight,

Represents the learning rate that determines how much to update the weights,
The error of the k-th weight with respect to the output value of the j-th hidden neuron

Is expressed by the following equation,
[Equation]

remind

Is a constant, the weight reduction ratio to reduce the existing weight,

Is the learning rate that determines how much to update the weights,

Is the error of the k-th neuron among the plurality of output neurons, and f'(z _k ) is

An integrated parking management method applied with vehicle license plate recognition technology using a neural network representing A roof coil that recognizes the vehicle's entry; And
A vehicle number recognizer for photographing the license plate of the vehicle and recognizing the vehicle license plate from the photographed vehicle license plate image,
The vehicle number recognizer,
A camera for photographing the license plate of the vehicle;
A communication module for transmitting the recognized vehicle number to the parking control server;
A memory for storing instructions related to vehicle license plate recognition technology using a neural network; And
And a processor that executes the instructions,
The above commands are:
Receives an image including the classified text area processed by the license plate photographed by the camera, performs a first convolution operation on the received image to output a first output feature map, and outputs the first Sampling is applied to the output feature map to output a second output feature map, a second convolution operation is performed on the second output feature map to output a third output feature map, and the output third output feature Sampling is applied to the map to output a fourth output feature map, and characters are recognized in the classified text area using a fully connected layer that completely connects the output fourth output feature map and a preset feature map. And, it is implemented to transmit the recognized text to a parking control server for calculating the parking fee of the vehicle,
The parking control server is an exit unmanned settlement system for transmitting the calculated parking fee of the vehicle to an exit unmanned fee calculator for calculating the parking fee. A roof coil that recognizes the vehicle's entry;
A vehicle number recognizer that photographs the license plate of the vehicle and recognizes the license plate of the vehicle from the photographed vehicle license plate image; And
And an unmanned fare calculator for the driver of the vehicle to pay the parking fee in advance before leaving the vehicle,
The vehicle number recognizer,
A camera for photographing the license plate of the vehicle;
A communication module for transmitting the recognized vehicle number to the parking control server;
A memory for storing instructions related to vehicle license plate recognition technology using a neural network; And
And a processor that executes the instructions,
The above commands are:
Receives an image including a character area classified in the license plate photographed by the camera, performs a first convolution operation on the received image to output a first output feature map, and outputs the first output feature Sampling is applied to the map to output a second output feature map, and a second convolution operation is performed on the second output feature map to output a third output feature map, and the outputted third output feature map The fourth output feature map is output by applying sampling, and is defined as a numerical expression of the outputted fourth output feature map and the character characteristics, and the character area is divided into certain areas and included in each area. It is the number of pixels of a character, white pixels located on multiple horizontal lines drawn along the height of the character, or the sum of black pixels, or white pixels located on multiple vertical lines drawn along the width, or black pixels. Recognizes characters in the classified character area using a fully connected layer that fully connects a preset feature map that can be defined as a sum, and uses the recognized characters as a parking control server for calculating the parking fee of the vehicle. Is implemented to transmit,
The parking control server transmits the calculated parking fee of the vehicle to an unmanned fee calculator for calculating the parking fee. A roof coil that recognizes the vehicle's entry;
A vehicle number recognizer that photographs the license plate of the vehicle and recognizes the license plate of the vehicle from the photographed vehicle license plate image; And
It includes a parking management booth that receives the parking fee of the vehicle from the parking control server,
The vehicle number recognizer,
A camera for photographing the license plate of the vehicle;
A communication module for transmitting the recognized vehicle number to the parking control server;
A memory for storing instructions related to vehicle license plate recognition technology using a neural network; And
And a processor that executes the instructions,
The above commands are:
Receives an image including the classified text area processed by the license plate photographed by the camera, performs a first convolution operation on the received image to output a first output feature map, and outputs the first Sampling is applied to the output feature map to output a second output feature map, a second convolution operation is performed on the second output feature map to output a third output feature map, and the output third output feature Sampling is applied to the map to output a fourth output feature map. It is defined as a numerical expression of the outputted fourth output feature map and the character characteristics. The character area is divided into certain areas, and each area is It is the number of pixels of the included character, white pixels located on multiple horizontal lines drawn along the height of the character, or the sum of black pixels, white pixels located on multiple vertical lines drawn along the width, or black It recognizes characters in the text region the classification using a pulley, connected layer to fully connect the preset feature map can be defined as the sum of the pixels, and parking for calculating a parking fee for the vehicle to the recognized character It is implemented to transmit to the control server,
The parking control server manned settlement system for transmitting the calculated parking fee of the vehicle to a PC of a parking management booth capable of calculating the parking fee. A camera for photographing the license plate of the vehicle;
A memory for storing instructions related to vehicle license plate recognition technology using a neural network;
A communication module for transmitting the number of the vehicle recognized using the vehicle license plate recognition technology using the neural network to a parking control server; And
And a processor that executes the instructions,
The above commands are:
Receives an image including the classified text area processed by the license plate photographed by the camera, performs a first convolution operation on the received image to output a first output feature map, and outputs the first Sampling is applied to the output feature map to output a second output feature map, a second convolution operation is performed on the second output feature map to output a third output feature map, and the output third output feature Sampling is applied to the map to output a fourth output feature map. It is defined as a numerical expression of the outputted fourth output feature map and the character characteristics. The character area is divided into certain areas, and each area is It is the number of pixels of the included character, white pixels located on multiple horizontal lines drawn along the height of the character, or the sum of black pixels, white pixels located on multiple vertical lines drawn along the width, or black A vehicle number recognizer implemented to recognize a character in the classified character region by using a fully connected layer that completely connects a preset feature map that can be defined as a sum of pixels. A roof coil that recognizes the vehicle's entry; And
A vehicle number recognizer for photographing the license plate of the vehicle and recognizing the vehicle license plate from the photographed vehicle license plate image,
The vehicle number recognizer,
A camera for photographing the license plate of the vehicle;
A communication module for transmitting the recognized vehicle number to the parking control server;
A memory for storing instructions related to vehicle license plate recognition technology using a neural network; And
And a processor that executes the instructions,
The above commands are:
Receives an image including the classified text area processed by the license plate photographed by the camera, performs a first convolution operation on the received image to output a first output feature map, and outputs the first Sampling is applied to the output feature map to output a second output feature map, a second convolution operation is performed on the second output feature map to output a third output feature map, and the output third output feature Sampling is applied to the map to output a fourth output feature map. It is defined as a numerical expression of the outputted fourth output feature map and the character characteristics. The character area is divided into certain areas, and each area is It is the number of pixels of the included character, white pixels located on multiple horizontal lines drawn along the height of the character, or the sum of black pixels, white pixels located on multiple vertical lines drawn along the width, or black A character is recognized in the classified character area using a fully connected layer that completely connects a preset feature map that can be defined as a sum of pixels, and whether the recognized character is a vehicle with the proper authority Access control system implemented to transmit to a terminal that can be determined.

Images