KR102238401B1 - Recognition Method of License Plate Number Using Data expansion and CNN - Google Patents

Abstract

The present invention relates to a vehicle number recognizing method using data extension and a CNN. The vehicle number recognizing method using data expansion and a CNN comprises: an image input step (S100) of inputting a vehicle image for learning; a vehicle area detection step (S200) of detecting a vehicle area (A1) in the input vehicle image; a number plate area detection step (S300) of detecting a number plate area (A2) located in the vehicle area (A1); a cropping step (S400) of extracting a cropping image (C1); an expansion image generation step (S500) of generating an expansion image (E1); an expanded vehicle number recognition learning data generation step (S600); and a vehicle number recognition step (S700) of recognizing a vehicle number from image data input from a camera. Therefore, the vehicle number recognizing method using data expansion and a CNN can accurately recognize a vehicle number from a video image.

Description

Recognition Method of License Plate Number Using Data expansion and CNN}

The present invention relates to a vehicle number recognition method and apparatus using artificial intelligence. In more detail, the present invention relates to a vehicle number recognition method and a vehicle number recognition apparatus using data expansion and CNN that can more accurately recognize vehicle numbers from image images using data expansion and CNN.

Vehicle license plates are attached to the front or rear of the vehicle, and standardized characters (including letters, numbers, and symbols) are written in a specific pattern. The vehicle number recognition device refers to a device that automatically recognizes and reads a vehicle license plate and characters described therein from an image including a vehicle. A typical vehicle number recognition apparatus includes the steps of photographing a vehicle, extracting a license plate, performing a pre-processing according to a shade or an inclination, and recognizing a character written on the license plate according to a predetermined reference letter. A recognition algorithm for the vehicle number recognition device to make characters more accurate is a very important factor in the vehicle number recognition device.

Conventional license plate recognition uses a circular matching method and a syntactic method, and the circular matching method is a method that improves the geometric matching method, and refers to a method of recognizing characters by matching an image input to a standard pattern. However, if there is an inclined image or noise, the recognition rate is lowered, and when the environment changes, the standard pattern must be reconstructed. The syntactic method uses structural information such as interrelationship or interconnectivity information between character features, and It has strong characteristics such as size and slope. In particular, there is a difficulty in being able to quantify and extract structural information, and it is difficult to obtain accurate structural information between features.

Recently, a lot of attention has been focused on a license plate recognition method using a neural network. Neural network or artificial neural network (ANN: Artificial Neural Network) is a statistical learning algorithm inspired by biological neural networks.In artificial neural networks, artificial neurons (nodes) that form a network by combining synapses change the strength of synapses through learning. It refers to the overall model with problem-solving ability. Furthermore, a convolutional neural network (CNN) is a field of machine learning based on a multi-layered artificial neural network, a machine learning algorithm that attempts a high level of abstraction through a combination of several nonlinear transducers. It is a set of.

The method using a neural network is a method of dividing a region of an input image as an input of a neural network, and recognizing a character using a neural network.

However, unlike general character recognition, the process of recognizing the characters on the vehicle license plate from the input image is greatly affected by the surrounding environment, and distortion occurs due to camera noise, lighting changes, and weather. Research on a robust treatment method for this is required. Accordingly, in the prior literature, the image captured before the vehicle number recognition is pre-processed to improve the image quality and then the vehicle number is recognized, but this has a disadvantage in that the time and resources required for each vehicle number recognition increase due to the pre-processing process. Furthermore, there is a problem in that it is practically difficult to acquire a large number of learning images captured in various surrounding environments for artificial neural network learning.

Korean Registered Patent Publication No. 10-1931804 (2018.12.17)

The problem to be solved in the present invention is to solve the above-described problem, and an object of the present invention is to extend data that can more accurately recognize vehicle number from image images using data expansion and CNN, and vehicle number recognition using CNN. It is to provide a method and a vehicle number recognition device.

In order to solve the above-described problem, the method of recognizing a vehicle number using data expansion and CNN

An image input step of inputting an image of a vehicle for learning (S100); A vehicle area detection step (S200) of detecting a vehicle area (A1) in the vehicle image input in the image input step (S100) using vehicle detection learning data and a single shot multibox detector (SSD) algorithm; The vehicle area detection step (S300) of detecting a license plate area A2 located in the vehicle area A1 in the vehicle area A1 detected in S200; detection in the vehicle area detection step S200 A cropping step (S400) of extracting a cropping image (C1) by cropping an area including the license plate area (A2) detected in the license plate area detection step (S300) within one vehicle area (A1); With respect to the cropping image C1 extracted in the cropping step (S400), an expanded image generation step (S500) of generating an expanded image (E1) transforming the image using a data expansion algorithm (S500); the expanded image generation step (S500) The extended vehicle number recognition learning data generation step (S600) of performing vehicle number recognition learning by inputting the extended image (E1) generated in) into a convolutional neural network (CNN) and generating extended vehicle number recognition learning data (S600); And a vehicle number recognition step (S700) of recognizing a vehicle number from image data input from the camera using the expanded vehicle number recognition learning data.

The data expansion algorithm of the extended image generation step (S500) generates an extended image using an optical transformation algorithm, and may generate an extended image E1 by adding a noise value to the color value of the cropping image C1.

The data expansion algorithm of the expanded image generation step (S500) includes a rotation operation to rotate the image, a warping operation to distort the image, and moving the image with respect to the cropping image C1 extracted in the cropping step (S400). A geometric transformation method that generates an extended image E1 by a transformation operation method including a movement operation and an enlargement operation that enlarges the image, and the extended image E1 by distorting each pixel included in the image in a random direction. It may include a generated elastic distortion (Elastic Distortion) method.

The elastic distortion method is characterized in that a smoothing value for removing noise is applied to an arbitrary uniform random displacement field obtained for each pixel position, and an alpha value for adjusting the intensity of the deformation is multiplied. .

The data expansion algorithm in the extended image generation step S500 may generate an extended image using an optical deformation algorithm, but the extended image may be generated using a light contrast deformation method of the cropping image C1.

According to the method of recognizing vehicle number using data expansion and CNN of the present invention, an expanded image (E1) transformed from an image using a data expansion algorithm is generated in the training image and input to an artificial neural network for training, so that even a small number of training images are used. You can learn to recognize the vehicle number more accurately.

In addition, according to the present invention, the vehicle number can be accurately recognized even if the pre-processing process of the captured image is insufficient by learning the CNN with the image expanded by the geometric transformation method, the elastic transformation method, and the optical transformation method.

1 is a flowchart illustrating a method of recognizing a vehicle number using a CNN and data expansion according to an embodiment of the present invention.
2 is a configuration diagram of a single shot multibox detector (SSD) of a method for recognizing a vehicle number using a CNN and data expansion according to an embodiment of the present invention.
3 is an exemplary view showing a vehicle area A1 and a license plate area A2 for vehicle number recognition using a vehicle number recognition method using data expansion and CNN according to an embodiment of the present invention.
FIG. 4 is a diagram for explaining a geometry transformation method as a data expansion algorithm in an expanded image generation step (S500) of a method for expanding data and recognizing a vehicle number using a CNN according to an exemplary embodiment of the present invention.
FIG. 5 is a diagram for explaining an elastic transformation method as a data expansion algorithm in an expanded image generation step (S500) of a method for expanding data and recognizing a vehicle number using a CNN according to an exemplary embodiment of the present invention.
6 is a diagram for explaining the structure of a convolutional neural network (CNN) in a method for recognizing a vehicle number using data expansion and CNN according to an embodiment of the present invention.

Specific structural or functional descriptions of the embodiments according to the concept of the present invention disclosed in the present specification are exemplified only for the purpose of describing the embodiments according to the concept of the present invention, and the embodiments according to the concept of the present invention are It may be implemented in various forms and is not limited to the embodiments described herein.

Since the embodiments according to the concept of the present invention can apply various changes and have various forms, the embodiments will be illustrated in the drawings and described in detail in the present specification. However, this is not intended to limit the embodiments according to the concept of the present invention to specific disclosed forms, and includes changes, equivalents, or substitutes included in the spirit and scope of the present invention.

Terms such as first or second may be used to describe various elements, but the elements should not be limited by the terms. The terms are only for the purpose of distinguishing one component from other components, for example, without departing from the scope of the rights according to the concept of the present invention, the first component may be referred to as the second component, Similarly, the second component may also be referred to as a first component.

When an element is referred to as being “connected” or “connected” to another element, it is understood that it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle. Expressions that describe the relationship between components, for example, “between” and “just between” or “neighbor to” and “directly adjacent to” should be interpreted as well.

The terms used in the present specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present specification, terms such as "include" or "have" are intended to be designated as features, numbers, steps, actions, components, parts, or a combination thereof, but one or more other features, numbers, steps, actions It is to be understood that the possibility of the presence or addition of, components, parts, or combinations thereof is not preliminarily excluded.

Unless otherwise defined, all terms used herein including technical or scientific terms have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms as defined in a commonly used dictionary should be construed as having a meaning consistent with the meaning of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in the present specification. Does not.

Hereinafter, a method and apparatus for recognizing a vehicle number using a CNN and data expansion according to a preferred embodiment of the present invention will be described with reference to the accompanying drawings.

In the present invention, the vehicle number is a character written on the license plate of the vehicle and includes letters, numbers and symbols. Data expansion means that the original data is processed in various ways to generate distorted data, thereby extending from the original data to other distorted data, and the type of data may be an image, but is not limited thereto.

1 is a flowchart illustrating a method of recognizing a vehicle number using data expansion and CNN according to a preferred embodiment of the present invention.

According to FIG. 1, the method of recognizing a vehicle number using data expansion and CNN includes an image input step of inputting a vehicle image for learning (S100); A vehicle area detection step (S200) of detecting a vehicle area (A1) in the vehicle image input in the image input step (S100) using vehicle detection learning data and a single shot multibox detector (SSD) algorithm; The vehicle area detection step (S300) of detecting a license plate area A2 located in the vehicle area A1 in the vehicle area A1 detected in S200; detection in the vehicle area detection step S200 A cropping step (S400) of extracting a cropping image (C1) by cropping an area including the license plate area (A2) detected in the license plate area detection step (S300) within one vehicle area (A1); With respect to the cropping image C1 extracted in the cropping step (S400), an expanded image generation step (S500) of generating an expanded image (E1) transforming the image using a data expansion algorithm (S500); the expanded image generation step (S500) The extended vehicle number recognition learning data generation step (S600) of performing vehicle number recognition learning by inputting the extended image (E1) generated in) into a convolutional neural network (CNN) and generating extended vehicle number recognition learning data (S600); And a vehicle number recognition step (S700) of recognizing a vehicle number from image data input from the camera using the expanded vehicle number recognition learning data.

First, the image input step (S100) is a step of inputting a vehicle image for learning, and a step of inputting a vehicle image for learning into a vehicle number recognition server including a convolution neural network (CNN) for learning vehicle number recognition. to be.

In general, a vehicle image for training is not one image, but a plurality of different vehicle images are prepared and input to perform CNN learning. In the image input step, as various images are input, CNN learning can be performed better.

In the vehicle area detection step (S200), in order to detect the vehicle area (A1) in the vehicle image input in the image input step (S100), the vehicle area is detected by using vehicle detection learning data and a single shot multibox detector (SSD) algorithm. This is the step of detecting (A1). The single shot multibox detector (SSD) algorithm may use a known algorithm, and FIG. 2 shows the structure of a single shot multibox detector (SSD) network.

In the vehicle area detection step (S200), a feature map in the form of a grid is set within the vehicle image input in the image input step (S100), and the existence of an object is determined in the set feature map to determine that the object exists. The vehicle area A1 is detected by connecting the grid pixels of the feature map to form a bounding box.

That is, the vehicle area A1 is detected by forming a bounding box in the area including the vehicle in the image.

In this case, it is characterized in that it is determined whether there is an object set in the feature map set in the vehicle area detection step (S200), but is determined using known vehicle detection learning data in order to determine that the object is a vehicle.

Accordingly, it is possible to recognize a vehicle among objects recognized in the grid of the set feature map by using the vehicle detection learning data.

The vehicle area A1 detected in the vehicle area detection step S200 appears as shown in FIG. 3.

License plate area detection step (S300), in the vehicle area (A1) detected in the vehicle area detection step (S200), in order to detect the license plate area (A2) located in the vehicle area (A1), license plate detection learning data and SSD This is a step of detecting the license plate area A2 using the (Single shot multibox detector) algorithm.

In the license plate area detection step (S300), a feature map in the form of a grid is set within the vehicle area A1 detected in the vehicle area detection step (S200), and a grid of a feature map including a vehicle license plate image in the set feature map. A bounding box is formed by connecting pixels to detect the license plate area A2.

That is, in order to detect the license plate area A2, a bounding box is formed in the area including the vehicle license plate to detect the license plate area A2.

In this regard, in order to detect the license plate area A2, when setting a grid-shaped feature map, a grid-shaped feature map smaller than the grid size of the feature map set in the vehicle region detection step S200 is selected. It is formed to exclude the vehicle area and detect the vehicle license plate within the vehicle area.

In this case, it is characterized in that it is determined whether an object exists in the feature map set in the license plate area detection step (S300), but is determined using vehicle license plate detection learning data in order to determine that the object is a vehicle license plate.

The vehicle license plate detection learning data means one or more images including the vehicle license plate for learning the vehicle license plate.

The license plate area A2 detected in the license plate area detection step S300 is positioned and detected in the vehicle area A1 detected in the vehicle area detection step S200, as shown in FIG. 3.

The vehicle area detection step (S200) and the license plate area detection step (S300) use a known deep learning single shot multibox detector (SSD) algorithm to detect the vehicle area A1 and the license plate area A2. Thus, the region can be detected.

In the vehicle area detection step (S200) and the license plate area detection step (S300), the use of a deep learning SSD algorithm to detect a corresponding area is performed by extracting a feature in order to detect a specific object and area in an existing image. , After the feature is extracted, a two-step process of detecting a corresponding region including the extracted feature is performed.In the vehicle region detection step (S200) and the license plate region detection step (S300), a deep learning method of single shot detection (SSD) By using the algorithm, feature extraction and region detection are simultaneously processed, thereby reducing the above-described two-step process, thereby improving processing speed.

Next, the cropping step (S400) is executed. The cropping step (S400) includes cropping (cutting out) the license plate area A2 detected in the license plate area detecting step S300 within the vehicle area A1 detected in the vehicle area detecting step S200. ) This is the step of extracting the cropping image C1.

The cropping step (S400) is a step for extracting a region including the vehicle license plate and deleting the other regions. Within the vehicle region A1 detected in the vehicle region detecting step S400, the license plate region ( The area is set to include A2) and cropped to extract the cropped image C1. This is to increase the data processing speed when generating the expanded image E1 later by extracting only the cropping image C1 and removing unnecessary portions. The cropping image C1 extracted in the cropping step S400 may be set based on the license plate area A2, as shown in FIG. 5.

The extended image generation step S500 is a step of generating an extended image E1 obtained by transforming the image using a data expansion algorithm with respect to the cropping image C1 extracted in the cropping step S400.

The data expansion algorithm in the extended image generation step S500 includes a geometric distortion method, an elastic distortion method, and an optical distortion method. These three modifications can be applied by combining one or two or more methods according to need.

FIG. 4 is a diagram for explaining a geometry transformation method as a data expansion algorithm in an expanded image generation step (S500) of a method for expanding data and recognizing a vehicle number using a CNN according to an exemplary embodiment of the present invention.

Referring to FIG. 4, the geometric transformation method includes a rotation operation for rotating each pixel of the cropping image C1, and for twisting the pixels with respect to the cropping image C1 extracted in the cropping step S400. It is characterized in that the extended image E1 is generated by a transformation operation method including a shearing operation, a translation operation to move a pixel, and a scaling operation to enlarge a pixel.

The transformation operation is performed using a 3X3 transformation matrix, but the rotational image RO, which is subjected to rotational transformation, the distorted image (SH), which is subjected to warping transformation, the moving image TR, or the enlarged image (SC), which has undergone expansion transformation. It characterized in that it generates an extended image (E1) to which is applied.

In this case, the transformation matrix for rotational transformation among the transformation operations performed on each pixel of the cropping image C1 in the extended image generation step S500 may be formed in the following matrix structure, but is not limited thereto.

Among the deformation operations performed on each pixel of the cropping image C1 in the extended image generation step S500, the deformation matrix for the warp deformation may be formed in the following matrix structure, but is not limited thereto.

Among the transformation operations performed on each pixel of the cropping image C1 in the extended image generation step S500, the transformation matrix for the movement transformation may be formed in the following matrix structure, but is not limited thereto.

Among the transformation operations performed on each pixel of the cropping image C1 in the step of generating the expanded image (S500 ), the transformation matrix for the expanded transformation may be formed in the following matrix structure, but is not limited thereto.

Meanwhile, by substituting each of the above-described transformation matrices into [Equation 1], a transformation operation may be performed on each pixel of the cropped image C1.

[Equation 1]

R is a transformation matrix for rotation transformation, Sh is a transformation matrix for shearing transformation, T is a transformation matrix for translation transformation, and Sc is a transformation matrix for scaling transformation.

When described with reference to [Equation 1], by setting a value of each transformation matrix, a transformation operation of each pixel of the cropping image C1 is performed to rotate, warp, move, and enlarge, and the [ The extended image E1 training data is generated through the operation of Equation 1].

In addition, as each transformation matrix value is set in [Equation 1], various extended images E1 can be variously generated as a result from one learning image through a transformation operation process.

That is, when a transformation operation is performed on the cropping image C1, rotation, distortion, movement, and enlargement transformation can be selectively performed on each pixel of the cropping image C1, and thus the expanded image E1 generated accordingly. Can be created in various ways.

In addition, the data expansion algorithm of the expanded image generation step (S500) is to generate an expanded image E1 by distorting each pixel included in the image in a random direction using an elastic distortion (elastic distortion) method. It is characterized. This is for transforming the shape of the image by obtaining an arbitrary displacement field with respect to the pixel position for each pixel of the image, and is for transforming a natural image by distorting each pixel of the image in a different direction.

More specifically, the elastic deformation (Elastic Distortion / elastic deformation) processing process will be described with reference to [Equation 2] to [Equation 5],

[Equation 2]

dx(x, y) = rand(-1, +1)

[Equation 3]

dy(x, y) = rand(-1, +1)

First, uniform random displacement fields dx and dy values for x and y are obtained using [Equation 2] and [Equation 3].

According to [Equation 2] and [Equation 3], the rand function obtains uniform random displacement fields dx and dy values of arbitrary values in the range of -1 to +1 for x and y.

[Equation 4]

Smoothing(dx, dy, kernel_size, sigma)

[Equation 5]

Smoothing(dx, dy, kernel_size, sigma)

Here, sigma means a standard deviation of a Gaussian kernel parameter, and a kernel size (kernel_size) is equal to "sigma X 6 + 1".

Using the above [Equation 4] and [Equation 5], smoothing (sommthing/ Smoothing).

In this case, as the sigma value increases, a smoothing value for removing noise in a wide area is obtained, and as the sigma value is small, a smoothing value for removing noise in a narrow area is obtained.

Next, the displacement field applied smoothing is multiplied by the alpha value. The reason for multiplying the alpha value is to control the intensity of the degree of deformation.

Multiplying the displacement field to which the smoothing is applied by an alpha value is used to adjust the intensity of the degree of deformation.The alpha value means the intensity value of the displacement, and the larger the alpha value, the higher the deformation strength. will be.

Finally, when the values of the original image positions (x, y) are obtained and mapped to the displacement fields dx and dy positions finally obtained by multiplying the alpha value, an extended image E1 to which the elastic deformation has been applied can be generated.

When generating an extended image E1 by distorting each pixel included in the image in a random direction using an elastic deformation (elastic deformation) data expansion algorithm in the extended image generating step (S500) , Rotation transformation by rotation operation, warping transformation by shearing operation, movement transformation by translation operation, and expansion transformation by scaling operation are linear transformations. Although it is deformed, since elastic distortion (Elastic Distortion) transforms each pixel included in the image in an arbitrary direction, it is possible to transform a natural image rather than a linear transformation.

Finally, the data expansion algorithm of the extended image generation step (S500) is a noise operation modification that adds random noise to the color value or brightness value of each pixel included in the image in a photometric distortion method. And, the extended image E1 is generated by using a brightness contrast operation modification that transforms a light contrast value. This is to create an extended image by transforming the image taken of the vehicle to be similar to the one distorted by the influence of the surrounding environment such as lighting and reflection, and to learn it so that the vehicle number recognition can be performed even in a situation where the image quality is poor. It is for sake.

In more detail, in the optical transformation method, noise calculation transformation adds a noise value to the color value of each image pixel, but the noise value is calculated by calculating Gaussian distribution noise according to the following [Equation 6]. Can be added to the original image.

[Equation 6]

img_nosiy = img_src + random_noise

Here, img_src is the color value of the original image, img_nosiy is the color value of the image to which the noise value is added, and random_noise is a Gaussian distribution noise value having a normal distribution with an average value of “0” and a standard deviation value of “1”. .

An image having a different color value can be generated through noise calculation transformation, and this can be learned to enable accurate recognition even when the color is distorted according to the shooting environment when recognizing the vehicle number.

In the optical transformation method, the operation transformation against brightness may be performed according to the following [Equation 7] for modifying the brightness value of the pixel of each image.

[Equation 7]

f(x) = (x-Vmin) * (max-min) / (Vmax-Vmin) + min

Here, f(x) is the result of transforming the brightness of the image, x is the brightness of the image, max is the maximum brightness of the image, min is the minimum brightness of the image, and Vmax is the brightness variation of the image. Is the maximum value, and Vmin is the minimum value of the brightness variation of the image.

An image with a different brightness value can be generated by changing the brightness contrast calculation, and this allows learning to be accurately recognized even when the shooting conditions are poor, such as when the lighting is bright or dark depending on the shooting environment when recognizing the vehicle number. I can.

The above-described optical transformation method generates a large number of extended images obtained by optically variously transforming a learning image, so that an artificial neural network or CNN can be trained more quickly with a large number of various learning images.

The data expansion algorithm that can be applied to the extended image generation step (S500) applies only one of the above-described geometric transformation method, elastic distortion (elastic distortion) method, and optical distortion method data expansion algorithm. Or, it is obvious to a person skilled in the art that two algorithms or three algorithms can be applied at the same time, and thus not described separately.

In the training data generation step S600, a feature pixel including a feature element is extracted from each pixel of the extended image E1 generated in the extended image generation step S500, and the extracted feature pixel is mapped to generate training data. Step.

The training data generation step (S600) includes a convolution layer and a pooling layer to extract features for each pixel of the expanded image E1 generated in the expanded image generation step (S500). In this step, feature pixels are extracted by filtering through a convolution neural network (CNN) algorithm, and training data is generated by mapping the extracted feature pixels.

Referring to FIG. 6, the training data generating step S600 may normalize the area size of one or more extended image data generated in the extended image generating step S500.

This is to facilitate filtering in order to extract features for each pixel in the training data generation step (S600).

This is to facilitate filtering in order to extract features for each pixel by using the extended image in the training data generation step (S600).

In the training data generation step (S600), filtering and mapping are performed using a convolutional neural network (CNN) algorithm.

In conclusion, in the training data generation step (S600), a convolutional neural network (CNN) is trained by using one or more expanded image data generated in the expanded image generation step (S500).

In the vehicle number recognition step (S700), the vehicle number is recognized and detected from an image captured by a camera using a convolutional neural network (CNN) learned in the training data generation step (S600).

Through the above-described process, the vehicle number recognition method using the data expansion and CNN of the present invention can more accurately recognize the vehicle number in fields such as crime prevention, traffic control, and parking control.

Meanwhile, the present invention may include a vehicle number recognition apparatus (not shown) using data expansion and CNN.

The vehicle number recognition apparatus using the data expansion and CNN includes an image input unit for inputting a vehicle image for training; A vehicle area detection unit detecting a vehicle area A1 in the input vehicle image using vehicle detection learning data and a single shot multibox detector (SSD) algorithm; A license plate area detection unit for detecting a license plate area A2 located in the vehicle area A1 in the detected vehicle area A1; A cropping unit for extracting a cropped image C1 by cropping an area including the detected license plate area A2; An extended image generator for generating an extended image E1 in which the image is transformed using a data extension algorithm with respect to the extracted cropping image C1; An extended vehicle number recognition learning data generator configured to perform vehicle number recognition learning by inputting the generated extended image E1 into a convolutional neural network (CNN) and generating extended vehicle number recognition learning data; A vehicle number recognition unit for recognizing a vehicle number from image data input from a camera using the expanded vehicle number recognition learning data, wherein the data expansion algorithm of the extended image generation unit generates an extended image using an optical transformation method, , An extended image E1 may be generated by adding a noise value to the color value of the cropping image C1 using the following equation.
[Equation] img_nosiy = img_src + random_noise
(Here, img_nosiy is an image to which noise values have been added, img_src is an original image, and random_noise is a noise value having a normal distribution with an average value of “0” and a standard deviation value of “1”)

The image input unit, the vehicle area detection unit, the license plate area detection unit, the cropping unit, the extended image generation unit, the learning data generation unit, and the vehicle number recognition unit of the vehicle number recognition device using data expansion and CNN are based on the above-described vehicle number recognition device and its operation principle Since they are the same, detailed descriptions are omitted.

According to the method and apparatus for recognizing vehicle number using data expansion and CNN according to the present invention, a small number of learning is performed by generating an expanded image (E1) transformed from an image using a data expansion algorithm in a limited training image and inputting it into an artificial neural network for training. You can learn to recognize the vehicle number more accurately with images.

In addition, the present invention provides an effect of accurately recognizing the vehicle number even if the pre-processing process of the captured image is insufficient by learning the CNN with the image expanded by the geometrical transformation method, the elastic transformation method, and the optical transformation method.

In the above, the present invention has been described based on what is shown in the drawings, but this is only illustrative, and various substitutions, modifications and changes are possible within the scope of the technical spirit of the present invention, so limited to the above-described embodiments and drawings. It does not become.

S100: Image input step S500: Extended image generation step
S200: vehicle area detection step S600: learning data generation step
S300: License plate area detection step S700: Vehicle number recognition step
S400: cropping step

Claims (6)

In the vehicle number recognition method using data expansion and CNN,
An image input step of inputting an image of a vehicle for learning (S100);
A vehicle area detection step (S200) of detecting a vehicle area (A1) in the vehicle image input in the image input step (S100) using vehicle detection learning data and a single shot multibox detector (SSD) algorithm;
A license plate area detection step (S300) of detecting a license plate area (A2) located in the vehicle area (A1) in the vehicle area (A1) detected in the vehicle area detection step (S200);
In the vehicle area A1 detected in the vehicle area detection step S200, a cropping image C1 is obtained by cropping the area including the license plate area A2 detected in the license plate area detection step S300. Cropping step of extracting (S400);
An expanded image generation step (S500) of generating an expanded image (E1) obtained by transforming the image using a data expansion algorithm with respect to the cropping image (C1) extracted in the cropping step (S400);
Extended vehicle number recognition learning data for performing vehicle number recognition learning by inputting the extended image E1 generated in the extended image generation step (S500) into a convolutional neural network (CNN) and generating extended vehicle number recognition learning data Generating step (S600);
A vehicle number recognition step (S700) of recognizing a vehicle number from image data input from a camera using the expanded vehicle number recognition learning data (S700); and,
The data expansion algorithm of the expanded image generation step (S500) generates an expanded image using an optical transformation method,
A method for recognizing a vehicle number using data expansion and CNN, comprising generating an extended image E1 by adding a noise value to the color value of the cropping image C1 using the following equation.
[Equation] img_nosiy = img_src + random_noise
(Here, img_nosiy is an image to which noise values have been added, img_src is an original image, and random_noise is a noise value having a normal distribution with an average value of “0” and a standard deviation value of “1”)
The method of claim 1,
The data expansion algorithm of the expanded image generation step (S500) includes a rotation operation to rotate the image, a warping operation to distort the image, and moving the image with respect to the cropping image C1 extracted in the cropping step (S400). A geometric transformation method that generates an extended image E1 by a transformation operation method including a movement operation and an enlargement operation that enlarges the image, and the extended image E1 by distorting each pixel included in the image in a random direction. A method for recognizing a vehicle number using data expansion and CNN, further comprising a generating elastic distortion method.
The method of claim 2,
The elastic distortion method is characterized in that a smoothing value for removing noise is applied to an arbitrary uniform random displacement field obtained for each pixel position, and an alpha value for adjusting the intensity of the deformation is multiplied. Vehicle number recognition method using data expansion and CNN.
delete The method of claim 1,
The data expansion algorithm of the expanded image generation step (S500) generates an expanded image using an optical transformation algorithm, and generates an expanded image using a light contrast transformation method of the cropping image C1. Vehicle number recognition method using extension and CNN.
In the vehicle number recognition device using data expansion and CNN,
An image input unit for inputting an image of a vehicle for learning;
A vehicle area detection unit detecting a vehicle area A1 in the input vehicle image using vehicle detection learning data and a single shot multibox detector (SSD) algorithm;
A license plate area detection unit for detecting a license plate area A2 located in the vehicle area A1 in the detected vehicle area A1;
A cropping unit for extracting a cropped image C1 by cropping an area including the detected license plate area A2;
An extended image generator for generating an extended image E1 in which the image is transformed using a data extension algorithm with respect to the extracted cropping image C1;
An extended vehicle number recognition learning data generator configured to perform vehicle number recognition learning by inputting the generated extended image E1 into a convolutional neural network (CNN) and generating extended vehicle number recognition learning data;
Includes; a vehicle number recognition unit for recognizing a vehicle number from image data input from a camera using the expanded vehicle number recognition learning data,
The data expansion algorithm of the expanded image generator generates an expanded image using an optical transformation method,
A vehicle number recognition apparatus using data expansion and CNN, characterized in that to generate an extended image (E1) by adding a noise value to the color value of the cropping image (C1) using the following equation.
[Equation] img_nosiy = img_src + random_noise
(Here, img_nosiy is an image to which noise values have been added, img_src is an original image, and random_noise is a noise value having a normal distribution with an average value of “0” and a standard deviation value of “1”)

Images