KR20220057876A - Method and system for detecting area based on deep learning - Google Patents

Abstract

Disclosed are a system and method for detecting a specific region (e.g., license plate region) in an image on the basis of deep learning to reduce processing time. According to one aspect of the present invention, an object detection method using deep learning comprises the following steps of: acquiring an image corresponding to each frame from a video stream photographed by a camera and inputting the image; using an image processing algorithm using deep learning to find a vanishing point of a road line from the input image; setting the vertical value of the vanishing point among the coordinates of the image as a horizontal line, setting the upper region and the lower region with respect to the horizontal line, and setting the upper and lower regions as regions of interest; and image-processing the region of interest through deep learning to detect a vehicle.

Description

{Method and system for detecting area based on deep learning}

The present invention relates to a system and method for detecting a specific area (eg, license plate area) in an image based on deep learning.

A representative example in which recognition of a specific area within an image is applied is license plate area recognition. The vehicle recognition system usually consists of a license plate area detection step, a zone separation step, and a character recognition step. It is a step of separating the license plate detected from the license plate area detection step from the image, and the character recognition step is a step of recognizing the characters formed on the license plate separated in the area separation step. In addition, if the vehicle recognition system does not properly detect the license plate area in the license plate area detection stage, the meaning of the system itself is lost.

As a conventional vehicle license plate extraction method, there are a method based on color information of the license plate, a method based on the morphological element of the license plate pattern, a method of detecting the edge contour of the license plate, and the like. Among them, the method of using the color information of the license plate is sensitive to weather changes or the quality of the image, and there is a disadvantage that the color of the license plate is easily omitted when the contrast with that of the vehicle is weak. The method based on the license plate border outline mainly uses the general-purpose Hach transform. This method generally requires a lot of computation and is highly dependent on the number of pixels constituting the license plate border. In addition, the image quality of the image acquired by the camera is deteriorated due to changes in the environment, such as a decrease in illuminance, and even if the number of pixels forming the boundary decreases, the license plate image is easily omitted in the extraction process.

The technical task to be achieved by the present invention is to solve the above problem, and by using deep learning, the disadvantage of an algorithm for detecting a region in an image in the existing image field is overcome to increase the accuracy of region detection, and the entire image It is to propose a system and method that can reduce the processing time compared to the case of searching.

.

According to one aspect of the present invention, as a method for detecting an area using deep learning, a binary image for an image of a predetermined detection target is input and vertical edge information including position information of each vertical boundary line from the received binary image. generating a table including information for designating a range of a candidate image region for the sensing target based on the vertical edge information, and calculating the candidate image region based on the table Including, wherein the step of receiving a binary image for the image captured by the sensing target and generating vertical edge information including position information of each vertical boundary line segment from the received binary image includes detecting the vertical boundary line segment, and There is provided a region sensing method using deep learning, characterized in that it generates vertical edge information including the detected position information of the vertical boundary line segment.

According to another aspect of the present invention, there is provided a computer program installed in a data processing apparatus and recorded on a computer-readable medium for performing the above-described method.

According to another aspect of the present invention, there is provided a vehicle area detection system using deep learning, including a processor and a memory for storing a computer program, wherein the computer program is executed by the processor. A vehicle area detection system using deep learning for causing the detection system to perform the above-described method is provided.

According to an embodiment of the present invention, by using deep learning, it is possible to overcome the disadvantages of an algorithm for detecting a region in an image in the existing image field, thereby increasing the accuracy of region detection, and reducing the processing time compared to the case of searching the entire image. There is an effect that can provide a system and method.

In order to more fully understand the drawings recited in the Detailed Description, a brief description of each drawing is provided.
1 is a view showing an apparatus for detecting an image area of a license plate according to an embodiment of the present invention.
2 is a diagram illustrating a method for detecting an image area of a license plate according to an embodiment of the present invention.
3 and 4 are views showing an example of a screen for recognizing a license plate according to the present invention.

Since the present invention can apply various transformations and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. In describing the present invention, if it is determined that a detailed description of a related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise.

In this specification, terms such as "include" or "have" are intended to designate that the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification exist, and one or more other It should be understood that this does not preclude the possibility of addition or presence of features or numbers, steps, operations, components, parts, or combinations thereof.

In addition, in the present specification, when any one component 'transmits' data to another component, the component may directly transmit the data to the other component or through at least one other component. This means that the data may be transmitted to the other component. Conversely, when one component 'directly transmits' data to another component, it means that the data is transmitted from the component to the other component without passing through the other component.

Hereinafter, the present invention will be described in detail focusing on embodiments of the present invention with reference to the accompanying drawings. Like reference numerals in each figure indicate like elements.

A system according to an embodiment of the present invention may include hardware resources and/or software necessary to implement the technical idea of the present invention, and necessarily means one physical component or one device. is not doing That is, the system may mean a logical combination of hardware and/or software provided to implement the technical idea of the present invention, and, if necessary, is installed in devices spaced apart from each other to perform each function of the present invention. It may be implemented as a set of logical configurations for implementing a technical idea. In addition, the system may refer to a set of components separately implemented for each function or role for implementing the technical idea of the present invention.

In this specification, a module may mean a functional and structural combination of hardware for carrying out the technical idea of the present invention and software for driving the hardware. For example, the module may mean a specified code and a logical unit of hardware resources for executing the specified code, and does not necessarily mean physically connected code or a single type of hardware. It can be easily deduced to an average expert in the art of the present invention.

Hereinafter, description will be made based on a vehicle area, which is a representative example of a detection target area, but it is easy for those of ordinary skill in the art to which the present invention pertains that the technical idea of the present invention can be equally applied to various objects including vehicles. you will be able to understand

1 is a view showing an apparatus for detecting an image area of a license plate according to an embodiment of the present invention. The device may be a computing system or any system including a computing system.

As shown in Figure 1, the apparatus for detecting the image area of the license plate according to the present invention is a color image acquisition unit 110, an image pre-processing unit 120, a location information generation unit 130, a reduced space table generation It may be configured to include the unit 140 , the candidate region extraction unit 150 , and the like.

The color image acquisition unit 110 may acquire a color image of the license plate through a camera.

The image preprocessor 120 may receive the received color image in frame units, convert it into a gray scale black and white image of 256 steps from 0 to 255, and convert the converted black and white image into a binary boundary image.

Specifically, it is as follows. That is, the image preprocessor 120 sets a reference value of a specific level to the brightness value of a pixel in the converted black-and-white image as a threshold, and gives a white value 255 when the brightness value of each pixel is greater than or equal to the threshold, and is less than or equal to the threshold value. A black value (0) is assigned to a pixel having a brightness value of , and converted into a binary image.

In the above process, the threshold value is determined so that the boundary of objects including the license plate of the vehicle is displayed as bright outlines as much as possible in the converted image. The method is not specified as the image pre-processing process in the present invention.

At this time, one frame of the image obtained by shooting with a camera so that an image of the license plate of the corresponding vehicle appears for at least one vehicle including the background image is converted to a gray level of 256 steps for the brightness value of each pixel One image frame is called an original image, and the brightness value of each pixel is abbreviated as a pixel value.

In addition, in the binary image, a vertical pixel value VER_P and a horizontal pixel value HOR_P are assigned to white pixels that are continuous in the vertical and horizontal directions, respectively, and NON_P values are assigned to other pixels to make a binary boundary image. For example, the vertical pixel value VER_P, the horizontal pixel value HOR_P, and NON_P may be distinguished by assigning values such as 254, 253, and 128, respectively.

The binary boundary image is defined to have the same size as the original image converted to gray level, and the pixel position of each contour is the same as the position on the coordinates of the object contour in the original image. Accordingly, the binary boundary image will be abbreviated as a boundary image.

The coordinates of the two-dimensional plane image displayed on the screen are represented by (X, Y), and with the upper left as the starting point (0,0), the right is the direction in which the X value increases and the downward direction in the direction in which the Y value increases. do.

And the number of pixels in the horizontal and vertical directions is expressed as W and H to indicate the resolution of the vehicle image as W x H. Also, the order of storing the images in the computer memory space is coordinates (0, 0, 0), (1,0), (2, 0),.... (W-1, 0), (0,1), (1,1), (2,1), ....( W-1, 1), .....(0, H-1), (1, H-1), (2, H-2), ... (W-1, H-1) It is decided to store each pixel value corresponding to .

In this case, the data structure for the video image img is expressed as img[H][W] as a two-dimensional array.

In the present invention, the license plate in the original image frame is to be defined by position coordinates and size. That is, the license plate of the vehicle is expressed as a combination of four factors (cx, cy, rx, ry) by expressing the central coordinate values of the license plate cx, cy and the radius values rx, ry on the coordinates in the X and Y directions.

The location information generator 130 may generate vertical edge information VerEdgeInfo including location information of each vertical boundary line segment with reference to the binary boundary image.

First, the location information generating unit 130 may secure W Ⅷ h storage spaces in the memory to generate the vertical edge information VerEdgeInfo. Here, h represents half the number of pixels in the vertical direction as a result of calculation of h = H / 2 .

At this time, the data structure of such vertical edge information is indicated as VerEdgeInfo[W][h] so that data is sequentially stored in the direction of the arrow, for example, from top to bottom. The order in which this data is stored is the scan direction.

The vertical edge information VerEdgeInfo consists of W entries having a storage capacity h in the memory. can

Each entry in the vertical edge information VerEdgeInfo scans the boundary image in the vertical direction for each X value in the horizontal direction, and fills out by irradiating vertical lines composed of at least three or more pixels. The order in which information is stored in each entry is to store the total number of vertical segments found in that scanline in the first storage Ei[0] of the entry Ei for the scanline (X = i, Y), and Ei[1] and The Y coordinate values of the start and end pixels of the first vertical line segment are stored in Ei[2], respectively, and the same is repeated in Ei[j+1] and Ei[j+2] for the start and end pixels of the vertical line segment v_lj. Store the Y coordinate value of In this way, each entry of VerEdgeInfo includes information on the number of constituent pixels as the position information of the effective vertical line segment belonging to the corresponding scan line and the difference between the position values of the start and end pixels.

[0059] Above, the reason for securing the size of each entry that stores information of vertical edge information VerEdgeInfo to half the height of the original image is that the maximum number of vertical line segments is only three pixels where each line segment is the minimum effective pixel standard. up to Hㆇ(3+1) line segments can exist per scan line even if each vertical line segment is separated by one pixel distance, and two storage spaces are required for another line segment. This is because (Hㆇ4)ㅧ2 storage space is sufficient.

The reduced space table generation unit 140 may generate the reduced space table VoteTable including information for designating the range of the candidate image area based on the vertical edge information VerEdgeInfo.

Hereinafter, the range of the license plate candidate image area in the boundary image space will be described.

If there is a vertical line segment of pixels that is continuous downward starting from the coordinate (i,j) in the boundary image, the vertical line segment is regarded as the left edge of the license plate and represents the range of possible remaining right edge positions as the license plate candidate image area. there is.

Based on the vertical line segment, it is possible to assume a license plate of the maximum size having Rx, Ry from the license plate of the minimum size having rx, ry, respectively, the horizontal radius and the vertical radius, and if the valid license plate candidate having the size of the range to be extracted If this is the case, it should have the remaining border outline segmentation pixels having a position value between this minimum license plate size and the maximum license plate size.

Here, the minimum license plate size and the maximum license plate size indicate a rectangular image to be extracted as a license plate candidate image area from the original image, and the minimum and maximum width of the rectangular image, and the range of the minimum height and maximum height are designated in units of pixels. will be.

The coordinates (i, j) are stored in the vertical edge information VerEdgeInfo, and the variance X of the coordinate value i is the position information of the vertical scan line to investigate the vertical line segment. If it is located on the right, the maximum X = W - (the width of the minimum number plate), and the variance Y of the coordinate value j is the effective range of the vertical line segment to be investigated in each scan line. If located at the bottom, the maximum is Y = H - (the height of the minimum number plate).

In addition, the minimum license plate size and the maximum license plate size can be represented by horizontal and vertical radii rx and ry, Rx and Ry, respectively. In fact, since the size of the license plate is determined by the standard, if you know only the vertical length from the ratio of the horizontal length to the vertical length The horizontal length is calculated. The ratio of the vertical length to the horizontal length of such a license plate is called an aspect ratio.

Therefore, if R is defined as a variable for the horizontal length of the license plate, the range becomes rx ≤ R ≤ Rx.

In the above, by limiting the range of each variable, the calculation speed can be increased.

The candidate region extractor 150 may calculate a candidate image region by using the generated reduced space table.

2 is a diagram illustrating a method for detecting an image area of a license plate according to an embodiment of the present invention.

As shown in Figure 2, the apparatus for detecting the image area of the license plate according to the present invention (hereinafter referred to as a detection device) may obtain a color image of the license plate through a camera (S910).

Next, the detection apparatus may convert the photographed color image into a binary image (S920). That is, the detection device receives the photographed color image in frame units, converts it into a grayscale black-and-white image, and converts the converted black-and-white image into a binary boundary image in which specific values are assigned to distinguishable pixels constituting vertical and horizontal lines. can

Next, the detection apparatus may generate vertical edge information including position information of each vertical boundary line segment from the converted binary boundary image (S930). That is, the detection apparatus detects a vertical boundary line segment composed of at least three or more pixels in a vertical direction and generates vertical edge information including position information of the detected vertical boundary line segment and the like.

In this case, the vertical edge information includes the X-axis coordinate value for each vertical scan line in the binary boundary image, the number of straight line segments included in the scan line, and the Y-axis coordinate value of the start and end pixels of each vertical line segment. do.

Next, the detection apparatus may generate a reduced space table including information for designating a range of a candidate image area for a license plate based on the generated vertical edge information (S940).

In this case, the reduced space table defines three-dimensional variables x, y, and r, each of which is obtained by dividing the horizontal and vertical variances X and Y of the binary boundary image and the variance for the horizontal radius R of the candidate image area by the reduction constant S, respectively. create an array table,

Calculate the x-axis value cx and the horizontal radius cr of the center coordinates of the candidate image by referring to each entry information of the vertical edge information, and progressively increase the table values within the vote range corresponding to the Y value of each pixel in the vertical line by 1 to obtain the central coordinates. Determine the location of cy.

Next, the detection apparatus may calculate a candidate image area based on the reduced space table ( S950 ). That is, the detection device determines the coordinate values of the center coordinates of the candidate image area based on the reduced space table, that is, the horizontal coordinate value and the vertical coordinate value, the size in the horizontal direction, the size in the vertical direction, and the horizontal length versus the vertical length of the license plate. Aspect ratio, which is a ratio, yields a different defined type.

As described above, in the present invention, when finding the VerEdgeInfo values in the boundary image among several steps, the minimum number of effective vertical lines can be specified up to 3, whereby the height of the license plate candidate image has a size of at least 3 pixels. It has the feature of being able to extract images of a small area. In reality, since symbols such as letters and numbers having a recognizable number of pixels in the license plate candidate image area must exist, the minimum height pixel number of the license plate image to be extracted in a situation where the method of the present invention is applied will exceed this The target image can be extracted without omission.

In addition, the present invention can extract the license plate image area only with the number of pixels above the threshold even when the quality of the image is deteriorated due to the change of weather and weather and the outline of the license plate image is not clearly taken.

In addition, since the present invention is performed in each step by referring to each pixel coordinate value of the horizontal and vertical line segments of the boundary image space and storing the result value processed by a simple operation in the memory data space, it is a complex mathematical formula compared to other statistical methods High-speed execution is possible because no calculation is required.

In addition, the present invention has a feature that by introducing abbreviation constants in the VoteTable data space, it is possible to extract a region where the license plate image is tilted or the outline is not a single line.

Meanwhile, according to an embodiment, a system according to an embodiment of the present invention may include a processor and a memory for storing a program executed by the processor. The processor may include a single-core CPU or a multi-core CPU. The memory may include high-speed random access memory and may include non-volatile memory such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to memory by the processor and other components may be controlled by a memory controller.

On the other hand, the method according to the embodiment of the present invention may be implemented in the form of a computer-readable program command and stored in a computer-readable recording medium, and the control program and the target program according to the embodiment of the present invention are also implemented in the computer. It may be stored in a readable recording medium. The computer-readable recording medium includes all types of recording devices in which data readable by a computer system is stored.

The program instructions recorded on the recording medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the software field.

Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and floppy disks, and hardware devices specially configured to store and execute program instructions, such as magneto-optical media and ROM, RAM, flash memory, and the like. In addition, the computer-readable recording medium is distributed in network-connected computer systems, and computer-readable codes can be stored and executed in a distributed manner.

Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by an apparatus for electronically processing information using an interpreter or the like, for example, a computer.

The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

The above description of the present invention is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and likewise components described as distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention. .

Claims (3)

A method for detecting an area using deep learning, comprising:
receiving a binary image of an image of a sensing target and generating vertical edge information including position information of each vertical boundary line segment from the received binary image;
generating a table including information for designating a range of a candidate image area for the sensing target based on the vertical edge information; and
Comprising the step of calculating the candidate image area based on the table,
The step of receiving a binary image for the image of the sensing target and generating vertical edge information including position information of each vertical boundary line segment from the received binary image includes:
A method for detecting a region using deep learning, comprising detecting a vertical boundary line segment and generating vertical edge information including position information of the detected vertical boundary line segment.
A computer program installed in a data processing apparatus and recorded on a computer readable medium for performing the method according to claim 1.
As an area detection system using deep learning,
processor; and a memory for storing a computer program;
When the computer program is executed by the processor, the region detection system using deep learning causes the region detection system using deep learning to perform the method according to claim 1 .

Images