KR101038669B1 - The automatic vehicle identification system of non-trigger type based on image processing and that of using identification method - Google Patents

Abstract

The present invention is an image-based non-detection vehicle recognition system and vehicle recognition method for receiving the image information of the vehicle driving through a camera installed in a specific place, such as a road and automatically recognizes the license plate to recognize the property of the vehicle ,

The method uses the video detection technology based on the received video information to detect the license plate in real time, so that the vehicle does not process the entire image for vehicle area detection and real time processing. Extracting only an existing region of interest (ROI) to detect a vehicle region only in the region; When the vehicle region is detected by using the moving image technique, the real-time license plate region is detected by extracting the feature points of the numbers / letters of the license plate area while scanning the vehicle image at regular intervals from the bottom to the upper direction of the detected vehicle image. process; When the vehicle region is detected using the moving image technique, the image information captured in the first process may be preprocessed from the detected vehicle image to remove noise, generate an edge image, and display a symbol mark and a radiator grille. radiator grill, front bumper, head light, low / high beam, turning light, fog lamp, side mirror, bonnet, hood, etc. Querying a feature area of each vehicle in a database in which the feature area of the classified vehicle is stored and detecting optimum unique information corresponding to each area; And judging the hierarchy of license plates based on the feature points of the numbers / characters of the license plate area of the vehicle extracted in the second step, extracting the serial numbers through the unique information of the layered license plates, and extracting the serial number images and recognizing the extracted images. Normalize the extracted image by size, rotation, and compare the normalized serial number image with the serial number stored in the database, and recognize the letters and numbers other than the serial number in the license plate of the application number, the competent authority, and the classification code. And a fourth step of recognizing the reliability of the license plate and recognizing the vehicle license plate by detecting letters / numbers of the detected license plate.

Description

The automatic vehicle identification system of non-trigger type based on image processing and that of using identification method}

The present invention relates to a driving vehicle automatic recognition system and method for automatically recognizing a license plate of a driving vehicle. More particularly, the license plate recognition can be performed in real time by using a vehicle detection technology based on a moving image. In addition to the unique information, the reliability of license plate recognition can be greatly improved, and an image-based non-detection vehicle recognition system and a vehicle recognition method that can easily recognize a license plate of a driving vehicle for multiple lanes from a single camera can be used. It is about.

In general, a system for automatically recognizing a property of a vehicle by automatically recognizing a license plate of a driving vehicle is referred to as an automatic vehicle identification (AVI) system.

Vehicle image information recognized using electronics, optics, computers, and information technologies is used as the basic information of the Intelligent Transport System (ITS).

Most of the conventional AVI systems refer to a system having a module for continuously detecting whether the vehicle is in or out in a trigger method for embedding a loop coil. These systems are largely divided into three modules. First, a module for taking a picture by receiving a signal from the detection device when the entry and exit of the vehicle is detected, second, a module for recognizing the license plate of the vehicle from the captured image, third, a module for transmitting the information of the recognized vehicle to the traffic information center Separated by.

The recognized license plate information of the vehicle is transmitted to a traffic information center using a short-range dedicated communication network, a wireless communication network, and a satellite communication network. The traffic information center analyzes the stored data and provides analysis information to related organizations. Such an AVI system is classified into a detection type and a non-detection type depending on whether or not a detection device is embedded in a lane.

The detection system using the roof coil shown in FIG. 1 or the non-detection system using the laser beam as shown in FIG. 2 is complicated to install and transfer the system, respectively. In case of moving the system, it is complicated to install and relocate the system, which requires additional road construction, which requires re-buying.In addition, the installation cost is high because roof coil, connecting line, laser beam generation and inspection equipment are required. It is difficult to expand the target lane for recognizing the license plate of the vehicle, and if the target lane is to be extended, an installation device and an operation module need to be added to each lane, thereby increasing installation and operating costs, and the laser beam type is installed on the side of the road. There is a problem that it is impossible to recognize the license plate from multiple lanes.

In addition, the image segmentation method based on the histogram's threshold, outline-based, and region-based is used to recognize the license plate, and the image segmentation method based on the threshold value of the histogram can be divided into global binarization and local binarization. First, the global binarization method divides an input image into a class larger than a single threshold (reference value) and a small class, and applies a uniform value (single threshold) to the entire input image. As a result, when the license plate is physically damaged or dirty, accurate image recognition results may not be obtained.

The global binarization method can be ideally used only when the histogram of the gradation is clearly divided into a background and a text.

Next, the local binarization method divides the input image into blocks of m × n and binarizes the threshold value for each block. This local binarization method can be robust to the overall lighting effect or physical damage to the license plate, but one letter is divided into several blocks, thereby losing the association between blocks. That is, since one letter area has different thresholds, different binarization results are obtained between blocks. This causes a decrease in the recognition rate, which causes part of the image to be cut off or stuck to noise.

As described above, the local binarization method is a method for recognizing an image of a license plate photographed through a camera, as shown in FIG. 3 to improve the disadvantage of the global binarization method. The image of the license plate is generated as an edge (edge) image consisting of the contour of the object (S10), and the generated edge image is generated as a binary edge image of the image consisting of black, white, the generated binary image A horizontal area is projected horizontally from the top to an area of 1/4 to 3/4 of the entire vertical length of the edge image to find a boundary area that is an area having a minimum number of pixels, and then the binarized edge image is centered around the found boundary area. The image is separated into upper and lower regions (S20). Then, the separated upper and lower regions are vertically and horizontally projected. Extracting a character area, which is a predetermined area included, respectively, wherein an average character width is calculated for at least one or more character areas determined to include an area, and the character width is smaller than the calculated average character width. Search for the text area having a (S30), it is recognized by merging the searched at least one or more text areas with each other (S40).

Subsequently, after setting a threshold value for each of the extracted character regions, binarization is performed for each character region based on the set threshold value (S50). If there is a partial region where characters are broken in the binarized character region, the partial region is determined. Is configured to perform binarization based on the reset threshold.

The method of recognizing the license plate image by the above procedure has a problem in that it is not possible to perform a clear and accurate image recognition due to the variation of the threshold value according to the binarization since the recognition of the license plate depends on the reference when the threshold is set.

The present invention has been made to solve the above problems, the purpose of which is to perform the vehicle area detection using the video technique that can recognize the license plate in real time by using the vehicle detection technology based on the video, the real-time license plate area of the vehicle The present invention provides an image-based non-detection vehicle recognition system and a vehicle recognition method for detecting unique vehicle information and detecting a license plate through real-time license plate area detection through license plate character / number detection.

According to a first aspect of the present invention for achieving the above object, a driving vehicle automatic recognition system for automatically recognizing a license plate of a vehicle being driven to recognize the property of the vehicle, the image pickup to recognize the license plate of the vehicle being driven A control unit for controlling overall operations for processing video, extracting license plates, detecting vehicle information, and determining license plate information according to the acquired image information, and a camera installed at a specific place such as a road to receive image information of a driving vehicle. Responding according to the control signal of the control unit and the image input unit for receiving the captured image information of the vehicle running in a position, the vehicle region detection and real-time processing of the captured image information received from the image input unit using a moving image technique Region of In (ROI) in which the vehicle exists without processing the entire image. a video processing unit which detects only a license plate by detecting only a terest) and stores a driving vehicle area in which the image information captured by the video input unit is detected in real time through the video processing unit in response to a control signal of the controller; When the vehicle region is detected by the video processing unit through the video processing unit, the image information captured by the image input unit may detect the vehicle image at a predetermined interval from below to upward from the detected vehicle image according to the control signal of the controller. License plate extraction unit for extracting the feature points of the number / character of the license plate area while scanning, and the vehicle information detected by the image input unit when the vehicle region is detected by the video technique through the video processing unit image information captured through the image input unit By performing preprocessing from A vehicle information detection unit for removing an image and generating an edge image, and a vehicle mark mark, a radiator grill, and a front bumper to provide detection of optimal information of an optimal vehicle of the vehicle information detection unit. Database of feature areas of the vehicle classified into head light, low / high beam, turning light, fog lamp, side mirror, bonnet, hood, etc. Determining the layer of the license plate based on the feature points of the numbers / characters of the license plate area of the vehicle extracted by the license plate extractor, extracting the serial number through the unique information of the layered license plate, extracting the serial number image, and recognizing the extracted image. Normalizes the extracted image through size and rotation, compares the normalized serial number image with the serial number stored in the database, and uses the application number, jurisdiction, and classification code. The license plate information determining unit recognizes letters and numbers other than the serial number in the license plate of the license plate and determines whether the license plate is reliable. The vehicle information detection unit includes each feature area of the vehicle divided for the feature area of the classified vehicle. An image-based non-detection vehicle recognition system is provided to query the database with unique feature information to detect optimal unique information corresponding to each region.

At this time, according to an additional feature of the present invention, the video processing unit is interested in the image information of the driving vehicle imaged through the image input unit in order to process the detection of the vehicle area through the video processing unit in real time without targeting the entire image Region of Interest (ROI) is extracted and the license plate is detected only therein, and image information obtained from the camera is input in a continuous frame to detect the driving vehicle region in real time using temporal median. The frame input unit and the image information received through the frame input unit are blocks for generating a frame in which the remaining frames are accumulated except for a range value having a large error between blocks by performing global block matching in a continuous frame as an initial background generation step. An image input through a matching unit and an image input unit of the camera An image classification unit for dividing the entire foreground and the background block, a block setting unit for dividing the image input from the camera into blocks of a predetermined size, and performing each block matching with the background image, and the background image in the block setting unit If the value is larger than the tolerance, the area is determined to be the foreground area. If the value is smaller than the tolerance, the area is determined to be the background area and the background image is updated in real time using a temporal median filter. And an image judging unit for displaying the background image updated by the image updating unit in a previous frame in a predetermined pixel by temporal median and determining whether a high frequency value among the values is used as the background image. The license plate extracting unit is configured to display an image of the vehicle detected by the video processing unit below. When the specific area of the license plate is extracted by scanning at regular intervals from the side to the upper side, it is grouped by referring to the characteristics of the numbers / letters in the extracted license plate, the license plate area is selected, and the selected candidate areas are extracted as blobs. In order to extract the license plate area from the number / character extractor and the number / character extractor, the brightness variation between the character and the background is large and remarkable, and the number of changes is different. A contrast change recognition unit for estimating that there is a license plate area using characteristics of the license plate that is larger than the area; and if the license plate area is estimated by the contrast change recognition unit, the estimated license plate candidate area is based on a local binarization method. A binarization performing unit performing a local adaptive threshold, and the contrast change factor If it is estimated whether or not the license plate area by planting, comprising a labeling operation unit for performing a local adaptive binarization by the binarization unit and performing a labeling operation to determine the license plate area, the candidate area selected by the number / character extraction unit Before extracting the blobs, the blobs may be extracted by performing local adaptive binarization and labeling operations on the candidate regions selected by the binarization and labeling operators.

According to a second aspect of the present invention for achieving the above object, the driving to recognize the property of the vehicle by automatically receiving the image information of the vehicle running through the camera installed in a specific place, such as a road to recognize the license plate In the automatic vehicle recognition method, the entire image is detected for vehicle area detection and real-time processing using a moving image technique that enables license plate recognition in real time by using the vehicle detection technology based on the received captured image information. Extracting only a region of interest (ROI) in which the vehicle exists without detecting the first region and detecting the vehicle region only in the region; When the vehicle region is detected by using the moving image technique, the real-time license plate region is detected by extracting the feature points of the numbers / letters of the license plate area while scanning the vehicle image at regular intervals from the bottom to the upper direction of the detected vehicle image. process; When the vehicle region is detected using the moving image technique, the image information captured in the first process may be preprocessed from the detected vehicle image to remove noise, generate an edge image, and display a symbol mark and a radiator grille. radiator grill, front bumper, head light, low / high beam, turning light, fog lamp, side mirror, bonnet, hood, etc. Querying a feature area of each vehicle in a database in which the feature area of the classified vehicle is stored and detecting optimum unique information corresponding to each area; And judging the hierarchy of license plates based on the feature points of the numbers / characters of the license plate area of the vehicle extracted in the second step, extracting the serial numbers through the unique information of the layered license plates, and extracting the serial number images and recognizing the extracted images. Normalize the extracted image by size, rotation, and compare the normalized serial number image with the serial number stored in the database, and recognize the letters and numbers other than the serial number in the license plate of the application number, the competent authority, and the classification code. And a fourth process of recognizing a license plate of the license plate by recognizing the reliability of the license plate and recognizing the license plate of the license plate.

In this case, according to an additional feature of the present invention, the video technique of the first process extracts only a region of interest (ROI) in which the vehicle exists without processing the entire image of the image of the driving vehicle. A first step of real-time processing the license plate only therein, wherein the image information acquired by the camera is input in a continuous frame to detect the driving vehicle region in real time using a temporal median; A second step of generating an initial background by accumulating the remaining frames to a predetermined frame except for a range value having a large error, by checking whether a block has a tolerance between movements using global block matching in successive frames; A third step of dividing the entire foreground and the background block from the image input through the camera, dividing the input image into blocks having a predetermined size, and performing block matching with each of the blocks; And a fourth step of extracting an image from which the noise is removed by performing a morphology and labeling operation after being obtained by the difference image from the generated background image of the vehicle region.

In addition, according to an additional feature of the present invention, the process of detecting the real-time license plate area of the second process is the number or letter portion of the license plate while scanning the vehicle image horizontally at regular intervals from the detected vehicle image to the bottom to the upper direction A first step of extracting feature points that may appear mainly in the license plate area by using a light-dark value having a continuous light change vector from negative to positive or positive to negative; The extracted feature points are grouped with reference to the properties that can appear in the numbers / characters in the license plate and are selected as candidate regions of the license plate, or the second feature of grouping the feature points to estimate that there are license plate areas where the change in contrast occurs symmetrically. step; A third step of extracting blobs of interest by performing local adaptive binning and labeling operations by further utilizing characteristics of the license plate based on a local binarization method for the selected candidate regions; And a fourth step of finally determining whether or not the license plate area is referred to the extracted blobs with reference to the properties that may appear in the license plate area.

At this time, according to the additional feature of the present invention, classification symbols, jurisdiction using relative coordinates based on the serial number area based on the characteristics of the license plate standard for each year by using the unique features of the license plate according to the business / non-business A first step of predicting a position of an office sign or the like, setting a candidate area, matching a database stored in the candidate area, and determining the license plate type based on the calculated value to perform the layering of the license plate; Extract the serial number image from the image based on the 4-digit coordinate of the serial number using the unique information of the layered license plate, or use the license plate layer information obtained in the license plate layering step to extract the image of the classification code, the competent authority symbol, and the area code. Extracting a real-time license plate area to be extracted; Normalization as a preprocessing step for recognizing the extracted image Normalizing the extracted image through size, rotation, etc. based on the small image to maintain the suitability of the image information in order to compare the extracted image with a pre-stored reference symbol. Third step; A fourth step of comparing the normalized image with the reference symbol and comparing the normalized image with the reference symbol stored in the dictionary database to calculate a matching value; A fifth step of recognizing a number, a character other than the serial number, in the license plate using the unique information of the license plate if the serial number is determined to be a recognition rate higher than a threshold in the license plate; And after the fifth step is performed, a symbol having a maximum similarity is calculated by comparing a matching value compared with a number and a character recognition criterion symbol using a symbol having a maximum similarity to recognize a corresponding character and a number, and whether the reliability is determined by a recognition rate. If it is determined that the reliability of the end of the extraction of the character / number region, and if it is determined that there is no trust, the sixth step of proceeding to the normalization step of the extracted image; preferably comprises.

The image-based non-detection vehicle recognition system and vehicle recognition method according to the present invention require a low cost of installation because the detection device of the driving vehicle is unnecessary, easy to maintain, and the detection device needs to be extended in multiple lanes. Since there is no need to install additionally, the detection device can be installed on the side of the road, so it is easy to freely select a place for traffic information, and improve the reliability of high speed vehicle detection and license plate recognition.

In addition, by using video-based vehicle detection technology, license plate recognition is possible in real time, and by utilizing the unique information of the vehicle, it is possible to greatly improve the reliability of license plate recognition. The license plate recognition of the vehicle is easy.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

4 illustrates an embodiment of an image-based non-detection multi-road AVI system for vehicle license plate recognition according to the present invention, and FIG. 5 is a block diagram illustrating a configuration of an image-based non-detection vehicle recognition system according to the present invention. 6 is an internal configuration diagram of a video processing unit of an image-based non-detectable vehicle recognition system according to FIG. 5, and FIG. 7 is an internal configuration diagram of a license plate extraction unit of an image-based non-detection vehicle recognition system according to FIG. 5. 8 is an internal configuration diagram of a vehicle information detection unit of the image-based non-detectable vehicle recognition system according to FIG. 5, and FIG. 9 is a flowchart illustrating an entire system for image-based non-detection vehicle recognition according to the present invention. 9 is a flowchart for detecting a vehicle area using the video technique according to FIG. 9, and FIG. 11 is a flowchart for extracting a real-time license plate region according to FIG. 9. FIG. 12 is a flowchart for detecting vehicle specific information according to FIG. 9, and FIG. 13 illustrates an example of unique vehicle specific information according to FIG. 12, and FIG. 14 shows license plate letters / numbers according to FIG. 9. A flow chart for detection and recognition is shown.

Referring to the configuration and operation of the image-based non-detection vehicle recognition system and vehicle recognition method according to the present invention having the above configuration as follows.

Referring to the configuration of FIG. 5, reference numeral 10 denotes a controller, which controls the overall operation of the vehicle recognition system to recognize a license plate of a driving vehicle, and processes video and license plate extraction according to the captured image information. In addition, the vehicle 20 detects vehicle information and controls the overall operation for determining the license plate information, and 20 represents an image input unit that receives the captured image information of a driving vehicle located inside a camera installed at a specific position of a road.

Reference numeral 30 denotes a moving image processor, which reacts according to a control signal of the controller 10, and uses the moving image technique to capture the captured image information received from the image input unit 20 for the vehicle area detection and real time processing. It extracts only the region of interest (ROI) in which the vehicle exists without detecting the target, and detects the license plate only in the region, and 40 responds to the control signal of the controller 10 and the image input unit 20. ) Represents a storage unit for storing the driving vehicle area detected by the video processing unit 30 in real time.

Reference numeral 50 denotes a license plate extractor. When the image information captured by the image input unit 20 detects a vehicle region by using a moving image technique through the moving image processing unit 30, the control signal of the control unit 10 Accordingly, feature points of numbers / characters in the license plate area are extracted while scanning the vehicle image at regular intervals from the vehicle image detected downward to upward, and 60 denotes the image information captured by the image input unit 20. When the vehicle region is detected using the moving image technique through 30, the vehicle information detector which removes noise by generating pre-processing from the vehicle image detected by the image input unit 20 and generates an edge image.

70 is a database, in which the vehicle information detection unit 60 detects a vehicle's symbol mark, a radiator grill, a front bumper, and a head light to provide optimal detection of unique information of the vehicle. stores characteristic areas of a vehicle classified into low / high beams, turning lights, fog lamps, side mirrors, bonnets, hoods, and the like.

At this time, the vehicle information detection unit 60 queries each of the feature areas of the divided vehicle with respect to the classified feature areas of the vehicle by querying the database 70 for vehicle specific feature information to obtain optimal unique information corresponding to each area. Will be detected.

On the other hand, reference numeral 80 denotes a license plate information determination unit, which determines the hierarchy of license plates based on the feature points of the numbers / characters of the license plate area of the vehicle extracted by the license plate extraction unit 50, and serializes them through unique information of the layered license plates. Extract the number to extract the serial number image, normalize the extracted image through size and rotation to recognize the extracted image, and compares the normalized serial number image with the serial number stored in the database 70 and uses the number and jurisdiction. The reliability of the license plate is judged by recognizing letters and numbers other than the serial number in license plates such as offices and classification codes.

Meanwhile, FIG. 6 is an internal configuration diagram of the video processing unit 30 of the image-based non-detection vehicle recognition system. Looking at the detailed configuration, image information of the driving vehicle captured by the image input unit 20 is shown. In order to process the detection of the vehicle area through the video processor 30 in real time, the ROI (Region of Interest) is extracted instead of the entire image and the license plate is detected only therein. The image information acquired by the camera through 31 is input in a continuous frame to detect the driving vehicle area in real time using a temporal median.

The image information received through the frame input unit 31 is the block matching unit 32 performs global block matching in a continuous frame in the initial background generation step, so that the remaining frames are set except for a range value having a large error between blocks. Accumulate until.

On the other hand, the image input through the image input unit 20 of the camera is to distinguish the entire foreground and the background block in the image classifier 33, the image input from the camera through the block setting unit 34 has a constant size The blocks are divided into blocks, and each block is subjected to a background image and block matching. In this case, the block setting unit 34 performs block matching with the background image, and if the value is larger than the tolerance, the area is determined to be the foreground area. If the block setting unit 34 is smaller than the tolerance, the block setting unit 34 determines the background area. The background image is updated. At this time, the image update is robustly updated the background image in real time using a temporal median filter.

In the meantime, the image determining unit 36 determines whether a high frequency value among the values used in the previous frame is displayed as a background image by temporal median.

In addition, FIG. 7 shows an internal configuration diagram of the license plate extractor 50 of the image-based non-detection vehicle recognition system according to FIG. 5, which is largely a number / character extractor 51 and a contrast change recognizer 53. ), The binarization performing unit 55 and the labeling calculating unit 57 detect the vehicle region through the video processing unit 30 using the moving image technique, and detect the image information captured from the vehicle image. By scanning the vehicle image at regular intervals, feature points of numbers / characters in the license plate area are extracted.

In more detail, the number / character extracting unit 51 scans the image of the detected vehicle at a predetermined interval from the bottom to the upper side at a predetermined interval to refer to the characteristics of the numbers / characters in the extracted license plate when the specification of the license plate area is extracted. Group and select license plate areas and extract blobs of interest for the selected candidate areas.

In this case, before extracting the blob of interest, the blob is extracted after performing local adaptive binarization and labeling operations on the candidate regions selected by the binarization performing unit 55 and the labeling operation unit 57 to be described below.

In order to extract the license plate area by the number / character extracting unit 51, the brightness change between the character and the background is large and remarkable in the vehicle license plate area by using the change of the intensity of the license plate area in the contrast change recognition unit 53. In addition, it is estimated that there is a license plate area by using the characteristics of the license plate that the number of changes is greater than that of other areas.

If it is estimated by the contrast change recognition unit 53 whether or not the license plate area is estimated, the candidate area estimated by the binarization performing unit 55 is local adaptive threshold (Local Adaptive Threshold), the method of binarizing the license plate area Performs local adaptive binarization based on local binarization methods.

In addition, if it is estimated whether the license plate change recognition unit 53 has a license plate area, the binarization performing unit 55 performs local adaptive binarization and the labeling operation unit 57 performs a labeling operation to obtain the license plate area. Confirm the area.

8 shows an internal configuration diagram of the vehicle information detection unit 60 of the image-based non-detection vehicle recognition system of the present invention, which is composed of an edge image generation unit 61 and an image filtering unit 63. The image area is detected by the moving image processor 30 using the moving image technique, and the vehicle region is detected and preprocessed from the detected vehicle image to remove noise and generate an edge image.

When the image signal captured by the camera is converted into a video signal, there is a limitation in generating contrast information of the digital image from the video signal due to the high frequency noise of the signal. In this case, the digital image is filtered through the filter of the filtering unit 63. Removes the job for smoothing, and normalizes the edge image to the database 70 through the edge image generation unit 61 and extracts a small noise as an edge by referring to structural feature information of a predefined front face of the vehicle. By doing so, the preprocessing process removes noise before creating the edge image.

Referring to FIG. 9 to FIG. 14, the image-based non-detection vehicle recognition recognition method according to the present invention will be described with reference to the configuration configured as described above.

First, as shown in FIG. 4, an image-based non-detection multi-track AVI system according to the present invention firstly recognizes license plate recognition in real time by using a vehicle detection technology based on video as shown in FIG. 9. Perform vehicle area detection (S100) using a possible video technique, detect the real-time license plate area of the vehicle (S200), detect vehicle specific information (S300), and license plate detected through the real-time license plate area detection license plate The vehicle license plate is recognized through the letter / number detection (S400). (S500)

Then, referring to the procedure of the vehicle region detection using the video technique (S100) in detail, as shown in FIG. 10, the image input unit 20 for real-time processing of the vehicle region detection using the video technique. Instead of processing the entire image of the driving vehicle captured by the target, only the region of interest (ROI) where the vehicle is present is extracted and the license plate is detected only inside this region.

Accordingly, the driving vehicle region is detected in real time using temporal median from successive frames acquired by the camera. In the initial background generation step, the global block matching is used to determine whether there is a tolerance between movements in a continuous frame. The remaining frames are accumulated up to a set frame except for a range value having a large error. In this case, temporal median method is used to generate a more robust background. The vehicle region is obtained from the generated background image by the difference image, and then a morphology S170 and a labeling operation S180 are performed and noise is removed to be finally extracted.

The process of distinguishing the foreground and the background block from the image input from the camera. The image input from the camera is divided into blocks of a certain size, and the blocks are matched with the background image (see step S110). If the value is larger than the tolerance (see step S120), the area is the foreground area (see step S130). If it is determined to be smaller than the tolerance (see step S120), it is determined as a background area and used to update the background image.

In this case, the background image update is robustly updated in real time using a temporal median filter. Temporal median uses the most frequent values of the values that appear in the previous frame in a given pixel as the background image, and the background appears much higher in the frame than in motion, so good results can be expected. have. However, in general, to reduce the amount of computation, this method is approximated using a formula that has the effect of the median.

Let B _n (x) be the background value in the nth frame at any pixel, B _{n + 1} (x) be the background value in the next frame at that pixel, and I _n (x) be n Supposing the value of the pixel in the first frame, the background value in the n + 1th frame in the pixel ｘ is obtained as shown in <Equation 1>.

(Equation 1. Background value of pixel x in n + 1th frame)

That is, if the value of the current pixel is larger than the background pixel value, the value of the background pixel is increased by one, and if it is small, the background pixel value is decreased by one. As the background image is continuously updated as described above, the current frame and the background image are compared to determine whether there is motion in each pixel, and when there are more pixels than the reference, the motion is determined in the entire frame. . The feature of this method is that the system used to create the background image minimizes resources and can be updated very quickly. That is, even if a large motion suddenly occurs in an arbitrary pixel that is kept as a background pixel, the change of the background pixel by this is fixed at 1, so that even if a sudden large motion occurs, the background pixel gradually changes, so that this temporal intermediate value is effective. will be.

When the vehicle area detection using the video technique (S100) is performed, the real-time license plate area is detected (S200), as shown in FIG. 11, which extracts the real-time license plate area from the detected vehicle image in this step. Scanning the vehicle image horizontally at regular intervals from the bottom to the top (see step S210), extracting feature points that may appear mainly in the license plate area (see step S220).

The extracted feature points are grouped with reference to the properties that may appear in the numbers / letters in the license plate and are selected as candidate regions of the license plate (see step S230).

Blobs of interest are extracted by performing Local Adaptive Threshold (see step S240) and labeling operation (see step S250) on the selected candidate regions. The extracted blobs are finally determined whether the license plate area is referred to by referring to characteristics that may appear in the license plate area.

In detail, the above process uses the change in the intensity of the license plate area to extract the license plate area in real time. In the license plate area, the characteristic of the license plate is that the range of brightness fluctuations between the text and the background is large and remarkable, and the number of changes is greater than that of other areas. That is, when reading a line in the horizontal direction (see step S210), the feature point is extracted by using a continuous contrast change vector from negative to positive or positive to negative in the numbers or letters of the license plate. (See step S220) and estimate the feature points by grouping that there are license plate areas where the change in contrast occurs symmetrically (see step S230).

Next, Local Adaptive Threshold is performed on the estimated candidate region and a labeling operation is performed. The binarization method of the license plate area is a method of binarizing by simply giving a threshold, obtaining this threshold from a histogram of gray levels, and a local binarization method of binarizing a threshold locally. In the present invention, a local adaptive binarization is performed by further utilizing the characteristics of the license plate based on the local binarization method (see step S240).

After performing the vehicle area detection using the video technique (S100), and then detecting the vehicle specific information (S300), proceeding to detect the unique information of the optimal vehicle. This will be described with reference to FIG. 12. In the vehicle specific information detection step, the noise is removed by performing preprocessing (step S310) from the detected vehicle image to generate an edge image using a canny edge detector. (Step S320)

The detected vehicle image is divided into feature regions of the vehicle by referring to structural feature information of the front part of the vehicle which is predefined in the database 70 from the generated edge image. That is, as shown in FIG. 13, the feature areas of the vehicle may include a vehicle symbol mark, a radiator grill, a front bumper, a head light, a low / high beam, and a turning light. ), Fog lamps, side mirrors, bonnets, hoods, and the like.

Finally, each feature region of the divided vehicle is queried into the database 70 in which vehicle unique characteristic information is stored in advance, and detects optimal unique information corresponding to each region.

First, before the edge image is made, a preprocessing process for removing noise and the like is performed (see step S310). May not be obtained correctly. At this time, the digital image is smoothed to remove noise. Normally, several filters are used to remove noise, such as the mean filter, the median filter, the erosion filter, and the dilation filter.

After the preprocessing process, the Canny edge extraction process is performed (see step S320). Since most edge extraction masks are sensitive to noise, even a small noise is often regarded as an edge and extracted. The edge extraction technique using the canisan mask that compensates for these drawbacks, it extracts the strong edge by making it practically insensitive to noise.

As described above, the edge image is normalized and the detected vehicle image is divided into feature regions of the vehicle by referring to structural feature information of the front part of the vehicle (see step S330). The feature regions are compared with a previously stored vehicle unique information database to detect unique information of the vehicle.

After detecting the vehicle area using the video technique, the real-time license plate area is detected and the letter / numerical area is extracted according to the detected area. The procedure will be described with reference to FIG. 14.

In the license plate number / number detection step, the license plate layering step (S410) of determining the layer of the license plate using the unique characteristics of the license plate according to the business / non-business, and the serial number using the unique information of the layered license plate Extract the serial number image to be extracted (see step S420), and normalize the extracted image through size, rotation, etc. as a preprocessing step for recognizing the extracted image (see step S430), and normalize the serial number image and the database 70. Compare the reference symbol for the serial number in the reference number (see step S440), and if the serial number shows a recognition rate higher than the threshold value, a number other than the serial number in the license plate may be used by using the unique information of the license plate. , The character is recognized (see step S450), and whether the reliability is determined based on the recognition rate (see step S460). If the extraction of the surface alphanumeric region is terminated and it is determined that there is no trust, the normalization of the extracted image is performed again.

In more detail, the process is described in detail. First, the layering of license plates is based on the characteristics of license plate specifications for each year, and the positions of classification codes, competent authority symbols, etc. are estimated using relative coordinates based on the serial number area. By setting the candidate area and matching the database stored in the candidate area, the license plate type is judged based on the calculated value (step S410).

Subsequently, a serial number image is extracted from the image based on the serial number 4-digit coordinates extracted in the serial number image extraction real-time license plate region extraction step. In addition, using the license plate layer information obtained in the license plate layering step, the image of the classification code, the competent authority code, the area code, etc. is extracted (see step S420). Normalization of the extracted image Comparison of the extracted image with the pre-stored reference symbol Normalization of the extracted image is performed. In order to maintain the suitability of the image information, normalization is performed on the basis of the small image (refer to step S430). The image normalized by the above and the standard symbol compared with the normalized symbol are pre-learned. Comparing to calculate a matching value (see step S440).

The symbol having the maximum similarity is compared with the matching value compared with the number and the character recognition standard symbol, and the symbol having the maximum similarity is calculated to recognize the corresponding character and the number (see step S450). If the result is higher than the minimum limit, recognition is terminated. If the result is lower than the minimum limit, the process from step S430 is repeated to improve the reliability of the recognition result.

The invention being thus described, it will be obvious that the same way may be varied in many ways. Such modifications are intended to be within the spirit and scope of the invention as defined by the appended claims.

Figure 1 is an embodiment of recognizing the license plate of the detection vehicle using a conventional loop coil

Figure 2 is an embodiment of license plate recognition of a non-detection vehicle using a laser beam of a conventional driving vehicle

3 is a flowchart illustrating an adaptive local binarization method for conventional license plate recognition.

4 is an embodiment of an image-based non-detection multi-lane AVI system for vehicle license plate recognition according to the present invention

5 is a block diagram showing the configuration of an image-based non-detection vehicle recognition system according to the present invention.

6 is an internal configuration diagram of a video processing unit of an image-based non-detection vehicle recognition system according to FIG. 5;

7 is an internal configuration diagram of a license plate extractor of the image-based non-detection vehicle recognition system of FIG.

FIG. 8 is a diagram illustrating an internal configuration of a vehicle information detection unit of the image-based non-detectable vehicle recognition system of FIG. 5.

9 is a flowchart of an entire system for image-based non-detection vehicle recognition according to the present invention.

FIG. 10 is a flowchart for detecting a vehicle area using the video technique according to FIG. 9.

11 is a flowchart for extracting a real-time license plate area according to FIG. 9.

12 is a flowchart for detecting vehicle specific information according to FIG. 9.

FIG. 13 is an embodiment showing an example of unique specific information of a vehicle according to FIG. 12; FIG.

14 is a flowchart for detecting and recognizing license plate letters / numbers according to FIG.

<Description of the symbols for the main parts of the drawings>

10: control unit 20: video input unit

30: video processing unit 31: frame input unit

32: block matching unit 33: image classification unit

34: block setting unit 35: video update unit

36: Image determining unit 40: Storage unit

50: license plate extraction unit 51: number / character extraction unit

53: contrast change recognition unit 55: binarization execution unit

57: labeling operation unit 60: vehicle information detection unit

61: edge image generation unit 63: image filtering unit

70: database 80: license plate information determination unit

Claims (6)

A control unit for controlling the overall operation for image processing, license plate extraction, vehicle information detection and license plate information according to the captured image information to recognize the license plate of the driving vehicle, and a road for receiving image information of the driving vehicle The video input unit, which is located inside a camera installed at a specific place such as a camera, receives the captured image information of a driving vehicle, and reacts according to a control signal of the controller, and captures the captured image information received from the image input unit. The video processing unit detects a license plate only within a region by extracting only a region of interest (ROI) in which a vehicle exists without processing the entire image for detecting a vehicle region and real-time processing using the control unit. The image information captured by the image input unit in response to the signal is the video The storage unit stores the driving vehicle region detected in real time through the re-storage unit, and the image information captured by the image input unit is detected according to the control signal of the controller when the vehicle region is detected using the moving image technique through the moving image processing unit. The license plate extraction unit extracts the feature points of the numbers / characters of the license plate area while scanning the vehicle image from the vehicle image at regular intervals from the lower side to the upper side, and the image information captured by the image input unit using the moving image processing method using the moving image technique. When the vehicle region is detected, a vehicle information detector which removes noise by generating pre-processing from the vehicle image detected by the image input unit and generates an edge image, and provides a vehicle to provide optimal vehicle information detection of the vehicle information detector. Symbol mark, radiator (radiator grill), front bumper, head light, low / high beam, turning light, fog lamp, side mirror, bonnet, hood, etc. Determining the hierarchy of license plates based on the database storing the feature areas of the vehicle classified into categories and the feature points of the numbers / letters of the license plate area of the vehicle extracted by the license plate extraction unit, and extracting the serial number through the unique information of the layered license plates. Extract the serial number image and normalize the extracted image through size and rotation to recognize the extracted image, and compare the normalized serial number image with the serial number stored in the database, and use the license plate of the application number, jurisdiction, and classification code. A license plate information determination unit for determining the reliability of the license plate by recognizing letters and numbers other than the serial number within the vehicle information detection unit, wherein the vehicle information detection unit Each feature area of the divided vehicle is configured to query the database with vehicle unique feature information to detect optimal unique information corresponding to each area, thereby automatically recognizing the license plate of the driving vehicle and recognizing the property of the vehicle. In the non-detection vehicle recognition system based on the image, The video processing unit The image information of the driving vehicle captured by the image input unit extracts a region of interest (ROI) instead of the entire image in order to process the detection of the vehicle region through the video processor in real time, and the license plate only therein. A frame input unit which detects a driving vehicle region in real time by using a temporal median which is inputted as a continuous frame and the image information obtained by the camera is detected; The image information received through the frame input unit is a block matching unit for accumulating and generating frames in which remaining frames are set except for a range value having a large error between blocks by performing global block matching in a continuous frame as an initial background generation step; An image classification unit for dividing the image input through the image input unit of the camera to the entire foreground and the background block; A block setting unit for dividing the image input from the camera into blocks having a predetermined size and performing block matching with each background block; When the block setting unit performs a block matching with the background image, if the value is larger than the tolerance, the area is determined as the foreground area, and if the value is smaller than the tolerance, the area is determined as the background area and the temporal median filter is determined. Image update unit for updating in real time by using, The image updater includes an image determining unit for displaying a background image updated by a temporal median in a previous frame in a predetermined pixel and determining whether to use a high frequency value among the values as a background image. The license plate extraction unit When the image of the vehicle detected by the moving picture processor is scanned at regular intervals from the bottom to the upper part at a predetermined interval, when the specification of the license plate area is extracted, it is grouped with reference to the characteristics of the numbers / letters in the extracted license plate, the license plate area is selected, and the selected candidate A number / character extraction unit for extracting areas of interest as a blob, In order to extract the license plate area from the number / character extracting unit, the brightness variation between the character and the background is large and remarkable, and the number of variations is greater than that of other areas by using the change of intensity of the license plate area. A contrast adjustment unit for estimating that there is a license plate area, A binarization performing unit performing local adaptive thresholding on the estimated license plate candidate area based on a local binarization method when it is estimated by the contrast change recognition unit; And a labeling operation unit configured to determine a license plate area by performing local adaptive binarization and performing a labeling operation by estimating whether the license plate area is recognized by the contrast change recognition unit. Configured to extract a blob by performing local adaptive binarization and labeling operations on the candidate areas selected by the binarization performing unit and the labeling operation unit before extracting the interest blobs about the candidate regions selected by the number / character extraction unit. Image-based non-detection vehicle recognition system, characterized in that. delete The image processing unit receives the captured image information of the driving vehicle through the image input unit located inside the camera installed at a specific place such as a road, and automatically recognizes the license plate. By using the video detection technology based on the vehicle detection technology, it is possible to recognize the license plate in real time, so that only the region of interest (ROI) where the vehicle exists is not processed for the entire image for vehicle area detection and real time processing. Extracting and detecting a vehicle region only in the region; When the vehicle area is detected using the moving picture technique, the license plate extractor scans the vehicle image at regular intervals from the bottom to the upper part of the vehicle image detected by extracting the feature points of the numbers / letters of the license plate area to extract the real-time license plate area. A second process of detecting; When the vehicle region is detected using the moving image technique, the image information captured in the first process is preprocessed by the vehicle information detector from the vehicle image detected by the image input unit to remove noise, generate an edge image, and generate the vehicle information. Detect vehicle's symbol mark, radiator grille, front bumper, headlight, low / high beam, turning lights The feature area of each vehicle is queried in the database that stores the feature areas of the vehicle classified into light, fog lamp, side mirror, bonnet, hood, etc. A third process of detecting the unique information; And based on the feature points of the numbers / characters of the license plate area of the vehicle extracted by the license plate extracting unit of the second process, the license plate information determining unit determines the layer of the license plate and extracts the serial number through the unique information of the layered license plate. Image extraction, normalize the extracted image through size and rotation to recognize the extracted image, and compare the normalized serial number image with the serial number stored in the database, the serial number in the license plate of the application number, jurisdiction, classification code, etc. In the image-based non-detection vehicle recognition method comprising: a fourth process of recognizing letters and numbers other than, and determining the reliability of the license plate to detect the letters / numbers of the detected license plate to recognize the vehicle license plate The moving image technique of the first process extracts only a region of interest (ROI) in which a vehicle exists without processing the entire image of a driving vehicle photographed through the frame input unit of the moving image processing unit, and then license plate only in this region. A first step of detecting the driving vehicle region in real time by inputting the image information acquired by the camera in a continuous frame and using temporal median; The image information received through the frame input unit is an initial background generation step, and the block matching unit checks the tolerance of movement between blocks by using global block matching of consecutive frames, and excludes the remaining frames except the range values having a large error. A second step of generating an initial background by accumulating up to a set frame; The image input unit divides the entire foreground and the background block from the image input unit through the image input unit of the camera, and divides the image input through the image input unit of the camera into blocks of a predetermined size in the block setting unit. Performing a block matching with the image; And a fourth step of extracting the image from which the noise is removed by performing a morphology and labeling operation after being obtained by the difference image from the generated background image of the vehicle area in the license plate extractor. Image-based non-detection vehicle recognition method. delete The method of claim 3, wherein The process of detecting the real-time license plate area of the second process By scanning the vehicle image horizontally from the detected vehicle image at regular intervals from the bottom to the top, using the continuous brightness change vector from the negative to the positive or the positive to the negative in the numbers or letters on the license plate. A first step of extracting feature points that may appear mainly in the license plate area; The extracted feature points are grouped with reference to the properties that can appear in the numbers / characters in the license plate and are selected as candidate regions of the license plate, or the second feature of grouping the feature points to estimate that there are license plate areas where the change in contrast occurs symmetrically. step; A third step of extracting blobs of interest by performing local adaptive binning and labeling operations by further utilizing characteristics of the license plate based on a local binarization method for the selected candidate regions; And And a fourth step of finally determining whether or not the license plate area is referred to the extracted blobs by referring to characteristics that may appear in the license plate area. The method of claim 3, wherein The fourth process is Based on the characteristics of the license plate standard for each year, using the unique characteristics of license plates according to business / non-business, the candidate area is estimated by using the relative coordinates based on the serial number area. A first step of performing a layering of license plates by determining a license plate type based on a value calculated by setting and matching databases stored in a candidate area; Extract the serial number image from the image based on the 4-digit coordinate of the serial number using the unique information of the layered license plate, or use the license plate layer information obtained in the license plate layering step to extract the image of the classification code, the competent authority symbol, and the area code. Extracting a real-time license plate area to be extracted; Normalization as a preprocessing step for recognizing the extracted image Normalizing the extracted image through size, rotation, etc. based on the small image to maintain the suitability of the image information in order to compare the extracted image with a pre-stored reference symbol. Third step; A fourth step of comparing a normalized image with the reference symbol and comparing the normalized image with a reference symbol stored in a dictionary database to calculate a matching value; A fifth step of recognizing a number, a character other than the serial number, in the license plate using the unique information of the license plate if the serial number is determined to be a recognition rate higher than a threshold in the license plate; And After the fifth step is performed, a symbol having the maximum similarity is calculated by comparing a matching value compared with a number and a character recognition criterion symbol using a symbol having the maximum similarity, and the corresponding character and the number are recognized. Determining if there is reliability and ending the extraction of the alphanumeric region, and if it is determined that there is no confidence, a sixth step of proceeding to the normalization of the extracted image again; image-based non-detection vehicle recognition, comprising: Way.

Images