WO2016163609A2 - Appareil pour amélioration d'images à faible éclairement à base de probabilité adaptative et traitement de restauration de bavure dans un système lpr, et procédé associé - Google Patents

Appareil pour amélioration d'images à faible éclairement à base de probabilité adaptative et traitement de restauration de bavure dans un système lpr, et procédé associé Download PDF

Info

Publication number
WO2016163609A2
WO2016163609A2 PCT/KR2015/010674 KR2015010674W WO2016163609A2 WO 2016163609 A2 WO2016163609 A2 WO 2016163609A2 KR 2015010674 W KR2015010674 W KR 2015010674W WO 2016163609 A2 WO2016163609 A2 WO 2016163609A2
Authority
WO
WIPO (PCT)
Prior art keywords
image
original image
smear
low
light
Prior art date
Application number
PCT/KR2015/010674
Other languages
English (en)
Korean (ko)
Other versions
WO2016163609A3 (fr
Inventor
김태경
김충성
김수경
Original Assignee
주식회사 넥스파시스템
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 주식회사 넥스파시스템 filed Critical 주식회사 넥스파시스템
Publication of WO2016163609A2 publication Critical patent/WO2016163609A2/fr
Publication of WO2016163609A3 publication Critical patent/WO2016163609A3/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/24—Aligning, centring, orientation detection or correction of the image
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/017—Detecting movement of traffic to be counted or controlled identifying vehicles
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/04—Detecting movement of traffic to be counted or controlled using optical or ultrasonic detectors

Definitions

  • the present invention relates to an adaptive probability-based low light image enhancement and smear reconstruction device and method in an LPR system. More particularly, the number recognition performance of an LPR system is corrected by correcting distortion of an image generated in a low light and high light region.
  • the present invention relates to an adaptive probability-based low light image enhancement and smear reconstruction device and a method thereof that can greatly improve the performance.
  • LPR License Plate Recognition
  • LPR system or number recognition technology was first developed in 1976 in the UK.
  • LPR systems have grown steadily and have expanded gradually in European countries, including Southeast Asia.
  • the LPR system market is growing significantly in North America. This led to strong motivation for effective crime suppression and prevention techniques, which in turn led to wider markets.
  • LPR vehicle number recognition
  • APR automated LPR
  • OCR Optical Character Recognition
  • the unmanned automation system (hereinafter referred to as LPR system) is still mainly used as a means of detecting a vehicle, but a non-embedded detector is required due to many inconveniences and maintenance difficulties of neighboring citizens due to buried construction. .
  • an ultrasonic sensor or a Doppler sensor is used to detect a vehicle or classify (classify) the type of the vehicle, which is used for expressway toll stations, and collects charges according to vehicle types differently.
  • a method of restoring and processing a smear phenomenon according to a high intensity distribution of a subject reflected from sunlight and an image quality improvement input to a camera to improve number recognition in an LPR system is developed.
  • the weather change when the vehicle is introduced in the environment where the LPR system is installed, the location of the building and the access road, the characteristics of the camera (CCD, CMOS type), the physical distortion state of the license plate, and the color color distribution of the license plate The recognition performance is determined in accordance with this.
  • weather changes, camera characteristics, and color distribution changes can deal with software-distorted information rather than hardware, and can lead to system stabilization.
  • the most easily applicable part is increasing the intensity value to an arbitrary value for the entire image, gamma correction method, histogram equalization, and the like.
  • gamma correction method there is a method of correcting the brightness of the low illumination region by simply adding a random value to the pixel value of each pixel, but information distortion occurs due to the addition value in the high illumination region.
  • Gamma correction refers to the nonlinear transformation of the intensity signal of light using a nonlinear transfer function.
  • human vision reacts nonlinearly to brightness according to Weber's law. For example, if the brightness of light is linear within a limited bit depth, such as 8 bits per channel, dark areas where the human eye responds sensitively do not feel smooth when the brightness changes, but are disconnected. Posterization occurs (distorted).
  • the characteristics of cameras generally include blooming as well as smearing caused by light.
  • CCD Charge Coupled Device
  • CMOS Complementary Metal Oxide Semiconductor
  • Blooming occurs in both CCD and CMOS sensors, caused by high brightness objects such as the sun or light sources. Blooming phenomenon refers to the formation of a round border around the light source and around the street light. This phenomenon is caused when the object (object) contains an object such as a light source or a light, and the light of the object is too bright to exceed the capacity that the image sensor can handle and the light bleeds around. will be. On the other hand, the smear phenomenon is similar to the blooming phenomenon, but it is different from the CCD sensor, and when a strong reflected light such as a light source or a light is taken, a line is vertically displayed.
  • the camera applied to the LPR system Due to the characteristics of the camera applied to the LPR system, it is composed of a CCD type, which is weak from the light of a strong light source, and in some cases, the degree of distortion may be evenly distributed in the entire image. In particular, this cause is commonly seen in images acquired by a CCD image sensor such as the sun or a car headlight such as a strong light source.
  • the present invention proposes a stabilized technology of a system having robust characteristics against number recognition performance and environmental change by constructing an improved LPR system capable of low illumination area and smear restoration processing for an image acquired from a camera in an LPR system. .
  • the present invention has been made to solve the above-mentioned problems, and is an adaptive probability based low light image that can greatly improve the number recognition performance of the LPR system by correcting the distortion of the image generated in the low light and high light areas.
  • An object of the present invention is to provide an improvement and smear restoration apparatus and a method thereof.
  • the present invention utilizes an improved cutting histogram smoothing (A_CHE) that sets the cutting ratio dynamically according to an image, thereby greatly improving the contrast of a low light image and securing a stable image based on adaptive probability.
  • A_CHE improved cutting histogram smoothing
  • the present invention solves the smear phenomenon through the post-processing process using the image processing rather than the sensor level unit, the structural complexity is not large, it can be implemented at low cost and adaptive probability-based low light image that can ensure efficient operation
  • An object of the present invention is to provide an improvement and smear restoration apparatus and a method thereof.
  • the adaptive probability-based low light image improvement and smear restoration processing apparatus is equipped with a CCD sensor and includes a license plate of a vehicle using the CCD sensor.
  • a discriminating unit classifying the image into one of a low light image, a high light image, and an unprocessed image;
  • an image processor configured to generate a corrected image using the original image, wherein the image processor includes an improved clipped histogram smoothing when the original image is classified as the low light image by the determination unit.
  • a low light image processor for generating the corrected image from the original image by using a histogram equalization method; And a high illumination image processor for generating the corrected image by removing the smear generated in the original image when the original image is classified into the high illumination image by the determination unit.
  • the target image is used for character recognition of the license plate of the vehicle, and the original image is classified into the low light image or the high light image by the determination unit, the target image is the correction image, and by the determination unit When the original image is classified as the raw image, the target image is the original image.
  • the determining factor may be the intensity of light of the original image converted to gray scale, and when the change of the intensity of light of the original image is higher than a predetermined threshold, the determination unit may determine the original image. Is classified into the raw image.
  • the determination unit classifies the original image as the low light image when the intensity of light of the original image is less than or equal to a predetermined first value. If the intensity of light of the image is higher than the first value, the original image is classified into the high illumination image.
  • the improved cutting histogram smoothing method determines an adaptive cutting ratio for the original image, and generates a cutting histogram in which an upper region of the histogram of the original image is removed according to the determined adaptive cutting ratio. At least a portion of the upper region is cut and reallocated to the cut histogram to generate the corrected image.
  • adaptive cutting ratio is Is determined by.
  • adaptive_ ⁇ _ratio is the adaptive truncation ratio
  • f (x, y) is a gray value of the original image, to be.
  • the cut portion of the upper region includes a cutting range for the low illumination distribution area and a cutting range for the high illumination distribution area.
  • the low light distribution area is expressed by And the high illuminance distribution region is It depends on.
  • GL is the low light distribution area in the upper region
  • GH is the high illuminance distribution region in the upper region
  • T is a value set arbitrarily to distinguish low and high illuminance
  • GlobalLevel is the original image. It is a global level.
  • the cutting range for the low illumination distribution area is determined according to Equation 3 below, and the cutting range for the high illumination distribution area is determined according to Equation 4 below.
  • GLL is a cutting range for the low light distribution area
  • GHH is a cutting range for the high light distribution area
  • C p is the cutting histogram
  • k low is a gray scale.
  • k high is the high illuminance distribution region at gray scale
  • k is the number of sums of gray scales.
  • the high illuminance image processor may further include: a detector configured to detect a position of the first column where the smear is generated among the columns constituting the original image; And a remover configured to remove the smear from the original image based on the detected position information of the first column.
  • the detection unit may include: an extraction unit extracting a signal distribution curve for each column constituting the original image by using the original image input to the number recognition module; And a converting unit converting the signal distribution curve into a normal distribution curve, wherein the signal distribution curve represents a sum of gray values of a plurality of pixels constituting each column constituting the original image.
  • the detection unit generates a binary pattern map by comparing the normal distribution curve with a preset threshold value, the binary pattern map has a value of 0 in an area where the normal distribution curve is smaller than the threshold value, and the normal distribution curve is In the region larger than the threshold, the binary pattern map has a value of 1, and the region in which the binary pattern map has a value of 1 corresponds to the first column of the original image.
  • the detection unit may extend a width of the first column by adding a part of a column adjacent to the first column to the first column, and the removal unit may include the binary pattern at gray values of a plurality of pixels constituting the original image.
  • the gray value of each pixel of the map may be subtracted, and the smear generated in the first column may be removed using the subtracted result.
  • the apparatus may further include a restoration unit for restoring the original image of the first column from which the smear is removed by using a predetermined interpolation method.
  • the interpolation method of the reconstruction unit may calculate a priority for each of a plurality of pixels in the patch, determine a pixel having the highest priority among the calculated priorities, and determine the pixel of the highest priority and the patch.
  • Reconstruction is performed by comparing the similarity of pixels which do not constitute the first column among a plurality of pixels in the pixel, and the patch includes at least a part of the pixels which constitute the first column.
  • the number recognition module may further include an image restoring unit generating a restored image by applying a focus degradation method to the target image, wherein the restored image is a license plate of the vehicle of the number recognition module.
  • the focus deterioration method is a combination of a high resolution image generation method and a detailing method, and the image reconstruction unit presets a target image having deterioration of focus in order to perform the high resolution image generation method.
  • Image estimation means for up-scaling according to up-scale coefficients;
  • image generating means for generating a high resolution image from which at least a portion of the focal deterioration is removed by applying bicubic interpolation to the target image and the up-scaled image.
  • the image reconstructing unit may further include image restoring means for removing at least a portion of the focal deterioration from the generated high resolution image by using a directional adaptive guided filter. can do.
  • the image pickup module equipped with a CCD sensor using the CCD sensor vehicle license plate Shooting step of shooting the original image including the;
  • the original image is classified as the high illumination image by the low light image processor and the discriminator to generate the corrected image from the original image by using an advanced clipped histogram equalization method
  • the original image is classified into the high illumination image.
  • a high illuminance image processor to generate the corrected image by removing the generated smear.
  • the target image is the corrected image.
  • the image is the original image.
  • the present invention has been made to solve the above-mentioned problems, and is an adaptive probability based low light image that can greatly improve the number recognition performance of the LPR system by correcting the distortion of the image generated in the low light and high light areas.
  • the improvement and smear restoration processing apparatus and method can be provided to a user.
  • the present invention utilizes an improved cutting histogram smoothing (A_CHE) that sets the cutting ratio dynamically according to an image, thereby greatly improving the contrast of a low light image and securing a stable image based on adaptive probability.
  • A_CHE improved cutting histogram smoothing
  • the present invention solves the smear phenomenon through the post-processing process using the image processing rather than the sensor level unit, the structural complexity is not large, it can be implemented at low cost and adaptive probability-based low light image that can ensure efficient operation
  • the improvement and smear restoration processing apparatus and method can be provided to a user.
  • FIG. 1 is an embodiment of a typical LPR system in accordance with the present invention.
  • FIG. 2 shows an example of a block diagram of an apparatus for adaptive low-light image enhancement and smear restoration according to the present invention.
  • 3 and 4 diagrammatically show histogram truncation of an image related to the present invention.
  • FIG. 5 is a schematic diagram for explaining an improved cut histogram smoothing that can be applied to the present invention.
  • FIG. 6 illustrates an example of a signal distribution curve according to each column of an image acquired by the photographing module.
  • FIG. 7 shows a normal distribution curve with the signal distribution curve of FIG. 6 as an input.
  • FIG. 8 is a flowchart of a method for adaptive low-light image enhancement and smear reconstruction according to an example of the present invention.
  • 9 to 11 and 12 to 14 are examples showing the results of processing the low light image according to the present invention.
  • 15 to 17 and 18 to 20 are examples showing the result of restoring the smear generated in the high illuminance image according to the present invention.
  • the adaptive probability based low light image improvement and smear restoration processing apparatus 100 includes a photographing module and a number recognition module.
  • the imaging module is equipped with a CCD sensor and photographs the original image including the license plate of the vehicle using the CCD sensor.
  • the number recognition module receives an original image photographed by the photographing module, and recognizes a character of the license plate of the vehicle using the original image.
  • the number recognition module includes a discrimination unit and an image processor, and the image processor includes a low light image processor and a high light image processor.
  • the discriminator classifies the original image into one of a low light image, a high light image, and an unprocessed image based on a discrimination factor related to the original image.
  • the image processor generates a corrected image using the original image.
  • the low light image processor When the original image is classified as a low light image, the low light image processor generates a corrected image from the original image by using an advanced clipped histogram equalization method.
  • the high illuminance image processor When the original image is classified as the high illuminance image, the high illuminance image processor generates a corrected image by removing the smear generated in the original image.
  • the number recognition module uses the corrected image for character recognition of the license plate of the vehicle. If the original image is classified as the raw image, the number recognition module uses the original image for character recognition of the license plate of the vehicle.
  • the general LPR system is used to collect charges according to the operating tariff system with the vehicle's entry / exit information, or is used for collecting and discharging status and information of the unspecified vehicle without the operation tariff system. Furthermore, it is suggested as an integrated direction for observing vehicle movement in geographically dispersed organizations. 1 is an embodiment of a typical LPR system in accordance with the present invention.
  • the LPR system detects a license plate area of a vehicle input by a camera and recognizes a license plate character of the vehicle using a number and character detection method, and transmits the license plate character to a local PC or a server. Can manage and supervise
  • the vehicle's number information is used for entering, leaving, and collecting charges according to the fare system, vehicle traffic analysis, regional congestion analysis, and time-of-day vehicle access analysis, thereby promoting smooth operation management and maximizing user convenience. There is a purpose.
  • the adaptive probability-based low light image enhancement and smear restoration processing apparatus 100 of the present invention includes a photographing module 10, a number recognition module 20, a discriminator 30, and an image processor 40. And an image restoring unit 70.
  • the adaptive probability-based low light image enhancement and smear reconstruction apparatus 100 having more or fewer components may be implemented.
  • the components shown in FIG. 2 are mutually connected, and it is possible for each component to be implemented separately or in combination with each other, as shown in FIG. 2. Hereinafter, each configuration will be described.
  • the photographing module 10 is installed in the LPR system to photograph the vehicle, and photographs the vehicle positioned in the preset section to generate the original image.
  • the license plate of the vehicle is photographed in the original image photographed by the photographing module 10, and the original image is transmitted to the number recognition module 20 and used to recognize the license plate of the vehicle.
  • the photographing module 10 includes a tilting device capable of tilting in the x-, y-, and z-axis directions, respectively.
  • the photographing module 10 has a configuration capable of photographing a zoom-in image or a zoom-out image of the vehicle by rotating the camera at a designated x and y coordinate.
  • the photographing module 10 may be implemented using a camera equipped with a fisheye lens. If a fisheye lens having a wide angle of view is used, an image of an omnidirectional (360 °) region may be captured based on the imaging module 10.
  • the imaging module 10 is equipped with a CCD sensor or a CMOS sensor, preferably using a CCD sensor.
  • the CCD image sensor and the CMOS image sensor have a light receiving unit which commonly receives light and converts it into an electric signal.
  • CCD-type image sensor transfers electric signal through CCD and converts to voltage at last stage.
  • CMOS image sensors convert the signal from each pixel to a voltage and pass it outside. That is, the CCD-type image sensor moves electrons generated by light to the output unit by using a gate pulse as it is, and the CMOS-type image sensor converts electrons generated by light into voltage within each pixel and then converts the electrons into light by the number of pixels. There is a difference between the output through the switch.
  • a smear phenomenon may occur due to its signal processing method.
  • the smear phenomenon refers to a phenomenon in which a single line appears vertically on the screen when a strong reflected light such as a light source or an illumination is photographed. This is often seen when using high-speed shutter, and is often seen when shooting very bright objects such as light sources.
  • CCD-type image sensor has a structure in which only one light exists in one cell, and smear phenomenon occurs when the amount of charge that can be stored in one cell flows due to reflection and interference between cells.
  • the exposure time of the CCD is achieved through the shutter of the camera body, and the exposure is controlled by directly controlling the CCD at a shutter speed higher than the synchronization speed. If the shutter of the camera body is open when the image is acquired by using the electronic shutter of the CCD, the light is continuously incident on the photodiode, and the charge is overflowed in the stored space, and if the charge of the vertical array of CCD is read out, the overcharge Because of this, bright streaks are produced, which is why smearing occurs.
  • the smear phenomenon generated may distort the photographed image and cause problems such as vehicle shape identification and number recognition of the vehicle in a system for detecting or controlling a vehicle.
  • the number recognition module 20 of the adaptive low-light image enhancement and smear decompression processing apparatus 100 of the present invention uses the determination unit 30, the image processing unit 40, the image restoration unit 70, and the like. It may include.
  • the determination unit 30 classifies the original image generated by the photographing module 10 into one of a low light image, a high light image, and an unprocessed image based on a determination factor related to the original image.
  • the discriminating factor related to the original image is the intensity of light of the original image converted to gray scale.
  • the determination unit 30 classifies the original image as an unprocessed image.
  • the change in the light intensity of the original image is lower than the threshold value, if the light intensity of the original image is less than or equal to the first predetermined value, the original image is classified as a low illumination image, and when the light intensity of the original image is higher than the first value, The original image is classified as a high illuminance image.
  • the image processor 40 improves the image of the original image classified as a low light image or a high light image.
  • the image processor 40 may further include a low light image processor 50 for an original image classified as a low light image and a high light image processor 60 for an original image classified as a high light image.
  • the low light image processor 50 When the original image is classified as a low light image, the low light image processor 50 generates a corrected image from the original image using an improved clipped histogram equalization (A_CHE) method.
  • A_CHE clipped histogram equalization
  • an adaptive cutting ratio for the original image is determined, and according to the determined adaptive cutting ratio, a cutting histogram in which the upper region of the histogram of the original image is removed is generated, At least a portion of the region is cut and reallocated to the cut histogram.
  • FIGS. 3 and 4 diagrammatically illustrate histogram truncation of an image related to the present invention.
  • 3 is a diagram showing the removal of the histogram top region of the original image according to the conventional CHE method
  • Figure 4 is a histogram top region of the original image according to the improved cutting histogram smoothing (A_CHE) method applied to the present invention It is shown graphically as being removed.
  • A_CHE improved cutting histogram smoothing
  • the upper region of the histogram is removed according to a fixed cutting ratio.
  • Patent Document 2 Korean Patent No. 10-0756318
  • the cutting ratio is fluidly determined according to the original image, and the top region removed according to the cutting ratio is cut histogram. Will be assigned to.
  • the adaptive truncation ratio adaptive_ ⁇ _ratio is determined by Equation 1 below.
  • Equation 1 f (x, y) is a gray value of the original image, to be.
  • Figure 5 is a schematic diagram for explaining the improved cutting histogram smoothing that can be applied to the present invention.
  • the upper region removed according to the adaptive cutting ratio is reassigned to the cutting histogram by clipping the low and high illumination distribution areas.
  • the cut portion of the upper region includes a cutting range for the low illumination distribution area and a cutting range for the high illumination distribution area.
  • the low light distribution area GL and the high light distribution area GH may be expressed as Equation 2 and Equation 3, respectively.
  • T is a value arbitrarily set to distinguish low and high illumination, for example, T is 128.
  • GlobalLevel means the global level of the original image.
  • the cut range GLL for the low light distribution area may be expressed as Equation 4
  • the cut range GHH for the high light distribution area may be expressed as Equation 5.
  • C p is the truncated histogram
  • k low is the low light distribution area in gray scale
  • k high is the high light distribution area in gray scale
  • k is the number of sums of gray scales.
  • the high illumination image processor 60 when a smear is generated in the original image classified as the high illumination image, the high illumination image processor 60 generates a correction image by removing the smear generated in the original image.
  • the high illuminance image processor 60 may further include a detector 62, a remover 64, a restorer 66, and the like.
  • the detector 62 may determine whether a smear has occurred in the input original image, and if it is determined that the smear has occurred, the detector 62 may detect the position of the column (first column) where the smear has occurred.
  • the detector 62 may further include an extractor and a converter.
  • the extractor extracts a signal distribution curve for each column constituting the original image using the original image.
  • the converting unit converts the signal distribution curve generated by the extracting unit into a normal distribution curve.
  • FIG. 6 illustrates an example of a signal distribution curve according to each column of an image acquired by the photographing module
  • FIG. 7 illustrates a normal distribution curve input to the signal distribution curve of FIG. 6.
  • the extractor of the detector 62 may curve the input original image into a signal distribution with respect to a column unit signal.
  • the signal distribution curve represents a sum of gray values of a plurality of pixels constituting each column constituting the original image.
  • the converter of the detector 62 may convert the signal distribution curve of the input original image into a normal distribution curve.
  • the remover 64 may remove smears generated in the original image based on the position information of the first column detected by the detector 62.
  • the reconstructor 66 may reconstruct the original image of the first column from which the smear is removed by using an interpolation method based on the priority of the patches. Specifically, the reconstructor 66 calculates a priority for each of the plurality of pixels in the patch, determines the pixel of the highest priority having the highest priority among the calculated priorities, and the pixel of the highest priority and the patch. The reconstruction may be performed by comparing the similarity of the pixels that do not constitute the first column among the plurality of pixels within the pixels.
  • the image reconstruction unit 70 generates a reconstructed image by applying a focus degradation method to the target image.
  • the image restoring unit 70 may include an image estimating means 72, an image generating means 74, an image restoring means 76, and the like.
  • the target image may be an original image or a corrected image. If the original image is classified into a low light image or a high light image by the discriminating unit 30, the corrected image is a target image.
  • the image estimating means 72 is used when a high resolution image generation method is used as the focus deterioration method.
  • the image estimating unit 72 may generate a super-resolution image by up-scaling the low resolution degraded image according to an up-scale coefficient.
  • the image estimating unit 72 may predict the focused image from the image of the target image.
  • an edge portion of a subject is blurred, and various algorithms may be used to predict actual edge information. Such algorithms are well known to those skilled in the art to which the present invention pertains, and detailed descriptions thereof will be omitted.
  • the image estimating unit 72 obtains the super resolution image SR from the low resolution image in the intermittent image having the deterioration of focus by using the algorithm, and estimates the focused image using the algorithm.
  • the image generating means 74 is used when using a high resolution image generating method as an improved focus degradation method.
  • the image generating means 74 When the super resolution image is generated from the low resolution image input by the image estimating means 74 according to an up-scale coefficient, the image generating means 74 focuses using the super resolution image and the target image.
  • a high resolution image from which at least a portion of the degradation is removed may be generated, and the high resolution image is calculated by interpolation.
  • the interpolation method may preferably improve the deterioration of the focus image by using bicubic interpolation.
  • Equation 6 The relationship between the deteriorated image and the high resolution image may be represented by Equation 6 below.
  • L is a focal deterioration input as a low resolution image
  • ⁇ H is a super resolution image to which interpolation is applied according to an up-scale factor in the low resolution image.
  • the image generating means 74 may generate a high resolution image focused in accordance with Equation 6 in the target image having deterioration of focus, and details of the generation will be omitted since it is obvious to a person skilled in the art.
  • the image restoring means 76 is used in the case of using the detailing method as an improved focus deterioration method.
  • the image restoring unit 76 receives a high resolution image that is in focus from an image having deterioration of focus, and uses a deterring method to improve sharpness.
  • the image restoring means 76 may remove a part of the focus deterioration from the target image using the generated high resolution image.
  • the removal of such focal deterioration may be made in plural. That is, the image generating means 74 calculates a high resolution image using the super resolution image estimated by the image estimating means 72, and the image restoring means 76 is used to improve the sharpness by the deterring method.
  • the image of which the image restoring means 76 partially removes the focus deterioration is input to the image estimating means 72 again to estimate the super resolution image, and the high resolution image is calculated again by the image generating means 74.
  • the image restoring means 76 uses this to remove the deterioration of focus. This repeated process may be repeated according to the set parameter value.
  • a direction adaptive guided filter may be used as an example of the deterring method that may be used in the image restoring means 76.
  • the guided filter is a local linear filter and has a property of smoothing while preserving edge components like the bilateral filter. This feature prevents the edges of the image from crushing and maintains the base layer.
  • the image quality is improved, but there may be local smoothing and edge defects, that is, artifact defects about the object or feature information.
  • a clear quality image can be obtained by using a directional adaptive guided filter.
  • the guided filter as a deterring method applicable to the present invention performs an operation as shown in Equation 7 below.
  • Equation 7 i and j denote pixel positions, w ij denotes a filter kernel, p j denotes an input image, and I denotes a linear transformation image.
  • Equation 8 The filter kernel of Equation 7 may be expressed as Equation 8 below.
  • I is a linear conversion image
  • w ij denotes a filter kernel
  • ⁇ is a dispersion
  • ⁇ is a normalization parameter
  • ⁇ k is the average of the I transform image of w k
  • k is w k Kernel center pixel location.
  • FIG. 8 is a flowchart of a method for adaptive low-light image enhancement and smear reconstruction according to an example of the present invention.
  • the photographing module 10 equipped with the CCD sensor photographs the original image including the license plate of the vehicle by using the CCD sensor, and the original image photographed by the photographing module 10 is input to the number recognition module 20. (S10).
  • the original image the vehicle is photographed, and in general, the license plate of the vehicle is located under the original image.
  • the determination unit 30 determines whether there is a change in the intensity of light in the original image (S20).
  • the step S20 after the original image is converted into a gray scale image, the feature change is observed for a region of interest ROI which is a part of the entire original image.
  • the determination unit 30 determines whether the light intensity of the original image is a low illumination component (S30).
  • the present invention can largely correct two kinds of distortion information according to the determination of step S30.
  • the first suggests an image improvement method by extending the dynamic range of low and high illumination areas, and the second method proposes a method of detecting and restoring smear phenomenon due to the charge of light in the high illumination area. .
  • the image processor 40 generates a corrected image using the original image. Specifically, when the original image is classified as a low light image, the low light image processor 50 generates a corrected image using an improved clipped histogram equalization method (S40), and the original image is a high light image. If it is classified as the high intensity image processing unit 60 removes the smear generated in the original image to generate a corrected image (S42).
  • S40 clipped histogram equalization method
  • S42 removes the smear generated in the original image to generate a corrected image
  • an improved truncated histogram smoothing method which is one of histogram smoothing methods, is used as a technique for improving image quality of the original image.
  • histogram equalization improves an image by uniformizing the distribution of contrast values by processing an image in which the distribution of contrast values is biased or uneven to one side.
  • the ultimate goal of histogram smoothing is to produce a histogram with a uniform distribution, and to uniformize the distribution of the histogram during processing.
  • the brightness can be significantly changed according to the input image, and unwanted noise can be amplified. Therefore, the contrast can be increased while maintaining the average brightness.
  • the histogram processing method is a simple way to solve the deterioration of the image quality, so there are various methods.
  • Typical examples include Bi Histogram Equalization, Recursive Mean-Separate Histogram Equalization, and Clipped Histogram Equalization.
  • the Clipped Histogram Equalization (CHE) method is the most effective and maintains the amount of information in the image without distortion of the image.
  • This method controls the maximum value of the histogram by setting an arbitrary maximum value, cutting off the top portion of the histogram above the maximum value and resetting it over the entire area of the threshold. It should be set to have a minimum range after histogram conversion, and dynamic thresholds according to image feature changes can be set by assigning thresholds to initial settings according to images. In this case, because the upper part of the histogram is reallocated for the entire area, it is noise-resistant, but in normal video, contrast improvement results in inefficiency compared to other methods.
  • the present invention does not reset the top portion of the histogram to the entire region, and divides the histogram section into several sections and evenly distributes the histogram section to the peripheral section of the histogram section by distance ratio, while maintaining a strong noise point.
  • A_CHE method of CHE we propose an improved A_CHE method of CHE.
  • the low illuminance region can be improved from the dynamic range and can be processed as an improved image that is robust to noise, and the high illuminance region is processed as a suppressed enhancement image rather than an extended region distribution.
  • step S42 is to detect and remove the smear from the original image using the image processing.
  • the input original image is analyzed statistically.
  • the extractor of the detector 62 may curve the input original image into a signal distribution with respect to a column unit signal.
  • the signal distribution curve represents a sum of gray values of a plurality of pixels constituting each column constituting the original image.
  • the converter of the detector 62 may convert the signal distribution curve of the input original image into a normal distribution curve.
  • a smear is generally generated at a specific place from the source of sunlight or passive light by a vehicle, it can be expressed as a normal distribution.
  • the original image is expressed in a normal distribution, it is determined whether a smear is present. Due to the characteristics of the smear, it is generated as a column in the image. In particular, since it has a white and bright shape, it can be determined that smear is generated in sections of the heat having a white and bright shape.
  • the sum of the gray values and the column distribution curve that is, the sum of the smears along the direction of the distribution occurring in the smear and other sections of the signal distribution curve, is found and the signal distribution In the curve, when there is a part that has a specific and significantly higher frequency than other parts in the normal distribution, it may be determined that the smear has occurred in the original image.
  • the position of the smear is determined.
  • the location of the smear may be determined by determining that the smear has occurred in the signal distribution curve of the original image having a specific and significantly higher frequency than the other parts.
  • the smear After determining that the smear area exists and determining the location, the smear is removed and a binary pattern map (alpha map) for restoring is generated.
  • the smear removal method is performed by estimating smear strength and accurate pure background strength.
  • a binary pattern map is generated by applying an average filter to signal strength in each column of the original image.
  • the binary pattern map has a value of 1 when the signal intensity on the normal distribution is greater than a preset threshold, and when the value is small, the column has a value of 0.
  • the smear positions are rearranged using the binary pattern map.
  • each pixel inverse it consists of vehicle, noise, background and smear signal.
  • an operation of estimating the intensity of the smear signal and determining and relocating the exact position is performed.
  • the smear After repositioning the smear, the smear is removed. Using the determined smear area and intensity, the smear can be removed from the entire image.
  • the original image After removing the smear, the original image is restored.
  • inpainting is applied.
  • the original image may be restored using interpolation, but is not suitable for an area, and thus the original image may be restored using a patch method having a predetermined size in the vicinity.
  • the image reconstruction unit generates a reconstructed image by applying a focus degradation method combined with a high resolution image generation method and a deterring method to the target image (S50).
  • a super-resolution image is generated by up-scaling according to an up-scale coefficient in a target image having deterioration of focus, and generating a super-resolution image.
  • High resolution images are calculated by applying bicubic interpolation to the generated super resolution images. The process can be repeated according to the value of the predetermined coefficients, preferably until the deterioration of focus is no longer improved.
  • a high resolution image obtained by removing a part of the focus deterioration can be obtained, and deterring using a direction adaptive guided filter to restore the generated high resolution image to a clear image having a visually good quality. Use the method.
  • step S50 is not an essential step, it is also possible to proceed to step S60 with the step S50 omitted.
  • the number recognition module 20 recognizes a character of the license plate of the vehicle by using the target image (S60).
  • one of three detection methods of vehicle license plate position may be executed first.
  • the first is to detect the feature region of the license plate using vertical and horizontal edge information from the captured image.
  • the second is to detect the position of the license plate by scanning data analysis.
  • the third is to search the numbers and letters directly to detect the correct license plate.
  • the character recognition Kerean consonant + number
  • the character recognition Haunulose vowel
  • the internal recognition algorithm uses numbers, letters (consonants, vowels) to reduce misunderstanding.
  • the smear is present in the CCD camera due to its characteristics and is obtained from the same position or strong light (light source) due to the reflector from the light (light source) according to the position of the camera and the subject (vehicle), resulting in a distorted image.
  • the proposed method finds the smeared column in the still image to remove smears, and the high intensity component only for the lowest region of the ROI to reduce the amount of computation.
  • the component distribution has a certain value or more, it is determined that a smear exists.
  • the smear area exists in the sections of the rows having white and bright shapes transmitted to the supersaturated object, and the gray characteristics are analyzed and detected.
  • the sum of gray values and a distribution curve of the distribution along the direction of the distribution generated in other sections, that is, the peak value of the sum of the smears are found.
  • Other areas of the curve show relatively similar or even distributions of the background and vehicle components.
  • the smear intensity intensity curve model may be expressed as Equation 9 below.
  • C S (j) is the gray value in the column j th
  • f i, j (x, y) represents the gray value for the pixel in (i, j).
  • T hs the threshold set for the smear area
  • ⁇ is the mean according to statistical analysis in the whole image
  • ⁇ is the standard deviation
  • ⁇ is the weight
  • Equation 11 the smear area vector I mask can be expressed as Equation 11 below. have.
  • the smear brightness intensity is greater than the threshold value as the smear area.
  • the smear generation area can be grabbed by an area wider than the start and end coordinates thus obtained. For example, as shown in Equation 12 below, up to 2 pixels before the start coordinate and 2 pixels after the end coordinate can be determined as the smear generation area.
  • the smear may be detected and removed by estimating the intensity of the smear in the smeared image and the background intensity of the smear removed state.
  • the average filter is applied to the signal strength in each column, and the method of reconstructing the smear area search size to align the gray values of the pixels in the column using the applied filter is used.
  • Equation 13 is an equation relating to an average filter applied to the signal strength of each column.
  • p is an intermediate position of the image data corresponding to the coordinate in the search size
  • d is a radius of the selected location area
  • I ' is a vector aligned at the selected search size.
  • Equation 13 when analyzing each pixel column, the gray values of the pixels are aligned, including the vehicle, noise, background, and smear signal.
  • the intensity of the I S (i) smear and the I b (i) background component can be accurately obtained.
  • the smear After determining the intensity and area of the smear, the smear is removed from the entire image.
  • the smear may be detected in the image including the smear, the smear position may be detected, and the smear may be removed from the image.
  • the removed smear result and the area (position) are detected to restore an image that is not distorted from the binary pattern map (binary pattern map or alpha map) generated.
  • inpainting is applied as a technique of reconstructing the lost part of the image among various restoration methods.
  • the simplest method is interpolation using surrounding values. This is simply applicable if the missing area is not large, but interpolation for the area is not suitable because of the propagation of the error. Therefore, the process is performed until all the occlusion areas on each layer are interpolated.
  • the interpolation priority is calculated, and the texture and the structure interpolation are performed from the priorities. It is determined by the product of the confidence of the center point of the patch centered on the pixel on the boundary and a value related to the structure within the patch.
  • Equation 15 P (p) denotes the priority of the pixel p to be interpolated, C (q) denotes a reliability value of the pixel in the patch, and O denotes an area to be interpolated.
  • W p means patch, Is the size of the patch, Denotes a direction unit vector of the structure in the image, and n p denotes a normal unit vector with respect to a contour at the pixel p.
  • ⁇ is a normalization constant, and the reliability of the pixel is set to 1 for pixels included in the source region S, and to 0 otherwise.
  • blending with the patch area on the target pixel is made by borrowing an area where the similarity is maximum by comparing the gray intensity with a patch within a certain range through template matching for the pixel.
  • process C recalculates the highest priority among pixels on the boundary and repeats process A to C until all target pixels are interpolated.
  • FIG. 9 to 11 and 12 to 14 are examples showing a result of processing a low light image according to the present invention
  • Figures 15 to 17 and 18 to 20 are smear generated in the high light image according to the present invention It is an example showing the result of the restoration process.
  • the vehicle image input from the 1.3M camera (PointGray) in the LPR system was used and converted to gray scale for processing.
  • the experimental environment was implemented to enable real-time processing using Visual Studio 2010 compiler in Windows 7, CPU 2.8GHz, 4G memory.
  • 9 to 11 and 12 to 14 are experimental results for comparative analysis between the conventional HE method and the proposed A_CHE method.
  • 9 is an actual input image
  • FIG. 10 is a result image by the conventional HE method
  • FIG. 11 is a result image by the proposed A_CHE method
  • 12 is an actual input image
  • FIG. 13 is a result image by the conventional HE method
  • FIG. 14 is a result image by the proposed A_CHE method.
  • the object of the present invention is that the vehicle number recognition is important in the LPR system, so it is a low light environment in which the license plate number is not visible in the input image when viewed with the naked eye.
  • the numbers are clear and sharp enough to be distinguished.
  • 15 to 17 and 18 to 20 are experimental results for analyzing the smear treatment.
  • 15 is an actual input image
  • FIG. 16 is a smear detection result
  • FIG. 17 is a smear restoration result
  • 18 is an actual input image
  • FIG. 19 is a smear detection result
  • FIG. 20 is a smear restoration result.
  • the smear region is detected after calculating the distribution of the strong region.
  • the detected area was set to be detected wider than the smear distribution, and the area of the smear area was reconstructed using the inpainting technique. As a result, in the smear area, some areas remain in the strong area, that is, the widely distributed area, but the smear decreased significantly.
  • information such as low light distribution image, color distortion, image blurring, noise, smear, etc., which are one of the existing problems in the LPR system, is set in advance by adjusting the shutter or aperture value of the camera or finding an optimized value. Operated. Because of this, it was a major issue that would be a problem in the performance of the system without considering the LPR installation place or the surrounding environment, but by utilizing the low light image improvement and smear restoration processing proposed by the present invention, the stability and number recognition performance of the system can be improved. It is expected to contribute greatly.
  • Computer-readable recording media include all kinds of recording devices that store data that can be read by a computer system. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disks, optical data storage devices, and the like, which are also implemented in the form of carrier waves (for example, transmission over the Internet). Include.
  • the computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.
  • functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers in the art to which the present invention belongs.
  • the above-described apparatus and method may not be limitedly applied to the configuration and method of the above-described embodiments, and the embodiments may be selectively combined in whole or in part in each of the embodiments so that various modifications may be made. It may be configured.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

La présente invention concerne un appareil pour l'amélioration d'images à faible éclairement à base de probabilité adaptative et un traitement de restauration de bavure susceptibles d'améliorer la performance de reconnaissance de numéros d'un système LPR par correction de la distorsion d'une image générée dans des zones à faible éclairement à éclairement fort. L'appareil selon un mode de réalisation de la présente invention comprend : un module de photographie pour photographier une image d'origine contenant une plaque minéralogique d'un véhicule à l'aide d'un capteur CCD ; et un module de reconnaissance de numéros pour recevoir l'image d'origine et reconnaître des caractères sur la plaque minéralogique du véhicule. Le module de reconnaissance de numéros comprend en outre : une partie de détermination pour classer l'image d'origine parmi une image à faible éclairement, une image à éclairement élevé, et une image non traitée sur la base d'un facteur de détermination donné ; et une partie de traitement d'image pour générer une image corrigée en utilisant l'image d'origine. Si l'image d'origine est classée en tant qu'image à faible éclairement lumineux, un procédé avancé d'uniformisation adaptative d'histogramme coupé est utilisé, et si l'image d'origine est classée en tant qu'image à éclairement élevé, une bavure générée dans l'image originale est supprimée.
PCT/KR2015/010674 2015-04-10 2015-10-08 Appareil pour amélioration d'images à faible éclairement à base de probabilité adaptative et traitement de restauration de bavure dans un système lpr, et procédé associé WO2016163609A2 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR10-2015-0050779 2015-04-10
KR1020150050779A KR101553589B1 (ko) 2015-04-10 2015-04-10 Lpr 시스템에서 적응적 확률기반의 저조도 영상개선 및 스미어 복원처리 장치와 그 방법

Publications (2)

Publication Number Publication Date
WO2016163609A2 true WO2016163609A2 (fr) 2016-10-13
WO2016163609A3 WO2016163609A3 (fr) 2017-05-18

Family

ID=54248449

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2015/010674 WO2016163609A2 (fr) 2015-04-10 2015-10-08 Appareil pour amélioration d'images à faible éclairement à base de probabilité adaptative et traitement de restauration de bavure dans un système lpr, et procédé associé

Country Status (2)

Country Link
KR (1) KR101553589B1 (fr)
WO (1) WO2016163609A2 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108280836A (zh) * 2018-01-24 2018-07-13 中科创达软件股份有限公司 一种图像处理方法及装置
CN112053310A (zh) * 2020-08-31 2020-12-08 暨南大学 一种ccd图像中空间目标的拖尾星象定心方法

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102355420B1 (ko) 2015-12-11 2022-01-26 현대자동차주식회사 영상 개선 방법 및 컴퓨터 판독 가능한 기록매체
CN108171982B (zh) * 2018-02-06 2019-05-21 江苏华正天和科技有限公司 路口过往车辆车型监控平台
CN113313788A (zh) * 2020-02-26 2021-08-27 北京小米移动软件有限公司 图像处理方法和装置、电子设备以及计算机可读存储介质
KR102533765B1 (ko) * 2022-12-13 2023-05-18 한국광기술원 영상 변환 네트워크를 이용한 영상 처리하는 전자 장치 및 영상 변환 네트워크의 학습 방법

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6360022B1 (en) * 1997-04-04 2002-03-19 Sarnoff Corporation Method and apparatus for assessing the visibility of differences between two signal sequences
KR100799333B1 (ko) * 2003-12-26 2008-01-30 재단법인 포항산업과학연구원 주차 관제용 차종 판별 장치 및 그 방법
KR100747719B1 (ko) * 2005-06-24 2007-08-08 엠텍비젼 주식회사 조도에 따른 가변적 보간 방법, 장치 및 이를 구현하기위한 프로그램이 기록된 기록매체
KR100888554B1 (ko) * 2007-04-26 2009-03-11 벽산건설 주식회사 인식 시스템
KR101246120B1 (ko) * 2012-07-26 2013-03-22 주식회사 비스타씨엔씨 전후면 번호판 영상 기반 차량번호 인식 시스템

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108280836A (zh) * 2018-01-24 2018-07-13 中科创达软件股份有限公司 一种图像处理方法及装置
CN108280836B (zh) * 2018-01-24 2020-08-28 中科创达软件股份有限公司 一种图像处理方法及装置
CN112053310A (zh) * 2020-08-31 2020-12-08 暨南大学 一种ccd图像中空间目标的拖尾星象定心方法
CN112053310B (zh) * 2020-08-31 2023-08-11 暨南大学 一种ccd图像中空间目标的拖尾星象定心方法

Also Published As

Publication number Publication date
WO2016163609A3 (fr) 2017-05-18
KR101553589B1 (ko) 2015-09-18

Similar Documents

Publication Publication Date Title
WO2016163609A2 (fr) Appareil pour amélioration d'images à faible éclairement à base de probabilité adaptative et traitement de restauration de bavure dans un système lpr, et procédé associé
US8879869B2 (en) Image defect map creation using batches of digital images
US20090079862A1 (en) Method and apparatus providing imaging auto-focus utilizing absolute blur value
WO2016122042A1 (fr) Système et procédé de détection automatique de rivière au moyen d'une combinaison d'images satellite et d'un classificateur de forêt aléatoire
KR100791391B1 (ko) 노이즈 저감 방법 및 장치
WO2014193055A1 (fr) Appareil permettant d'améliorer une image floue à l'aide d'un opérateur racinaire pouvant être commandé par l'utilisateur
CN101742123A (zh) 图像处理设备和方法
US20120127336A1 (en) Imaging apparatus, imaging method and computer program
JP6531542B2 (ja) 画像処理システム、画像処理装置、撮像装置、画像処理方法、プログラムおよび記録媒体
US6816200B1 (en) Method and apparatus for detecting camera sensor intensity saturation
WO2018070844A1 (fr) Procédé de réglage de bord flou pour modélisation de bord
JP4466015B2 (ja) 画像処理装置および画像処理プログラム
WO2021091161A1 (fr) Dispositif électronique et son procédé de commande
WO2019045521A1 (fr) Dispositif électronique et procédé de commande associé
KR101534646B1 (ko) 영상에서 안개를 제거하기 위한 영상 처리 장치 및 방법
WO2023219364A1 (fr) Dispositif électronique et procédé d'atténuation d'artéfacts de reflet dans une image capturée
CN106993111B (zh) 摄像装置、摄像方法及记录介质
JP2005079856A (ja) 画像処理装置および画像処理プログラム
US6693669B1 (en) Method for reducing image blurring
JP2009146259A (ja) 車両ナンバー撮影システム、車両ナンバー撮影方法
US8553108B2 (en) Method of compensating for turbulance when recording film sequences
WO2023277473A1 (fr) Procédé pour photographier un objet pour identifier un animal de compagnie, et dispositif électronique
WO2024111797A1 (fr) Procédé et dispositif d'affinement de sous-pixels de vecteurs de mouvement
KR102401018B1 (ko) 보정 구간내　강제 설정이 가능한 저시정 디지털 이미지 보정 장치 및 그 방법
WO2023038219A1 (fr) Dispositif électronique d'affinage d'une image et son procédé de fonctionnement

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15888594

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15888594

Country of ref document: EP

Kind code of ref document: A2