WO2022000860A1 - 车牌图像增强方法、系统、设备及存储介质 - Google Patents

车牌图像增强方法、系统、设备及存储介质 Download PDF

Info

Publication number
WO2022000860A1
WO2022000860A1 PCT/CN2020/121507 CN2020121507W WO2022000860A1 WO 2022000860 A1 WO2022000860 A1 WO 2022000860A1 CN 2020121507 W CN2020121507 W CN 2020121507W WO 2022000860 A1 WO2022000860 A1 WO 2022000860A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
license plate
area
enhanced
brightness
Prior art date
Application number
PCT/CN2020/121507
Other languages
English (en)
French (fr)
Inventor
张兆阳
王佛荣
章勇
Original Assignee
苏州科达科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 苏州科达科技股份有限公司 filed Critical 苏州科达科技股份有限公司
Publication of WO2022000860A1 publication Critical patent/WO2022000860A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/63—Scene text, e.g. street names
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/625—License plates
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Definitions

  • the present invention relates to the technical field of image processing, and in particular, to a license plate image enhancement method, system, device and storage medium.
  • a strobe light is often installed next to the capture device. At the moment when the vehicle is approaching, the strobe light is turned on to fill in the light, and a frame of picture is captured, which can not only be used under all weather conditions.
  • a bright picture of the vehicle can be obtained, and the behavior or actions of the front-seat driver or passenger in the cab can be seen at a glance, which can be used to assist in judging whether there is a violation of traffic laws (such as not wearing a seat belt, answering a phone while driving, etc.) , but because the brightness of the strobe light is often higher, the license plate in some scenes is sometimes severely overexposed due to strong reflection, or because multiple vehicles pass by, there is occlusion, resulting in the phenomenon that the license plate area of a certain car is not filled with light. , it is difficult to clearly identify the license plate number in the captured picture at this time, which affects the acquisition of license plate number information. Therefore, in the process of license plate information recognition, if the collected license plate image has defects that are unclear or difficult to identify, the collected license plate image can be enhanced and restored by means of image enhancement in advance, which is beneficial to the subsequent license plate information. identify.
  • the contrast of the brightness image of the license plate is enhanced, and then the image is denoised to improve the font clarity on the license plate and remove the background noise of the license plate, so as to achieve enhancement. Purpose.
  • the license plate image enhancement method in the prior art the brightness of the entire license plate image is enhanced, which may cause serious overexposure of the license plate area, and during license plate recognition, if the license plate location is not accurate, the information in the license plate area is missing. Or when it is incomplete, it is difficult to restore the license plate number and license plate color.
  • the purpose of the present invention is to provide a license plate image enhancement method, system, equipment and storage medium, through different image enhancement methods and image fusion of the license plate area and the non-license plate area, to achieve suitable for different scenarios.
  • Image enhancement below.
  • An embodiment of the present invention provides a license plate image enhancement method, comprising the following steps:
  • the license plate expansion area image includes a license plate area and a peripheral area surrounding the license plate area;
  • the first image enhancement processing includes a first white balance processing
  • the second image enhancement processing includes a second white balance processing
  • the brightness of the image obtained by the second white balance processing is smaller than the first white balance processing. Balances the brightness of the resulting image.
  • the first white balance processing includes multiplying the pixel value of the license plate extension area image by a preset white balance gain coefficient to obtain a first enhanced image
  • the second white balance processing includes multiplying the pixel value of the license plate extension area image by a preset white balance gain coefficient and a preset brightness reduction ratio to obtain a second enhanced image.
  • the second white balance processing includes:
  • the brightness reduction ratio is a numerical value between 0 and 1;
  • the pixel R(x,y) For each pixel G(x,y) of the green channel of the first enhanced image, the pixel R(x,y) corresponding to the position of the red channel and the pixel G(x,y) and the green channel and the pixel G The position of (x, y) corresponds to the pixel value of pixel B (x, y), and adjusts the pixel value of pixel G (x, y).
  • the license plate extension area image further includes a transition area between the license plate area and the peripheral area;
  • the license plate area of the license plate enhanced image is filled with the image data of the second enhanced image, and the peripheral area of the license plate enhanced image is filled with the image data of the first enhanced image;
  • p(x, y) is the pixel value of the pixel whose coordinates are (x, y) in the transition area of the license plate enhanced image
  • p A (x, y) is the first enhanced image whose coordinates are (x, y)
  • p B (x, y) is the pixel value of the pixel with coordinates (x, y) in the second enhanced image
  • L1 is the distance between the pixel with coordinates (x, y) and the license plate area
  • L2 is the distance between the pixel whose coordinates are (x, y) and the peripheral area.
  • the input license plate expansion area image it also includes brightness adjustment of the license plate expansion area, and the brightness adjustment includes the following steps:
  • the performing first image enhancement processing on the license plate extension area image includes performing the first image enhancement processing on the brightness-adjusted license plate extension area image;
  • the performing a second image enhancement process on the license plate extension area image includes performing a second image enhancement process on the brightness-adjusted license plate extension area image.
  • the corresponding target brightness is determined.
  • determining the license plate color according to the license plate area image includes the following steps:
  • the color of the license plate is determined according to the preset mapping relationship between different colors and the ratios RG ratio and BG ratio.
  • the input license plate expansion area image includes the license plate expansion area image of the input raw format
  • Both the first image enhancement processing and the second image enhancement processing are directly performed on the license plate extension area image in the raw format.
  • the license plate image enhancement method provided by the present invention, on the one hand, by performing image enhancement on the license plate extended image, even if the license plate area is deviated, due to the supplement of the peripheral area of the license plate extended image, the complete license plate information can be obtained without As for the lack of license plate information due to the inaccurate positioning of the license plate; on the other hand, by adopting two processing methods for the license plate image enhancement, the first enhanced image and the second enhanced image are obtained respectively, and the first enhanced image and the second enhanced image are processed. Fusion, so that different enhancement methods are used in the license plate area and the peripheral area in the finally obtained license plate enhanced image, which can avoid the problem of overexposure of the license plate area caused by using the same image enhancement method for the entire image in the prior art. Image enhancement methods can be applied to more application scenarios.
  • the embodiment of the present invention also provides a license plate image enhancement system, which is characterized in that, applied to the license plate image enhancement method, the system includes:
  • an image extraction module for inputting a license plate extension area image, the license plate extension area image including a license plate area and a peripheral area surrounding the license plate area;
  • a first processing module configured to perform a first image enhancement process on the license plate extension area image
  • the second processing module is configured to perform a second image enhancement process on the image of the license plate extension area to obtain a second enhanced image, and the second image enhancement process is used to perform the image enhancement process of suppressing exposure on the image of the license plate extension area ;
  • An image fusion module configured to perform image fusion on the first enhanced image and the second enhanced image to obtain a fused license plate enhanced image, wherein the license plate enhanced image includes a license plate area and a peripheral area, and the license plate area
  • the peripheral area is filled with image data of the second enhanced image
  • the peripheral area is filled with image data of the first enhanced image.
  • the license plate image enhancement system provided by the present invention, on the one hand, by performing image enhancement on the license plate extended image, even if the license plate area is deviated, due to the supplement of the peripheral area of the license plate extended image, the complete license plate information can be obtained without As for the lack of license plate information due to the inaccurate positioning of the license plate; on the other hand, by adopting two processing methods for the license plate image enhancement, the first enhanced image and the second enhanced image are obtained respectively, and the first enhanced image and the second enhanced image are processed. Fusion, so that different enhancement methods are used in the license plate area and the peripheral area in the finally obtained license plate enhanced image, which can avoid the problem of overexposure of the license plate area caused by using the same image enhancement method for the entire image in the prior art. Image enhancement methods can be applied to more application scenarios.
  • the embodiment of the present invention also provides a license plate image enhancement device, including:
  • the processor is configured to execute the steps of the license plate image enhancement method by executing the executable instructions.
  • the processor executes the license plate image enhancement method when executing the executable instructions, thereby obtaining the beneficial effects of the license plate image enhancement method. It is guaranteed to obtain complete license plate information, and on the other hand, it can be applied to more application scenarios.
  • Embodiments of the present invention further provide a computer-readable storage medium for storing a program, which implements the steps of the license plate image enhancement method when the program is executed.
  • the stored program realizes the steps of the license plate image enhancement method when it is executed, so that the beneficial effects of the above-mentioned license plate image enhancement method can be obtained, that is, on the one hand, it can ensure Obtaining complete license plate information, on the other hand, can be applied to more application scenarios.
  • FIG. 1 is a flowchart of a license plate image enhancement method according to an embodiment of the present invention
  • FIG. 2 is a flowchart of image enhancement processing on a license plate image according to an embodiment of the present invention
  • FIG. 3 is a flowchart of processing a license plate area image according to an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of the area division of a license plate extension area image according to an embodiment of the present invention.
  • Fig. 5 is the flow chart of the license plate image enhancement method of a specific example of the present invention.
  • FIG. 6 is a schematic diagram of a license plate image enhancement system according to an embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of a license plate image enhancement device according to an embodiment of the present invention.
  • FIG. 8 is a schematic structural diagram of a computer storage medium according to an embodiment of the present invention.
  • Example embodiments will now be described more fully with reference to the accompanying drawings.
  • Example embodiments can be embodied in various forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.
  • the same reference numerals in the drawings denote the same or similar structures, and thus their repeated descriptions will be omitted.
  • the embodiment of the present invention provides a license plate image enhancement method, which can conveniently enhance the license plate image in the existing Elasticsearch cluster into a relational database.
  • the license plate image enhancement method includes the following steps:
  • S100 Input a license plate extension area image, where the license plate extension area image includes a license plate area and a peripheral area surrounding the license plate area;
  • S200 Perform a first image enhancement process on the license plate extension area image to obtain a first enhanced image, where the first image enhancement process is used to restore the color and brightness of the license plate extension area image; here, the color and brightness are restored It refers to the color and brightness processing of the license plate extension area image to make it as close as possible to the brightness and image of the original license plate image;
  • S300 Perform a second image enhancement process on the license plate extension area image to obtain a second enhanced image, where the second image enhancement process is used to perform an image enhancement process for suppressing exposure on the license plate extension area image.
  • the suppression of the exposure here can be realized by processing the brightness to avoid overexposure of the second enhanced image;
  • S400 Perform image fusion on the first enhanced image and the second enhanced image to obtain a fused enhanced license plate image, wherein the enhanced license plate image includes a license plate area and a peripheral area surrounding the license plate area, and the The license plate area is filled with image data of the second enhanced image, and the peripheral area is filled with image data of the first enhanced image.
  • the peripheral area may refer to all areas other than the license plate area in the image, and may also include a transition area between the peripheral area and the license plate area, and the transition area may use the first enhanced image
  • the padding or the second enhanced image padding may also be padding after the first enhanced image and the second enhanced image are fused in proportion.
  • the license plate extended area image is extracted and obtained through the step S100, and the first image enhancement and the second image enhancement are respectively performed on the license plate extended image in the step S200 and the step S300, even if There is a deviation in the positioning of the license plate area. Due to the supplementation of the peripheral area of the license plate extended image, the complete license plate information can be obtained, and the license plate information will not be missing due to inaccurate license plate positioning. Further, in the present invention, through steps S200 and S300, by adopting two processing means for the license plate image enhancement, the first enhanced image (only color and brightness restoration is performed) and the second enhanced image (image enhancement processing to suppress exposure is performed) are obtained respectively.
  • the image enhancement method can be applied to more application scenarios. For example, it can be applied to the scene where the overall brightness of the license plate image is very low or the scene where the entire license plate image is overexposed.
  • the first image enhancement processing may include a first white balance processing
  • the second image enhancement processing may include a second white balance processing
  • the brightness of the image obtained by the second white balance processing is smaller than the the brightness of the image obtained by the first white balance processing.
  • the first image enhancement processing may also include first color correction processing after the first white balance processing
  • the second image enhancement processing may further include G channel recovery and second color processing after the second white balance processing. The correction processing will be described in detail below.
  • the step S100 may directly process the license plate extension area image in raw format.
  • the raw image is the raw data that the CMOS or CCD image sensor converts the captured light source signal into a digital signal.
  • the extracted license plate extension area image can be directly input, for example, an image of a certain size bayer format is input, or after the original image data (ie license plate image) captured by the camera is obtained, the location according to the license plate can be obtained.
  • the license plate extension area image is extracted and obtained, and the license plate extension area image may be an image obtained by expanding the license plate area to a predetermined size around the center.
  • the method of the present invention can restore the license plate information to a greater extent, even if the captured image is captured.
  • the license plate is seriously overexposed, and the license plate information can also be well recovered through the image data in raw format. Therefore, the adaptability of the license plate image enhancement method is stronger, and it is not limited to the license plate image enhancement of the license plate with normal exposure.
  • the brightness adaptive adjustment of the license plate extension area image includes the following steps:
  • S510 Calculate the brightness adjustment ratio according to the target brightness and the brightness average value of the license plate extension area image
  • the input image of the license plate extension area is located in different environments and has different brightness, so it is necessary to adjust the brightness of the image to a uniform level.
  • the target brightness T, G G ave the average luminance of all pixels (green) channels of raw statistical data.
  • m is the number of pixels in the G channel in the license plate extension area image. According to the bayer arrangement, m is half of the raw data size. p(i) is the value corresponding to the current pixel.
  • the brightness adjustment ratio ratio is the ratio of the brightness average value Gave1 of the target brightness T and G channels.
  • the raw format image Q(x, y) after brightness adjustment can be obtained by the following formula.
  • P(x,y) is the value of each pixel on the input original image.
  • step S200 before the brightness of the license plate extension area image is adaptively adjusted, it may further include performing Black Level Correction (BLC) processing on the license plate extension area image, that is, each The pixel value of the pixel minus the value of the black level.
  • BLC Black Level Correction
  • step S530 may be further included: performing denoising processing on the license plate extension area image.
  • Q(x, y) is the image after brightness adjustment in step S220
  • G( ⁇ ) is a Gaussian convolution kernel
  • ⁇ is corresponding to different filter strengths, and different ⁇ values are selected, and the range can be, for example, 0.5-0.8.
  • F(x,y) is the filtered image.
  • x and y are the image coordinate positions corresponding to the pixels.
  • S540 Perform demosaic interpolation (demosaic) processing on the license plate extension area image to obtain a three-channel RGB image.
  • the input in step S100 is a 396*216-size bayer-format license plate extension area image, that is, the image before the demosaicing interpolation process is 396*216 size, in which the G channel pixel accounts for half, and the R and B channel pixels each account for A quarter, after demosaicing interpolation, an image with a size of 396*216*3 is obtained, where the number of pixels in each channel of RGB is one-third of the total number of pixels, that is, 396*216 pixels.
  • both the first image enhancement processing and the second image enhancement processing in the step S200 and the step S300 are processed based on the license plate extension area image obtained after the processing in the step S540.
  • the processing of the license plate area image includes the following steps:
  • S610 Input the license plate area image, and the input here is also preferably the license plate area image in raw format;
  • S620 Determine the license plate color according to the license plate area image
  • the target brightness here is the target brightness used to calculate the brightness adjustment ratio of the license plate extension area image in step S510. Therefore, the present invention can further realize the differentiated parameter configuration when the brightness is adaptively adjusted for different license plate colors.
  • the license plate area image can be processed separately to obtain the license plate color, and the license plate color can be output together with the enhanced license plate image, and can be used as the basis for selecting the target brightness in step S510.
  • step S620 before adopting step S620 to perform color recognition on the license plate area image, it may also include S611: the step of performing brightness adaptive adjustment on the license plate area image, so as to improve the step S620.
  • the brightness adjustment method of the above steps S510 and S520 can be used for the brightness adaptive adjustment of the license plate area image, and the target brightness can be a fixed preset value.
  • step S620: determining the license plate color according to the license plate area image includes the following steps:
  • the following formula may be employed are statistical average R ave plate region RGB color channel, G ave, B ave, and be classified by the ratio between the color plates of the three calculations.
  • the color of the license plate is determined.
  • the following formula can be used to determine the color of the license plate:
  • R M , G M , and B M represent the number of pixels corresponding to the image Q(i) after brightness adjustment, respectively
  • RG ratio and BG ratio represent the ratio of the average value of R channel to the average value of G channel, and the average value of B channel The ratio of the value to the G channel mean.
  • T1 to T8 are preset.
  • the threshold is used to distinguish different license plates, and the range is between 0 and 1.
  • the present invention recognizes the color information of the license plate more accurately and stably through the ratio and position of each color channel. Compared with the license plate color recognition in the prior art, it can be classified into more categories, such as blue license plate, yellow license plate, mixed yellow and green license plate (new energy bus), other solid color license plate (white license plate, green license plate) and so on. And after obtaining the color data, the target brightness parameter configuration can be differentiated for different license plates, so that each color license plate can achieve an ideal enhancement effect.
  • step S200: performing a first image enhancement process on the license plate extension area image, including a first white balance process, and the first white balance process includes the following steps:
  • white balance is processed by the following formula:
  • F(X, Y) is the RGB three-channel image obtained in step S540
  • Rgain and Bgain are white balance gain coefficients obtained a priori, and the set value is usually greater than 1. Then the first enhanced image H A (X, Y) can be obtained.
  • the step S300: performing a second image enhancement process on the license plate extension area image, including a second white balance process, and the second white balance process includes the following steps:
  • multiplying the pixel value of the license plate extension area image by a preset white balance gain coefficient and a preset brightness drop ratio includes the following steps:
  • the pixel value of the green channel of the license plate extension area image is multiplied by a preset brightness reduction ratio, and the brightness reduction ratio is a value between 0 and 1.
  • F(X, Y) is the RGB three-channel image obtained in step S540
  • ratio_bright is the brightness reduction ratio
  • the setting range is about 0.5, for example, it can be set between 0.4 and 0.6, which basically reduces the brightness by half
  • Rgain and Bgain is the white balance gain coefficient obtained a priori, and the set value is usually greater than 1.
  • the white-balanced second enhanced image H A (X, Y) can be obtained.
  • two different white balance processing methods are performed on the license plate extension area image through steps S250 and S300 to obtain a first enhanced image and a second enhanced image respectively, which are used for fusion in step S400.
  • the different processing methods for the G channel, the R channel and the B channel make the color of the second enhanced image after the second white balance processing greatly different from the original input image of the license plate extension area, so it is necessary to restore the G channel,
  • the pixel value of the G channel is restored according to the pixel value of the R channel and the B channel, so as to achieve color enhancement, so that while the second white balance processing is used to achieve the effect of suppressing exposure in the second enhanced image, it can also be restored through the G channel.
  • the color of the second enhanced image tends to be consistent with the image of the original license plate extension area.
  • the step S300 after the second white balance processing is performed, the step of G channel recovery is further included, and the G channel recovery includes the following steps:
  • R(x, y) and B(x, y) represent the pixel values of the R channel and B channel at the same coordinate position (x, y) of the current image, respectively
  • Gin(x, y) is the pixel value of the G channel at the corresponding coordinates before restoration.
  • the pixel value when it meets the judgment condition, recalculates the G channel value of the current point to obtain Gout(x,y).
  • t1, t2, t3, t4 are adjustment parameters, the range is between 0-1.
  • a first color correction step may be included, and in the step S300, after the G channel is restored, a second color correction step may be included , the first color correction and the second color correction can use the same color correction method to process the first enhanced image and the second enhanced image respectively, so as to better restore the color and brightness of the license plate .
  • the license plate extension area image further includes a transition area between the license plate area and the peripheral area.
  • the license plate area is defined as area B
  • the peripheral area is defined as area A
  • the transition area is defined as area C.
  • the distance between the inner side of the peripheral area A and the license plate area B can be preset to realize the area division of the license plate extension area image, and define the coverage areas of the peripheral area A and the transition area C respectively.
  • step S400 performing image fusion on the first enhanced image and the second enhanced image to obtain a fused enhanced license plate image, including the following steps:
  • the license plate area of the license plate enhanced image is filled with the image data of the second enhanced image, and the peripheral area of the license plate enhanced image is filled with the image data of the first enhanced image;
  • p(x, y) is the pixel value of the pixel whose coordinates are (x, y) in the transition area of the license plate enhanced image
  • p A (x, y) is the first enhanced image whose coordinates are (x, y)
  • p B (x, y) is the pixel value of the pixel with coordinates (x, y) in the second enhanced image
  • L1 is the distance between the pixel with coordinates (x, y) and the license plate area B
  • L2 is the distance between the pixel whose coordinates are (x, y) and the peripheral area A.
  • This method can not only realize that the license plate area and the peripheral area in the final enhanced license plate enhanced image adopt different image enhancement methods and have different brightness, so as to avoid the situation that the license plate area is overexposed by adjusting the brightness of the overall image. Also through the setting of the transition area, the transition between the license plate area and the peripheral area can be well realized.
  • the transition area may not be set between the license plate area and the peripheral area in the license plate enhanced image, but only the license plate area and the peripheral area are included, that is, the peripheral part of the license plate area is divided into peripheral areas
  • the data of the first enhanced image is used to fill the peripheral area
  • the data of the second enhanced image is used to fill the license plate area, which can also achieve the purpose of avoiding the overexposure of the license plate area caused by adjusting the brightness of the overall image in the present invention, which also belongs to the present invention. within the scope of protection.
  • step S400 it may also include a step of sharpening the license plate enhanced image, first converting the license plate enhanced image to a YUV color mode, where Y represents brightness, and U and V represent Chroma, in this embodiment, only the Y channel is sharpened by the following formula:
  • F Y represents the Y channel data of the license plate enhanced image to be sharpened and enhanced
  • h(w) is the high-pass filter convolution kernel
  • w is the filter size
  • the specific value can be adjusted generally between 3-7
  • t1 is The enhancement coefficient is between 0 and 1.
  • the result of the high-pass filter convolution of the y channel is added to the Y channel of the original image according to the ratio of t1 to obtain the final enhanced Y channel data G Y , and then the YUV channel data is converted to RGB channel to complete image sharpening enhancement.
  • FIG. 5 it is a schematic diagram of a complete operation flow of license plate image enhancement in a specific example of the present invention. It can be understood that, FIG. 5 shows only an optional example, and in practical applications, some of the steps may be removed.
  • inputting the license plate area image corresponds to the above step S610.
  • the license plate area image After the license plate area image is input, it can be further processed by black level, the black level can be subtracted, and then the brightness can be adaptively adjusted, and then the license plate color can be classified.
  • the result of the license plate color classification can be used for the differential processing of the brightness adaptive adjustment of the license plate extension area image.
  • step S530 performing demosaic interpolation processing, which corresponds to the above step S540.
  • the image in step S540 is subjected to two different white balance processes, wherein the first white balance process corresponds to the above step S200.
  • the correction matrix M can be used to correct the RGB three-channel pixel values, that is, to perform The first color correction process.
  • the second white balance processing corresponds to the above step S300, and then only the G channel is restored for the second enhanced image, that is, only the G channel whose brightness has been decreased in the second enhanced image is restored, and then the second enhanced image is restored. Color correction, that is, the second color correction process.
  • image fusion is performed on the first enhanced image and the second enhanced image, which corresponds to the above step S400, and after the image fusion, the images can be further sharpened to obtain the final output enhanced license plate image.
  • the license plate image enhancement process in Figure 5 adopts a process design similar to the ISP pipeline in image processing. Each processing module is connected to each other and performs pipeline work. It is simple and efficient, the algorithm complexity is reduced, and the amount of calculation is greatly reduced. The calculation formula and algorithm, and the method of image fusion can be used to make the enhanced license plate image back to the original image without any obvious sense of violation.
  • the image enhancement methods of step S200 and step S300 are only examples, and other image enhancement methods are also possible, as long as the first image enhancement processing can restore the color and brightness of the image, the second image enhancement processing The color and brightness of the image can be restored and the exposure can be suppressed.
  • the first image enhancement process may not include white balance, but only the first color correction, or other color and brightness processing methods
  • the second image enhancement process may combine the white balance process with the The process of multiplying by the luminance drop ratio is divided into two steps. For example, first multiply the pixel values of the RGB channels by the luminance drop ratio, and then white balance the R and B channels, or firstly white-balance the R and B channels.
  • an embodiment of the present invention further provides a license plate image enhancement system for implementing the license plate image enhancement method, and the system includes:
  • the image extraction module M100 is used to input a license plate extension area image, and the license plate extension area image includes a license plate area and a peripheral area surrounding the license plate area;
  • a first processing module M200 configured to perform a first image enhancement process on the license plate extension area image, and the first image enhancement process is used to perform color and brightness restoration on the license plate extension area image;
  • the second processing module M300 is configured to perform a second image enhancement process on the license plate extension area image to obtain a second enhanced image, and the second image enhancement process is used to perform image enhancement for suppressing exposure on the license plate extension area image deal with;
  • the image fusion module M400 is used for image fusion of the first enhanced image and the second enhanced image to obtain an enhanced license plate image after fusion, wherein the enhanced license plate image includes a license plate area and a peripheral area, and the license plate The area is filled with image data of the second enhanced image, and the peripheral area is filled with image data of the first enhanced image.
  • the license plate extended area image is extracted by the image extraction module M100, and the first image enhancement and the second 2.
  • Image enhancement even if there is a deviation in the positioning of the license plate area, due to the supplement of the peripheral area of the license plate extended image, the complete license plate information can be obtained, and the license plate information will not be missing due to the inaccurate positioning of the license plate.
  • the present invention uses the first processing module M200 and the second processing module M300 to obtain the first enhanced image (for image color and brightness restoration) and the second enhanced image (for image color and brightness restoration) by adopting two processing means for the license plate image enhancement.
  • the license plate image enhancement system can be implemented by a server or a common personal computer, for example, the license plate image enhancement system runs on the CPU of the ARM platform.
  • the license plate image enhancement system of the present invention can provide a callable parameter interface for several subsequent processing modules, and through the adjustment and setting of parameters, the enhanced license plate can be adjusted for hue, saturation, sharpness, and brightness, thereby improving the performance of the license plate. Subsequent license plates enhance the adaptability of technology applications.
  • An embodiment of the present invention further provides a license plate image enhancement device, including a processor; a memory, in which executable instructions of the processor are stored; wherein the processor is configured to execute the executable instructions by executing the executable instructions.
  • the steps of the license plate image enhancement method including a processor; a memory, in which executable instructions of the processor are stored; wherein the processor is configured to execute the executable instructions by executing the executable instructions.
  • aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention can be embodied in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, which may be collectively referred to herein as implementations "circuit", “module” or "system”.
  • FIG. 7 An electronic device 600 according to this embodiment of the present invention is described below with reference to FIG. 7 .
  • the electronic device 600 shown in FIG. 7 is only an example, and should not impose any limitation on the function and scope of use of the embodiments of the present invention.
  • electronic device 600 takes the form of a general-purpose computing device.
  • Components of the electronic device 600 may include, but are not limited to, at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610), a display unit 640, and the like.
  • the storage unit stores program codes, and the program codes can be executed by the processing unit 610, so that the processing unit 610 executes the various exemplary embodiments according to the present invention described in the above-mentioned part of the electronic prescription circulation processing method of this specification.
  • the processing unit 610 may perform the steps shown in FIG. 1 .
  • the storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 6201 and/or a cache storage unit 6202 , and may further include a read only storage unit (ROM) 6203 .
  • RAM random access storage unit
  • ROM read only storage unit
  • the storage unit 620 may also include a program/utility 6204 having a set (at least one) of program modules 6205 including, but not limited to, an operating system, one or more application programs, other program modules, and programs Data, each or some combination of these examples may include an implementation of a network environment.
  • the bus 630 may be representative of one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local area using any of a variety of bus structures bus.
  • the electronic device 600 may also communicate with one or more external devices 700 (eg, keyboards, pointing devices, Bluetooth devices, etc.), with one or more devices that enable a user to interact with the electronic device 600, and/or with Any device (eg, router, modem, etc.) that enables the electronic device 600 to communicate with one or more other computing devices. Such communication may occur through input/output (I/O) interface 650 . Also, the electronic device 600 may communicate with one or more networks (eg, a local area network (LAN), a wide area network (WAN), and/or a public network such as the Internet) through a network adapter 660 . Network adapter 660 may communicate with other modules of electronic device 600 through bus 630 . It should be appreciated that, although not shown, other hardware and/or software modules may be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives and data backup storage systems.
  • the processor executes the license plate image enhancement method when executing the executable instructions, thereby obtaining the beneficial effects of the license plate image enhancement method. It is guaranteed to obtain complete license plate information, and on the other hand, it can be applied to more application scenarios.
  • Embodiments of the present invention further provide a computer-readable storage medium for storing a program, which implements the steps of the license plate image enhancement method when the program is executed.
  • aspects of the present invention can also be implemented in the form of a program product comprising program code for enabling the program product to run on a terminal device The terminal device executes the steps according to various exemplary embodiments of the present invention described in the above-mentioned electronic prescription flow processing method section of this specification.
  • a program product 800 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read only memory (CD-ROM) and include program codes, and can be used in a terminal device, For example running on a personal computer.
  • CD-ROM compact disk read only memory
  • the program product of the present invention is not limited thereto, and in this document, a readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
  • the program product may employ any combination of one or more readable media.
  • the readable medium can be a readable signal medium or a readable storage medium.
  • the readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples (non-exhaustive list) of readable storage media include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
  • the computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code therein. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • a readable storage medium can also be any readable medium other than a readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
  • Program code embodied on a readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
  • Program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages—such as Java, C++, etc., as well as conventional procedural Programming Language - such as the "C" language or similar programming language.
  • the program code may execute entirely on the user computing device, partly on the user device, as a stand-alone software package, partly on the user computing device and partly on a remote computing device, or entirely on the remote computing device or cluster execute on.
  • the remote computing device may be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (eg, using an Internet service provider business via an Internet connection).
  • LAN local area network
  • WAN wide area network
  • the stored program realizes the steps of the license plate image enhancement method when it is executed, so that the beneficial effects of the above-mentioned license plate image enhancement method can be obtained, that is, on the one hand, it can ensure Obtaining complete license plate information, on the other hand, can be applied to more application scenarios.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Processing (AREA)

Abstract

一种车牌图像增强方法、系统、设备及存储介质，所述方法包括：输入车牌扩展区域图像(S100)，车牌扩展区域图像包括车牌区域和环绕车牌区域的外围区域；将车牌扩展区域图像进行图像增强处理，得到第一增强图像(S200)；将所述车牌扩展区域图像进行第二图像增强处理，得到第二增强图像(S300)；将第一增强图像和第二增强图像进行图像融合，得到融合后的车牌增强图像(S400)，其中，车牌增强图像的车牌区域采用第二增强图像的图像数据填充，车牌增强图像的外围区域采用第一增强图像的图像数据填充。通过对车牌区域和非车牌区域的不同图像增强方法和图像融合，实现适用于不同场景下的图像增强。

Description

车牌图像增强方法、系统、设备及存储介质
本申请要求申请号为：CN202010604429.7、申请日为2020.06.29的中国国家知识产权局的在先专利申请为优先权，该在先专利申请文本中的内容通过引用而完全加入本专利申请中。
技术领域
本发明涉及图像处理技术领域，尤其涉及一种车牌图像增强方法、系统、设备及存储介质。
背景技术
在监控领域的车辆卡口环境中，往往是在抓拍设备旁边安装一个爆闪灯，在车辆驶来的一瞬间，打开爆闪灯补光，并且抓拍一帧图片，这样不仅可以在全天候下都可以得到车辆的明亮的图片，并且驾驶室里的前排司机或者乘客的行为或者动作也会一览无余，用以辅助判断是否有违反交通法规的行为(比如未系安全带，开车接听电话等)，但是因为爆闪灯的亮度往往更高，导致某些场景下车牌由于反光强烈，有时会严重过曝，或者由于多辆车经过，存在遮挡，导致某辆车车牌区域未补到光的现象，这时抓拍到的图片中车牌号码很难清晰的辨认出来，影响车牌号码信息的获取。因此，在车牌信息识别的过程中，如果采集到的车牌图像存在不清晰或难以识别的缺陷，可以预先通过图像增强的方式对采集到的车牌图像进行增强和复原处理，有利于后续车牌信息的识别。
基于此，现有技术中出现了一种车牌图像增强方法，首先对车牌的亮度图像进行了对比度增强，随后对图片进行去噪处理，提高车牌上字体清晰度并去除车牌背景噪声，从而达到增强目的。然而，现有技术中的车牌图像增强方法中，是对整个车牌图像进行亮度增强，可能会造成车牌区域严重过曝的情况，并且进行车牌识别时，如果车牌定位不准确，在车牌区域信息缺失或者不全时，很难将车牌号码和车牌颜色复原出来。
发明内容
针对现有技术中的问题，本发明的目的在于提供一种车牌图像增强方法、系统、 设备及存储介质，通过对车牌区域和非车牌区域的不同图像增强方法和图像融合，实现适用于不同场景下的图像增强。
本发明实施例提供一种车牌图像增强方法，包括如下步骤：
输入车牌扩展区域图像，所述车牌扩展区域图像包括车牌区域和环绕所述车牌区域的外围区域；
将所述车牌扩展区域图像进行第一图像增强处理，得到第一增强图像；
将所述车牌扩展区域图像进行第二图像增强处理，得到第二增强图像，所述第二图像增强处理用于将所述车牌扩展区域图像进行抑制曝光的图像增强处理；
将所述第一增强图像和所述第二增强图像进行图像融合，得到融合后具有车牌区域和外围区域的车牌增强图像，所述车牌区域采用所述第二增强图像的图像数据填充，所述外围区域采用所述第一增强图像的图像数据填充。
可选地，所述第一图像增强处理包括第一白平衡处理，所述第二图像增强处理包括第二白平衡处理，所述第二白平衡处理得到的图像的亮度小于所述第一白平衡处理得到的图像的亮度。
可选地，所述第一白平衡处理，包括将所述车牌扩展区域图像的像素值乘以预设的白平衡增益系数，得到第一增强图像；
所述第二白平衡处理，包括将所述车牌扩展区域图像的像素值乘以预设的白平衡增益系数和一预设的亮度下降比值，得到第二增强图像。
可选地，所述第二白平衡处理包括：
将所述车牌扩展区域图像的红色通道和蓝色通道的像素值分别乘以预设的白平衡增益系数和一预设的亮度下降比值；
将所述车牌扩展区域图像的绿色通道的像素值乘以预设的亮度下降比值，所述亮度下降比值为介于0和1之间的数值；
所述第二图像增强处理中，所述第二白平衡处理之后，还包括如下步骤：
对于所述第一增强图像的绿色通道的每个像素G(x,y)，根据红色通道与像素G(x,y)的位置相对应的像素R(x,y)和绿色通道与像素G(x,y)的位置相对应的像素B(x,y)的像素值，调整像素G(x,y)的像素值。
可选地，所述车牌扩展区域图像还包括位于所述车牌区域和所述外围区域之间的过渡区域；
将所述第一增强图像和所述第二增强图像进行图像融合，得到融合后的车牌增 强图像，包括如下步骤：
采用所述第二增强图像的图像数据填充所述车牌增强图像的车牌区域，采用所述第一增强图像的图像数据填充所述车牌增强图像的外围区域；
采用如下公式计算所述车牌增强图像的过渡区域的各个像素的像素值：
Figure PCTCN2020121507-appb-000001
其中，p(x,y)为所述车牌增强图像的过渡区域中坐标为(x,y)的像素的像素值，p A(x,y)为第一增强图像中坐标为(x,y)的像素的像素值，p B(x,y)为第二增强图像中坐标为(x,y)的像素的像素值，L1为坐标为(x,y)的像素与车牌区域的距离，L2为坐标为(x,y)的像素与外围区域的距离。
可选地，将所述输入车牌扩展区域图像之后，还包括车牌扩展区域的亮度调节，所述亮度调节包括如下步骤：
根据目标亮度和所述车牌扩展区域图像的亮度平均值计算亮度调节比值；
将所述车牌扩展区域图像中各个像素点的像素值乘以所述亮度调节比值，得到亮度调节后的车牌扩展区域图像；
所述将所述车牌扩展区域图像进行第一图像增强处理，包括将所述亮度调节后的车牌扩展区域图像进行第一图像增强处理；
所述将所述车牌扩展区域图像进行第二图像增强处理，包括将所述亮度调节后的车牌扩展区域图像进行第二图像增强处理。
可选地，所述亮度调节之前，还包括如下步骤：
输入车牌区域图像；
根据所述车牌区域图像确定车牌颜色；
根据车牌颜色与目标亮度的映射关系，确定所对应的目标亮度。
可选地，所述根据所述车牌区域图像确定车牌颜色，包括如下步骤：
分别统计所述车牌区域图像中红色、绿色和蓝色通道的像素平均值；
计算红色通道像素平均值和绿色通道像素平均值的比值RG ratio；
计算蓝色通道像素平均值和绿色通道像素平均值的比值BG ratio；
根据预设的不同颜色与比值RG ratio以及BG ratio的映射关系，确定车牌颜色。
可选地，所述输入车牌扩展区域图像，包括输入raw格式的车牌扩展区域图像；
所述第一图像增强处理和所述第二图像增强处理均为直接对所述raw格式的车牌扩展区域图像进行的处理。
通过采用本发明所提供的车牌图像增强方法，一方面通过对车牌扩展图像进行图像增强，即使车牌区域定位有偏差，由于车牌扩展图像的外围区域的补充，可以保证获得完整的车牌信息，而不至于因为车牌定位不准而缺少车牌信息；另一方面，通过对车牌图像增强采用两种处理手段，分别获得第一增强图像和第二增强图像，并且将第一增强图像和第二增强图像进行融合，使得最终得到的车牌增强图像中车牌区域和外围区域中采用不同的增强方法，可以避免现有技术中对整张图像采用同样的图像增强方法而导致车牌区域过曝的问题，因此，该图像增强方法可以适用于更多的应用场景。
本发明实施例还提供一种车牌图像增强系统，其特征在于，应用于所述的车牌图像增强方法，所述系统包括：
图像提取模块，用于输入车牌扩展区域图像，所述车牌扩展区域图像包括车牌区域和环绕所述车牌区域的外围区域；
第一处理模块，用于将所述车牌扩展区域图像进行第一图像增强处理；
第二处理模块，用于将所述车牌扩展区域图像进行第二图像增强处理，得到第二增强图像，所述第二图像增强处理用于将所述车牌扩展区域图像进行抑制曝光的图像增强处理；
图像融合模块，用于将所述第一增强图像和所述第二增强图像进行图像融合，得到融合后的车牌增强图像，其中，所述车牌增强图像包括车牌区域和外围区域，所述车牌区域采用所述第二增强图像的图像数据填充，所述外围区域采用所述第一增强图像的图像数据填充。
通过采用本发明所提供的车牌图像增强系统，一方面通过对车牌扩展图像进行图像增强，即使车牌区域定位有偏差，由于车牌扩展图像的外围区域的补充，可以保证获得完整的车牌信息，而不至于因为车牌定位不准而缺少车牌信息；另一方面，通过对车牌图像增强采用两种处理手段，分别获得第一增强图像和第二增强图像，并且将第一增强图像和第二增强图像进行融合，使得最终得到的车牌增强图像中车牌区域和外围区域中采用不同的增强方法，可以避免现有技术中对整张图像采用同样的图像增强方法而导致车牌区域过曝的问题，因此，该图像增强方法可以适用于更多的应用场景。
本发明实施例还提供一种车牌图像增强设备，包括：
处理器；
存储器，其中存储有所述处理器的可执行指令；
其中，所述处理器配置为经由执行所述可执行指令来执行所述的车牌图像增强方法的步骤。
通过采用本发明所提供的车牌图像增强设备，所述处理器在执行所述可执行指令时执行所述的车牌图像增强方法，由此可以获得上述车牌图像增强方法的有益效果，即一方面可以保证获得完整的车牌信息，另一方面可以适用于更多的应用场景。
本发明实施例还提供一种计算机可读存储介质，用于存储程序，所述程序被执行时实现所述的车牌图像增强方法的步骤。
通过采用本发明所提供的计算机可读存储介质，其中存储的程序在被执行时实现所述的车牌图像增强方法的步骤，由此可以获得上述车牌图像增强方法的有益效果，即一方面可以保证获得完整的车牌信息，另一方面可以适用于更多的应用场景。
附图说明
通过阅读参照以下附图对非限制性实施例所作的详细描述，本发明的其它特征、目的和优点将会变得更明显。
图1是本发明一实施例的车牌图像增强方法的流程图；
图2是本发明一实施例的将车牌图像进行图像增强处理的流程图；
图3是本发明一实施例的对车牌区域图像的处理的流程图；
图4是本发明一实施例的车牌扩展区域图像进行区域划分的示意图；
图5是本发明一具体实例的车牌图像增强方法的流程图；
图6是本发明一实施例的车牌图像增强系统的示意图；
图7是本发明一实施例的车牌图像增强设备的结构示意图；
图8是本发明一实施例的计算机存储介质的结构示意图。
具体实施方式
现在将参考附图更全面地描述示例实施方式。然而，示例实施方式能够以多种形式实施，且不应被理解为限于在此阐述的实施方式；相反，提供这些实施方式使得本发明将全面和完整，并将示例实施方式的构思全面地传达给本领域的技术人员。在图中相同的附图标记表示相同或类似的结构，因而将省略对它们的重复描述。
为了解决现有技术的技术问题，本发明实施例提供一种车牌图像增强方法，可 以方便地将已有的Elasticsearch集群中的车牌图像增强至关系型数据库中。
如图1所示，在该实施例中，所述车牌图像增强方法包括如下步骤：
S100：输入车牌扩展区域图像，所述车牌扩展区域图像包括车牌区域和环绕所述车牌区域的外围区域；
S200：将所述车牌扩展区域图像进行第一图像增强处理，得到第一增强图像，所述第一图像增强处理用于将所述车牌扩展区域图像进行颜色和亮度恢复；此处颜色和亮度恢复指的是将车牌扩展区域图像进行颜色和亮度处理，使其尽可能贴近原有车牌图像的亮度和图像；
S300：将所述车牌扩展区域图像进行第二图像增强处理，得到第二增强图像，所述第二图像增强处理用于将所述车牌扩展区域图像进行抑制曝光的图像增强处理，具体地，可以为抑制曝光的颜色和亮度恢复，此处抑制曝光可以通过对亮度的处理来实现避免第二增强图像过度曝光；
S400：将所述第一增强图像和所述第二增强图像进行图像融合，得到融合后的车牌增强图像，其中，所述车牌增强图像包括车牌区域和环绕所述车牌区域的外围区域，所述车牌区域采用所述第二增强图像的图像数据填充，所述外围区域采用所述第一增强图像的图像数据填充。此处，所述外围区域可以指图像中所述车牌区域以外的所有区域，也可以在所述外围区域和所述车牌区域中间还包括一个过渡区域，该过渡区域可以采用所述第一增强图像填充或所述第二增强图像填充，也可以是使用第一增强图像和第二增强图像按比例融合后填充。
通过采用本发明所提供的车牌图像增强方法，首先通过所述步骤S100提取得到车牌扩展区域图像，并且在步骤S200和步骤S300中分别对车牌扩展图像进行第一图像增强和第二图像增强，即使车牌区域定位有偏差，由于车牌扩展图像的外围区域的补充，可以保证获得完整的车牌信息，而不至于因为车牌定位不准而缺少车牌信息。进一步地，本发明通过步骤S200和步骤S300，通过对车牌图像增强采用两种处理手段，分别获得第一增强图像(仅进行颜色和亮度恢复)和第二增强图像(进行抑制曝光的图像增强处理)，并且将第一增强图像和第二增强图像进行融合，使得最终得到的车牌增强图像中车牌区域和外围区域中采用不同的增强方法，由于在步骤S400中融合时，车牌区域采用亮度较低的第二增强图像的数据进行填充，可以避免现有技术中对整张图像采用同样的图像增强方法而导致车牌区域过曝的问题，因此，该图像增强方法可以适用于更多的应用场景，例如可以适用于车牌图像整体亮度很低的场景或 者车牌图像整体过曝的场景等。
在该实施例中，所述第一图像增强处理可以包括第一白平衡处理，所述第二图像增强处理可以包括第二白平衡处理，所述第二白平衡处理得到的图像的亮度小于所述第一白平衡处理得到的图像的亮度。进一步地，所述第一图像增强处理还可以包括第一白平衡处理后的第一颜色校正处理，所述第二图像增强处理还可以包括第二白平衡处理后的G通道恢复和第二颜色校正处理，具体会在下文中进行详细描述。
在该实施例中，所述步骤S100中可以直接对raw格式的车牌扩展区域图像进行处理。Raw图像就是CMOS或者CCD图像感应器将捕捉到的光源信号转化为数字信号的原始数据。所述步骤S100中，可以直接输入已经提取好的车牌扩展区域图像，例如输入一定尺寸的bayer格式的图像，也可以在获取到摄像机抓拍的原始图像数据(即车牌图像)之后，根据车牌定位的结果，提取得到车牌扩展区域图像，车牌扩展区域图像可以是以车牌区域为中心向四周扩展预设尺寸之后得到的图像。由此此处对raw格式的原始数据直接进行图像增强，不同于现有其他增强算法中对抓拍后图像进行复原和增强，本发明的方法可以更大程度地复原出车牌信息，即使抓拍到的车牌严重过曝，通过raw格式的图像数据也可以很好地恢复出车牌信息。由此，车牌图像增强方法的适应性更强，不仅局限于对曝光正常的车牌进行车牌图像增强。
如图2所示，在该实施例中，所述步骤S100和步骤S200之间，还包括车牌扩展区域图像的亮度自适应调节，具体地，车牌扩展区域图像的亮度自适应调节包括如下步骤：
S510：根据目标亮度和所述车牌扩展区域图像的亮度平均值计算亮度调节比值；
输入的车牌扩展区域图像所处环境不同，亮度也不同，所以需要对图像进行亮度调节到统一水平。例如：目标亮度为T，统计raw数据上G(绿色)通道的所有像素的亮度平均值G ave。
Figure PCTCN2020121507-appb-000002
其中m为此车牌扩展区域图像中G通道的像素数，根据bayer排列，m为raw数据尺寸的一半。p(i)为当前像素对应的值。
根据如下公式计算亮度调节比值ratio：
Figure PCTCN2020121507-appb-000003
即亮度调节比值ratio为目标亮度T和G通道的亮度平均值G ave1比值。
S520：将所述车牌图像中各个像素点的像素值乘以所述亮度调节比值。
即通过如下公式可以得到亮度调节后的raw格式图像Q(x,y)。其中，P(x,y)为输入原图像上每个像素点的值。
Q(x,y)＝P(x,y)×ratio1
在该实施例中，所述步骤S200中，在所述车牌扩展区域图像的亮度自适应调节之前，还可以包括对所述车牌扩展区域图像进行黑电平(BlackLevelCorrection，BLC)处理，即每个像素的像素值减去黑电平的数值。
如图2所示，在该实施例中，在所述车牌扩展区域图像的亮度自适应调节之后，还可以包括步骤S530：对所述车牌扩展区域图像进行去噪处理。
具体地，可以采用如下公式对图像进行去噪处理：
Figure PCTCN2020121507-appb-000004
其中，Q(x,y)为步骤S220亮度调节后的图像，G(δ)为高斯卷积核，δ是对应不同的滤波强度，选择不同的δ值，范围例如可以为0.5～0.8。F(x,y)为滤波后的图像。x和y是像素点对应的图像坐标位置。
进一步地，如图2所示，所述步骤S530之后，还包括如下步骤：
S540：将所述车牌扩展区域图像进行去马赛克插值(demosaic)处理，得到三通道的RGB图像。
例如步骤S100中输入的是396*216尺寸的bayer格式的车牌扩展区域图像，即去马赛克插值处理前图像为396*216尺寸，其中G通道像素占二分之一，R、B通道像素各占四分之一，在进行了去马赛克插值后得到一张尺寸为396*216*3的图像，其中R G B各个通道像素数为总像素数的三分之一，即396*216个像素。
在该实施例中，所述步骤S200和步骤S300中第一图像增强处理和第二图像增强处理均是基于所述步骤S540处理后得到的车牌扩展区域图像进行处理的。
在该实施例中，所述步骤S100和步骤S200之间，还包括对车牌区域图像的处理，所述对车牌区域图像的处理包括如下步骤：
S610：输入车牌区域图像，此处输入的也优选是raw格式的车牌区域图像；
S620：根据所述车牌区域图像确定车牌颜色；
S630：根据车牌颜色与目标亮度的映射关系，确定所对应的目标亮度。此处目标亮度即为步骤S510中用于计算车牌扩展区域图像的亮度调节比值的目标亮度。因此，本发明可以进一步实现针对于不同车牌颜色的亮度自适应调节时的差异化参数配 置。
因此，该实施例中进一步可以通过单独对车牌区域图像进行处理，得到车牌颜色，车牌颜色可以与增强的车牌图像一起输出，并且可以作为步骤S510中的目标亮度的选择的依据。
进一步地，在所述对车牌区域图像的处理中，在采用步骤S620对车牌区域图像进行颜色识别之前，还可以包括S611：对所述车牌区域图像进行亮度自适应调节的步骤，以提高步骤S620中颜色识别的准确率，具体地，对车牌区域图像的亮度自适应调节可以采用如上步骤S510和步骤S520的亮度调节方法，其中目标亮度可以是一个固定的预设值。
在该实施例中，所述步骤S620：根据所述车牌区域图像确定车牌颜色，包括如下步骤：
分别统计所述车牌区域图像中红色、绿色和蓝色通道的像素平均值；
例如，可以采用如下公式分别统计车牌区域RGB三通道的平均值R ave，G ave，B ave，并通过计算三者之间的比值来进行车牌颜色分类。
Figure PCTCN2020121507-appb-000005
Figure PCTCN2020121507-appb-000006
Figure PCTCN2020121507-appb-000007
Figure PCTCN2020121507-appb-000008
Figure PCTCN2020121507-appb-000009
计算红色通道像素平均值和绿色通道像素平均值的比值RG ratio；
计算蓝色通道像素平均值和绿色通道像素平均值的比值BG ratio；
根据预设的不同颜色与比值RG ratio以及BG ratio的映射关系，确定车牌颜色，例如可以采用如下公式来判断车牌颜色：
Figure PCTCN2020121507-appb-000010
其中R M，G M，B M分别表示在亮度调节后的图像Q(i)中所对应的像素数，RG ratio 和BG ratio分别代表R通道平均值和G通道平均值的比值，B通道平均值和G通道平均值的比值。
即通过比较这两个比值RG ratio和BG ratio，分类为blue蓝色车牌，yellow黄色车牌，mix黄绿混合车牌，other其他纯色车牌(绿牌或白牌)，T1到T8为预先设定的阈值，来区分不同车牌，范围均在0～1之间。
因此，本发明通过颜色各个通道的比值和位置，更加精准和稳定地识别出车牌的颜色信息。相比于现有技术中的车牌颜色识别，可以由更多的分类类别，例如蓝牌、黄牌、黄绿混合牌(新能源公交车)、其他纯色牌(白牌、绿牌)等。并且通过获取到颜色数据之后，可以对不同车牌进行差异化的目标亮度参数配置，使得各个颜色车牌都可以达到理想的增强效果。
具体地，所述步骤S200：将所述车牌扩展区域图像进行第一图像增强处理，包括第一白平衡处理，所述第一白平衡处理包括如下步骤：
将所述第一增强图像的RGB三通道的像素值分别乘以预设的白平衡增益系数，将得到的图像作为第一增强图像；
例如，通过如下公式对白平衡处理：
Figure PCTCN2020121507-appb-000011
在该实施例中，其中，F(X,Y)为步骤S540得到的RGB三通道图像，Rgain和Bgain为先验得到的白平衡增益系数，设定值通常大于1。之后便可得到第一增强图像H A(X,Y)。
所述步骤S300：将所述车牌扩展区域图像进行第二图像增强处理，包括第二白平衡处理，所述第二白平衡处理包括如下步骤：
将所述车牌扩展区域图像的像素值乘以预设的白平衡增益系数和一预设的亮度下降比值，将得到的图像作为第二增强图像：
进一步地，将所述车牌扩展区域图像的像素值乘以预设的白平衡增益系数和一预设的亮度下降比值，包括如下步骤：
将所述车牌扩展区域图像的红色通道和蓝色通道的像素值分别乘以预设的白平衡增益系数和一预设的亮度下降比值；
将所述车牌扩展区域图像的绿色通道的像素值乘以预设的亮度下降比值，所述亮度下降比值为介于0和1之间的数值。
具体地，可以采用如下公式对所述车牌扩展区域图像进行处理，得到第二增强图像：
Figure PCTCN2020121507-appb-000012
其中F(X,Y)为步骤S540得到的RGB三通道图像，ratio_bright为亮度下降比值，设定范围在0.5左右，例如可以设定为0.4～0.6之间，基本上使得亮度降低一半，Rgain和Bgain为先验得到的白平衡增益系数，设定值通常大于1。之后便可得到白平衡后的第二增强图H A(X,Y)。
因此，该实施例通过步骤S250和步骤S300对所述车牌扩展区域图像进行了两种不同的白平衡处理方法，分别得到了第一增强图像和第二增强图像，用于步骤S400中的融合。
在该实施例中，所述步骤S300中，所述第二白平衡处理时，由于只有R通道和B通道同时乘以白平衡增益系数和亮度下降比值，而G通道则只乘以亮度下降比值，对G通道与R通道和B通道不同的处理方式使得第二白平衡处理后的第二增强图像的色彩会与最初输入的车牌扩展区域图像有较大的差别，因此需要进行G通道恢复，将G通道的像素值根据R通道和B通道的像素值进行恢复，从而实现色彩增强，使得在采用第二白平衡处理实现第二增强图像中抑制曝光的效果的同时，也可以通过G通道恢复使得第二增强图像色彩与原车牌扩展区域图像趋于一致。
因此，所述步骤S300中，进行第二白平衡处理之后，还包括G通道恢复的步骤，所述G通道恢复包括如下步骤：
对所述第二增强图像中只做了亮度下降的G通道进行恢复，具体地，对于所述第一增强图像的绿色通道的每个像素G(x,y)，根据红色通道与像素G(x,y)的位置相对应的像素R(x,y)和绿色通道与像素G(x,y)的位置相对应的像素B(x,y)的像素值，调整像素G(x,y)的像素值。
具体地，可以采用如下公式对G通道进行恢复：
如果R(x,y)×t1+B(x,y)×t2≥Gin(x,y)
则Gout(x,y)＝R(x,y)×t3+B(x,y)×t4
其中R(x,y)和B(x,y)分别表示当前图像同一坐标位置(x，y)下R通道和B通道像素值，Gin(x,y)为复原前对应坐标下G通道的像素值，当其满足判断条件时，即重新计算当前点的G通道值，得到Gout(x,y)。t1，t2，t3，t4是调节参数， 范围在0-1之间。
进一步地，所述步骤S200中，所述第一白平衡处理之后，还可以包括第一颜色校正的步骤，所述步骤S300中，所述G通道恢复之后，还可以包括第二颜色校正的步骤，所述第一颜色校正和所述第二颜色校正可以采用同样的颜色校正方法，分别对所述第一增强图像和第二增强图像进行处理，从而更好地恢复所述车牌的颜色和亮度。例如，可以通过预设的校正矩阵M，采用K′＝M×K调整RGB三通道的像素值，其中K′为校正后的RGB三通道像素值，K为校正前的RGB三通道像素值。
如图4所示，在该实施例中，所述车牌扩展区域图像还包括位于所述车牌区域和所述外围区域之间的过渡区域。在图4中，定义车牌区域为B区，外围区域为A区，过渡区域为C区。外围区域A的内侧边与车牌区域B之间的距离可以预先设定，以实现车牌扩展区域图像的区域划分，分别界定外围区域A和过渡区域C的覆盖范围。
在该实施例中，所述步骤S400：将所述第一增强图像和所述第二增强图像进行图像融合，得到融合后的车牌增强图像，包括如下步骤：
采用所述第二增强图像的图像数据填充所述车牌增强图像的车牌区域，采用所述第一增强图像的图像数据填充所述车牌增强图像的外围区域；
采用如下公式计算所述车牌增强图像的过渡区域的各个像素的像素值：
Figure PCTCN2020121507-appb-000013
其中，p(x,y)为所述车牌增强图像的过渡区域中坐标为(x,y)的像素的像素值，p A(x,y)为第一增强图像中坐标为(x,y)的像素的像素值，p B(x,y)为第二增强图像中坐标为(x,y)的像素的像素值，L1为坐标为(x,y)的像素与车牌区域B的距离，L2为坐标为(x,y)的像素与外围区域A的距离。
该种方式不仅可以实现最终增强得到的车牌增强图像中车牌区域与外围区域采用不同图像增强方式，具有不同的亮度，从而避免对整体图像进行亮度调节而引起车牌区域过曝的情况。还通过过渡区域的设置，可以很好地实现车牌区域与外围区域之间的过渡。
在另一种可替代的实施方式中，所述车牌增强图像中车牌区域与外围区域之间也可以不设置过渡区，而只包括车牌区域和外围区域，即车牌区域外围的部分均划分为外围区域，并且采用第一增强图像的数据填充外围区域，第二增强图像的数据填充车牌区域，也可以实现本发明中避免对整体图像进行亮度调节而引起车牌区域过曝的 目的，也属于本发明的保护范围之内。
进一步地，在所述步骤S400之后，还可以包括对所述车牌增强图像进行锐化处理的步骤，首先将所述车牌增强图像转换到YUV色彩模式下，其中，Y代表亮度，U和V代表色度，该实施例中仅对Y通道采用如下公式进行图像锐化操作：
Figure PCTCN2020121507-appb-000014
其中F Y表示待锐化增强的车牌增强图像的Y通道数据，h(w)是高通滤波卷积核，w为滤波器尺寸，具体数值可以调整，一般介于3-7之间，t1为增强系数，0-1之间，将y通道高通滤波卷积后的结果按照t1的比值和原图像Y通道相加得到最后的增强后Y通道数据G Y，之后将YUV各个通道数据转化到RGB通道，完成图像锐化增强。
如图5所示，为本发明一具体实例的车牌图像增强的完整操作流程示意图。可理解的是，图5示出的仅为一种可选的实例，在实际应用中，其中的部分步骤可以去除。
如图5所示，输入车牌区域图像对应于上述步骤S610，输入车牌区域图像之后可以进一步对其进行黑电平处理，减去黑电平，然后进行亮度自适应调节，然后进行车牌颜色分类，对应于上述步骤S620。车牌颜色分类的结果可以用于车牌扩展区域图像的亮度自适应调节的差异化处理。输入车牌扩展区域图像对应于上述步骤S100，可以首先对车牌扩展区域图像进行黑电平处理，然后进行亮度自适应调节，对应于上述步骤S510和步骤S520，然后进行去噪处理，对应于上述步骤S530，进行去马赛克插值处理，对应于上述步骤S540。然后将步骤S540的图像进行两种不同的白平衡处理，其中第一白平衡处理对应于上述步骤S200，得到第一增强图像后，可以采用校正矩阵M对RGB三通道像素值进行校正，即进行第一颜色校正处理。第二白平衡处理对应于上述步骤S300，然后只针对第二增强图像进行G通道恢复，即对所述第二增强图像中只做了亮度下降的G通道进行恢复，然后对第二增强图像进行颜色校正，即第二颜色校正处理。然后将第一增强图像和第二增强图像进行了图像融合，对应于上述步骤S400，并且在图像融合之后，可以进一步对图像进行锐化处理，得到最终输出的车牌增强图像。
图5中的车牌图像增强流程采用类似于图像处理中的ISP pipeline的流程设计，各个处理模块相互连接，进行流水线工作，简单高效，算法复杂度降低，计算量也大大减少，不需要使用复杂的计算公式和算法，并且可以通过图像融合的方法，使得增强后的车牌图像贴回原图后，没有明显的违和感。
在本发明中，步骤S200和步骤S300的图像增强方式仅为示例，采用其他的图像增强方式也是可以的，只要能实现第一图像增强处理对图像的颜色和亮度进行恢复，第二图像增强处理对图像的颜色和亮度进行恢复并且可以抑制曝光即可。例如，在其他可替代的实施方式中，第一图像增强处理可以不包括白平衡，只包括第一颜色校正，或者是其他的颜色和亮度处理方式，第二图像增强处理可以将白平衡过程与乘以亮度下降比值过程分成两步来做，例如先将RGB通道的像素值分别乘以亮度下降比值，然后再将R通道和B通道进行白平衡处理，或者先将R通道和B通道进行白平衡处理，然后再RGB通道的像素值分别乘以亮度下降比值，或者只将RGB通道的像素值分别乘以亮度下降比值，然后不经过白平衡直接进行G通道恢复和第二颜色校正等等，均属于本发明的保护范围之内。
如图6所示，本发明实施例还提供一种车牌图像增强系统，用于实现所述车牌图像增强方法，所述系统包括：
图像提取模块M100，用于输入车牌扩展区域图像，所述车牌扩展区域图像包括车牌区域和环绕所述车牌区域的外围区域；
第一处理模块M200，用于将所述车牌扩展区域图像进行第一图像增强处理，所述第一图像增强处理用于将所述车牌扩展区域图像进行颜色和亮度恢复；
第二处理模块M300，用于将所述车牌扩展区域图像进行第二图像增强处理，得到第二增强图像，所述第二图像增强处理用于将所述车牌扩展区域图像进行抑制曝光的图像增强处理；
图像融合模块M400，用于将所述第一增强图像和所述第二增强图像进行图像融合，得到融合后的车牌增强图像，其中，所述车牌增强图像包括车牌区域和外围区域，所述车牌区域采用所述第二增强图像的图像数据填充，所述外围区域采用所述第一增强图像的图像数据填充。
通过采用本发明所提供的车牌图像增强系统，首先通过图像提取模块M100提取得到车牌扩展区域图像，并且通过第一处理模块M200和第二处理模块M200分别对车牌扩展图像进行第一图像增强和第二图像增强，即使车牌区域定位有偏差，由于车牌扩展图像的外围区域的补充，可以保证获得完整的车牌信息，而不至于因为车牌定位不准而缺少车牌信息。进一步地，本发明通过第一处理模块M200和第二处理模块M300，通过对车牌图像增强采用两种处理手段，分别获得第一增强图像(进行图像颜色和亮度恢复)和第二增强图像(进行抑制曝光的图像颜色和亮度恢复)，并且将 第一增强图像和第二增强图像进行融合，使得最终得到的车牌增强图像中车牌区域和外围区域中采用不同的增强方法，由于在图像融合模块M400中融合时，车牌区域采用亮度较低的第二增强图像的数据进行填充，可以避免现有技术中对整张图像采用同样的图像增强方法而导致车牌区域过曝的问题，因此，该图像增强方法可以适用于更多的应用场景，例如可以适用于车牌图像整体亮度很低的场景或者车牌图像整体过曝的场景等。该车牌图像增强系统可以通过一台服务器或普通的个人计算机等来实现，例如将该车牌图像增强系统运行在ARM平台的CPU上。
本发明的车牌图像增强系统可以为若干后续处理模块提供可调用的参数接口，通过参数的调节和设置，可以对增强后的车牌进行色调调节、饱和度调节、清晰度调节、亮度调节，提高了后续的车牌增强技术应用的适应性。
本发明实施例还提供一种车牌图像增强设备，包括处理器；存储器，其中存储有所述处理器的可执行指令；其中，所述处理器配置为经由执行所述可执行指令来执行所述的车牌图像增强方法的步骤。
所属技术领域的技术人员能够理解，本发明的各个方面可以实现为系统、方法或程序产品。因此，本发明的各个方面可以具体实现为以下形式，即：完全的硬件实施方式、完全的软件实施方式(包括固件、微代码等)，或硬件和软件方面结合的实施方式，这里可以统称为“电路”、“模块”或“系统”。
下面参照图7来描述根据本发明的这种实施方式的电子设备600。图7显示的电子设备600仅仅是一个示例，不应对本发明实施例的功能和使用范围带来任何限制。
如图7所示，电子设备600以通用计算设备的形式表现。电子设备600的组件可以包括但不限于：至少一个处理单元610、至少一个存储单元620、连接不同系统组件(包括存储单元620和处理单元610)的总线630、显示单元640等。
其中，所述存储单元存储有程序代码，所述程序代码可以被所述处理单元610执行，使得所述处理单元610执行本说明书上述电子处方流转处理方法部分中描述的根据本发明各种示例性实施方式的步骤。例如，所述处理单元610可以执行如图1中所示的步骤。
所述存储单元620可以包括易失性存储单元形式的可读介质，例如随机存取存储单元(RAM)6201和/或高速缓存存储单元6202，还可以进一步包括只读存储单元(ROM)6203。
所述存储单元620还可以包括具有一组(至少一个)程序模块6205的程序/实用 工具6204，这样的程序模块6205包括但不限于：操作系统、一个或者多个应用程序、其它程序模块以及程序数据，这些示例中的每一个或某种组合中可能包括网络环境的实现。
总线630可以为表示几类总线结构中的一种或多种，包括存储单元总线或者存储单元控制器、外围总线、图形加速端口、处理单元或者使用多种总线结构中的任意总线结构的局域总线。
电子设备600也可以与一个或多个外部设备700(例如键盘、指向设备、蓝牙设备等)通信，还可与一个或者多个使得用户能与该电子设备600交互的设备通信，和/或与使得该电子设备600能与一个或多个其它计算设备进行通信的任何设备(例如路由器、调制解调器等等)通信。这种通信可以通过输入/输出(I/O)接口650进行。并且，电子设备600还可以通过网络适配器660与一个或者多个网络(例如局域网(LAN)，广域网(WAN)和/或公共网络，例如因特网)通信。网络适配器660可以通过总线630与电子设备600的其它模块通信。应当明白，尽管图中未示出，可以结合电子设备600使用其它硬件和/或软件模块，包括但不限于：微代码、设备驱动器、冗余处理单元、外部磁盘驱动阵列、RAID系统、磁带驱动器以及数据备份存储系统等。
通过采用本发明所提供的车牌图像增强设备，所述处理器在执行所述可执行指令时执行所述的车牌图像增强方法，由此可以获得上述车牌图像增强方法的有益效果，即一方面可以保证获得完整的车牌信息，另一方面可以适用于更多的应用场景。
本发明实施例还提供一种计算机可读存储介质，用于存储程序，所述程序被执行时实现所述的车牌图像增强方法的步骤。在一些可能的实施方式中，本发明的各个方面还可以实现为一种程序产品的形式，其包括程序代码，当所述程序产品在终端设备上运行时，所述程序代码用于使所述终端设备执行本说明书上述电子处方流转处理方法部分中描述的根据本发明各种示例性实施方式的步骤。
参考图8所示，描述了根据本发明的实施方式的用于实现上述方法的程序产品800，其可以采用便携式紧凑盘只读存储器(CD-ROM)并包括程序代码，并可以在终端设备，例如个人电脑上运行。然而，本发明的程序产品不限于此，在本文件中，可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。
所述程序产品可以采用一个或多个可读介质的任意组合。可读介质可以是可读信 号介质或者可读存储介质。可读存储介质例如可以为但不限于电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。可读存储介质的更具体的例子(非穷举的列表)包括：具有一个或多个导线的电连接、便携式盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。
所述计算机可读存储介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了可读程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。可读存储介质还可以是可读存储介质以外的任何可读介质，该可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。可读存储介质上包含的程序代码可以用任何适当的介质传输，包括但不限于无线、有线、光缆、RF等等，或者上述的任意合适的组合。
可以以一种或多种程序设计语言的任意组合来编写用于执行本发明操作的程序代码，所述程序设计语言包括面向对象的程序设计语言—诸如Java、C++等，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算设备上执行、部分地在用户设备上执行、作为一个独立的软件包执行、部分在用户计算设备上部分在远程计算设备上执行、或者完全在远程计算设备或集群上执行。在涉及远程计算设备的情形中，远程计算设备可以通过任意种类的网络，包括局域网(LAN)或广域网(WAN)，连接到用户计算设备，或者，可以连接到外部计算设备(例如利用因特网服务提供商来通过因特网连接)。
通过采用本发明所提供的计算机可读存储介质，其中存储的程序在被执行时实现所述的车牌图像增强方法的步骤，由此可以获得上述车牌图像增强方法的有益效果，即一方面可以保证获得完整的车牌信息，另一方面可以适用于更多的应用场景。
以上内容是结合具体的优选实施方式对本发明所作的进一步详细说明，不能认定本发明的具体实施只局限于这些说明。对于本发明所属技术领域的普通技术人员来说，在不脱离本发明构思的前提下，还可以做出若干简单推演或替换，都应当视为属于本发明的保护范围。

Claims (12)

  1. 一种车牌图像增强方法，其特征在于，包括如下步骤：
    输入车牌扩展区域图像，所述车牌扩展区域图像包括车牌区域和环绕所述车牌区域的外围区域；
    将所述车牌扩展区域图像进行第一图像增强处理，得到第一增强图像；
    将所述车牌扩展区域图像进行第二图像增强处理，得到第二增强图像，所述第二图像增强处理用于将所述车牌扩展区域图像进行抑制曝光的图像增强处理；
    将所述第一增强图像和所述第二增强图像进行图像融合，得到融合后具有车牌区域和外围区域的车牌增强图像，所述车牌区域采用所述第二增强图像的图像数据填充，所述外围区域采用所述第一增强图像的图像数据填充。
  2. 根据权利要求1所述的车牌图像增强方法，其特征在于，所述第一图像增强处理包括第一白平衡处理，所述第二图像增强处理包括第二白平衡处理，所述第二白平衡处理得到的图像的亮度小于所述第一白平衡处理得到的图像的亮度。
  3. 根据权利要求2所述的车牌图像增强方法，其特征在于，所述第一白平衡处理，包括将所述车牌扩展区域图像的像素值乘以预设的白平衡增益系数，得到第一增强图像；
    所述第二白平衡处理，包括将所述车牌扩展区域图像的像素值乘以预设的白平衡增益系数和一预设的亮度下降比值，得到第二增强图像。
  4. 根据权利要求2所述的车牌图像增强方法，其特征在于，所述第二白平衡处理包括：
    将所述车牌扩展区域图像的红色通道和蓝色通道的像素值分别乘以预设的白平衡增益系数和一预设的亮度下降比值；
    将所述车牌扩展区域图像的绿色通道的像素值乘以预设的亮度下降比值，所述亮度下降比值为介于0和1之间的数值；
    所述第二图像增强处理中，所述第二白平衡处理之后，还包括如下步骤：
    对于所述第一增强图像的绿色通道的每个像素G(x,y)，根据红色通道与像素G(x,y)的位置相对应的像素R(x,y)和绿色通道与像素G(x,y)的位置相对应的像素B(x,y)的像素值，调整像素G(x,y)的像素值。
  5. 根据权利要求1所述的车牌图像增强方法，其特征在于，所述车 牌扩展区域图像还包括位于所述车牌区域和所述外围区域之间的过渡区域；
    将所述第一增强图像和所述第二增强图像进行图像融合，得到融合后的车牌增强图像，包括如下步骤：
    采用所述第二增强图像的图像数据填充所述车牌增强图像的车牌区域，采用所述第一增强图像的图像数据填充所述车牌增强图像的外围区域；
    采用如下公式计算所述车牌增强图像的过渡区域的各个像素的像素值：
    Figure PCTCN2020121507-appb-100001
    其中，p(x,y)为所述车牌增强图像的过渡区域中坐标为(x,y)的像素的像素值，p A(x,y)为第一增强图像中坐标为(x,y)的像素的像素值，p B(x,y)为第二增强图像中坐标为(x,y)的像素的像素值，L1为坐标为(x,y)的像素与车牌区域的距离，L2为坐标为(x,y)的像素与外围区域的距离。
  6. 根据权利要求1所述的车牌图像增强方法，其特征在于，将所述输入车牌扩展区域图像之后，还包括车牌扩展区域的亮度调节，所述亮度调节包括如下步骤：
    根据目标亮度和所述车牌扩展区域图像的亮度平均值计算亮度调节比值；
    将所述车牌扩展区域图像中各个像素点的像素值乘以所述亮度调节比值，得到亮度调节后的车牌扩展区域图像；
    所述将所述车牌扩展区域图像进行第一图像增强处理，包括将所述亮度调节后的车牌扩展区域图像进行第一图像增强处理；
    所述将所述车牌扩展区域图像进行第二图像增强处理，包括将所述亮度调节后的车牌扩展区域图像进行第二图像增强处理。
  7. 根据权利要求6所述的车牌图像增强方法，其特征在于，所述亮度调节之前，还包括如下步骤：
    输入车牌区域图像；
    根据所述车牌区域图像确定车牌颜色；
    根据车牌颜色与目标亮度的映射关系，确定所对应的目标亮度。
  8. 根据权利要求7所述的车牌图像增强方法，其特征在于，所述根据所述车牌区域图像确定车牌颜色，包括如下步骤：
    分别统计所述车牌区域图像中红色、绿色和蓝色通道的像素平均值；
    计算红色通道像素平均值和绿色通道像素平均值的比值RG ratio；
    计算蓝色通道像素平均值和绿色通道像素平均值的比值BG ratio；
    根据预设的不同颜色与比值RG ratio以及BG ratio的映射关系，确定车牌颜色。
  9. 根据权利要求1所述的车牌图像增强方法，其特征在于，所述输入车牌扩展区域图像，包括输入raw格式的车牌扩展区域图像；
    所述第一图像增强处理和所述第二图像增强处理均为直接对所述raw格式的车牌扩展区域图像进行的处理。
  10. 一种车牌图像增强系统，其特征在于，应用于权利要求1至9中任一项所述的车牌图像增强方法，所述系统包括：
    图像提取模块，用于输入车牌扩展区域图像，所述车牌扩展区域图像包括车牌区域和环绕所述车牌区域的外围区域；
    第一处理模块，用于将所述车牌扩展区域图像进行第一图像增强处理；
    第二处理模块，用于将所述车牌扩展区域图像进行第二图像增强处理，得到第二增强图像，所述第二图像增强处理用于将所述车牌扩展区域图像进行抑制曝光的图像增强处理；
    图像融合模块，用于将所述第一增强图像和所述第二增强图像进行图像融合，得到融合后的车牌增强图像，其中，所述车牌增强图像包括车牌区域和外围区域，所述车牌区域采用所述第二增强图像的图像数据填充，所述外围区域采用所述第一增强图像的图像数据填充。
  11. 一种车牌图像增强设备，其特征在于，包括：
    处理器；
    存储器，其中存储有所述处理器的可执行指令；
    其中，所述处理器配置为经由执行所述可执行指令来执行权利要求1至9中任一项所述的车牌图像增强方法的步骤。
  12. 一种计算机可读存储介质，用于存储程序，其特征在于，所述程序被执行时实现权利要求1至9中任一项所述的车牌图像增强方法的步骤。
PCT/CN2020/121507 2020-06-29 2020-10-16 车牌图像增强方法、系统、设备及存储介质 WO2022000860A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010604429.7 2020-06-29
CN202010604429.7A CN111754440B (zh) 2020-06-29 2020-06-29 车牌图像增强方法、系统、设备及存储介质

Publications (1)

Publication Number Publication Date
WO2022000860A1 true WO2022000860A1 (zh) 2022-01-06

Family

ID=72676730

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/121507 WO2022000860A1 (zh) 2020-06-29 2020-10-16 车牌图像增强方法、系统、设备及存储介质

Country Status (2)

Country Link
CN (1) CN111754440B (zh)
WO (1) WO2022000860A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114511462A (zh) * 2022-02-11 2022-05-17 电子科技大学 一种视觉图像增强方法
CN114663842A (zh) * 2022-05-25 2022-06-24 深圳比特微电子科技有限公司 图像融合处理方法、装置、电子设备及存储介质
CN118229984A (zh) * 2024-03-15 2024-06-21 珠海安联锐视科技股份有限公司 一种车牌识别图像增强方法

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111754440B (zh) * 2020-06-29 2023-05-05 苏州科达科技股份有限公司 车牌图像增强方法、系统、设备及存储介质
CN113379869B (zh) * 2021-07-23 2023-03-24 浙江大华技术股份有限公司 一种车牌图像生成方法、装置、电子设备及存储介质
CN113793291B (zh) * 2021-10-14 2023-08-08 杭州海康威视数字技术股份有限公司 一种图像融合方法、装置、电子设备及存储介质
CN115526786B (zh) * 2022-01-25 2023-10-20 荣耀终端有限公司 图像处理方法及其相关设备
CN114549383A (zh) * 2022-02-23 2022-05-27 浙江大华技术股份有限公司 一种基于深度学习的图像增强方法、装置、设备及介质
CN115128570B (zh) * 2022-08-30 2022-11-25 北京海兰信数据科技股份有限公司 一种雷达图像的处理方法、装置及设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1085455A2 (en) * 1999-09-15 2001-03-21 Siemens Corporate Research, Inc. License plate recognition with an intelligent camera
CN103247036A (zh) * 2012-02-10 2013-08-14 株式会社理光 多曝光图像融合方法和装置
CN110599433A (zh) * 2019-07-30 2019-12-20 西安电子科技大学 一种基于动态场景的双曝光图像融合方法
CN110837826A (zh) * 2019-10-24 2020-02-25 深圳市天视通电子科技有限公司 一种自适应强光照射的车牌识别方法及相关装置
CN111754440A (zh) * 2020-06-29 2020-10-09 苏州科达科技股份有限公司 车牌图像增强方法、系统、设备及存储介质

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108205671A (zh) * 2016-12-16 2018-06-26 浙江宇视科技有限公司 图像处理方法及装置
CN110390643B (zh) * 2018-04-20 2022-04-26 杭州海康威视数字技术股份有限公司 一种车牌增强方法、装置及电子设备
CN109714582B (zh) * 2019-01-04 2021-08-06 Oppo广东移动通信有限公司 白平衡调整方法、装置、存储介质及终端
CN110070499A (zh) * 2019-03-14 2019-07-30 北京字节跳动网络技术有限公司 图像处理方法、装置和计算机可读存储介质
CN110022469B (zh) * 2019-04-09 2021-03-02 Oppo广东移动通信有限公司 图像处理方法、装置、存储介质及电子设备
CN110400270B (zh) * 2019-07-05 2023-05-12 常州大学 一种利用图像分解和多重校正融合的车牌去雾方法
CN110718069B (zh) * 2019-10-10 2021-05-11 浙江大华技术股份有限公司 图像亮度调节方法、装置和存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1085455A2 (en) * 1999-09-15 2001-03-21 Siemens Corporate Research, Inc. License plate recognition with an intelligent camera
CN103247036A (zh) * 2012-02-10 2013-08-14 株式会社理光 多曝光图像融合方法和装置
CN110599433A (zh) * 2019-07-30 2019-12-20 西安电子科技大学 一种基于动态场景的双曝光图像融合方法
CN110837826A (zh) * 2019-10-24 2020-02-25 深圳市天视通电子科技有限公司 一种自适应强光照射的车牌识别方法及相关装置
CN111754440A (zh) * 2020-06-29 2020-10-09 苏州科达科技股份有限公司 车牌图像增强方法、系统、设备及存储介质

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114511462A (zh) * 2022-02-11 2022-05-17 电子科技大学 一种视觉图像增强方法
CN114511462B (zh) * 2022-02-11 2023-04-18 电子科技大学 一种视觉图像增强方法
CN114663842A (zh) * 2022-05-25 2022-06-24 深圳比特微电子科技有限公司 图像融合处理方法、装置、电子设备及存储介质
CN118229984A (zh) * 2024-03-15 2024-06-21 珠海安联锐视科技股份有限公司 一种车牌识别图像增强方法

Also Published As

Publication number Publication date
CN111754440B (zh) 2023-05-05
CN111754440A (zh) 2020-10-09

Similar Documents

Publication Publication Date Title
WO2022000860A1 (zh) 车牌图像增强方法、系统、设备及存储介质
US20100278423A1 (en) Methods and systems for contrast enhancement
US10290081B2 (en) System for image dehazing by modifying lower bound of transmittance and method therefor
CN102202163B (zh) 一种监控视频的自适应增强方法及其装置
CN101950416B (zh) 基于双边滤波的实时图像去雾增强方法
US8639050B2 (en) Dynamic adjustment of noise filter strengths for use with dynamic range enhancement of images
WO2016206087A1 (zh) 一种低照度图像处理方法和装置
US20100149420A1 (en) Method and apparatus for improving automatic white balance with scene information
CN110827225A (zh) 一种基于双重曝光框架的非均匀光照水下图像增强方法
US8675102B2 (en) Real time denoising of video
CN107862672B (zh) 图像去雾的方法及装置
US10609303B2 (en) Method and apparatus for rapid improvement of smog/low-light-level image using mapping table
WO2021004066A1 (zh) 图像处理方法及装置
US20230306569A1 (en) Image processing method and an image processing device
WO2020119454A1 (zh) 一种对图像进行颜色还原的方法和装置
US10070109B2 (en) Highlight recovery in images
CN106408520B (zh) 一种高颜色保真度的图像去雾方法
CN111476744A (zh) 一种基于分类及大气成像模型的水下图像增强方法
CN110717864B (zh) 一种图像增强方法、装置、终端设备及计算机可读介质
CN105791710B (zh) 一种信号灯图像增强处理方法
CN107277369A (zh) 图像处理方法、装置、计算机可读存储介质和计算机设备
CN109509161B (zh) 图像增强装置及图像增强方法
CN108898566B (zh) 一种利用时空照度图的低照度彩色视频增强方法
CN115272103A (zh) 一种沙尘图像清晰化方法、系统、介质、设备及终端
CN112788321B (zh) 图像颜色恢复方法和装置、摄像装置、存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20942984

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20942984

Country of ref document: EP

Kind code of ref document: A1