CN114998135A - Image enhancement method and device, field programmable logic gate array and equipment - Google Patents

Image enhancement method and device, field programmable logic gate array and equipment Download PDF

Info

Publication number
CN114998135A
CN114998135A CN202210612662.9A CN202210612662A CN114998135A CN 114998135 A CN114998135 A CN 114998135A CN 202210612662 A CN202210612662 A CN 202210612662A CN 114998135 A CN114998135 A CN 114998135A
Authority
CN
China
Prior art keywords
value
image
pixel
luminance
domain
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210612662.9A
Other languages
Chinese (zh)
Inventor
李玉霞
李瞻
邢宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Yili Technology Co ltd
Original Assignee
Beijing Yili Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Yili Technology Co ltd filed Critical Beijing Yili Technology Co ltd
Priority to CN202210612662.9A priority Critical patent/CN114998135A/en
Publication of CN114998135A publication Critical patent/CN114998135A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/90Dynamic range modification of images or parts thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00General purpose image data processing
    • G06T1/20Processor architectures; Processor configuration, e.g. pipelining
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00General purpose image data processing
    • G06T1/60Memory management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/40Image enhancement or restoration using histogram techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00Indexing scheme for image data processing or generation, in general
    • G06T2200/28Indexing scheme for image data processing or generation, in general involving image processing hardware
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10004Still image; Photographic image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10024Color image

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

The disclosure relates to an image enhancement method, an image enhancement device, a field programmable gate array and a field programmable gate array device. Obtaining a luminance domain representation of an image, the luminance domain representation being used to characterize a distribution of pixel values of the image in a luminance domain; calculating a stretch coefficient based on the luminance domain representation; and updating pixel values of the image in the luminance domain based on the stretch coefficient. Therefore, the overall brightness of the image can be adjusted without influencing the color of the image (namely, without distortion), so that the contrast is enhanced, the calculation amount can be reduced to a certain extent, and the method is friendly to hardware.

Description

Image enhancement method and device, field programmable logic gate array and equipment
Technical Field
The present disclosure relates to the field of image enhancement technologies, and in particular, to an image enhancement method and apparatus, a field programmable gate array, and a device suitable for enhancing an image with low contrast and improper exposure.
Background
The image enhancement technology is suitable for scenes for enhancing the image quality and a preprocessing part of various image processing and analyzing modules. The technology improves the quality of the image by emphasizing or changing the local or overall characteristics of the image, and adapts to special needs. Image enhancement techniques have therefore seen rapid development and progress in recent years.
Histogram equalization is a more common technique in image enhancement. The histogram of an image is a graphical representation of the intensity and probability of occurrence of intensity values in a given image. Image histogram equalization is a method of equalizing the occurrence probability distribution of intensity values in an image. Due to the limitation of the camera device or the characteristics of the scene, the captured image pixels are concentrated in some parts of the pixel value range, so that the image is dim, has no layering sense, and cannot truly reproduce the scene information, namely, the image contrast is low. The histogram of the image describes the distribution condition of the pixel values of the image on the value range, and the distribution characteristics of the pixel values of the image are shown. The above-mentioned problem of low image contrast can be represented by constructing a histogram of image pixel values. Meanwhile, the histogram of the image is adjusted, so that the contrast and other relevant characteristics of the image can be changed.
The current color image enhancement methods using histogram equalization mainly include the following:
1. a mapping relationship is generated based on the cumulative histogram of image color values to enhance image contrast. In order to protect the overall brightness or achieve other purposes, the method needs to divide sub-regions on the cumulative histogram and calculate cumulative density functions respectively, the sub-regions are more in excess in the dividing process, the manual selection difficulty is higher, and the method is difficult to adapt to different scenes.
2. Using a locally adaptive scheme, digital images exhibiting improved exposure and tone uniformity are generated by employing histogram equalization on a local basis, even though stretching and compression of pixel brightness levels adapts to the local distribution of pixels in the pixels. The method has a good effect of adjusting local areas, but discontinuous edges are easily generated among different areas, and the overall effect is not good because the local effect is emphasized.
3. The color tone of the RGB channels is changed by performing histogram equalization on the RGB channels in the RGB domain color space, so a color tone preserving module is also needed to be added, for example, wavelet detail coefficients of an image are preserved by using wavelet transform and the result after histogram equalization is reconstructed. This type of method is computationally complex and introduces problems of tonal distortion.
Therefore, the existing histogram equalization implementation scheme has the problem of image distortion or overlarge calculated amount, so that an image enhancement method which has a good effect, is friendly to hardware and can realize real-time processing is needed.
Disclosure of Invention
The technical problem to be solved by the present disclosure is to provide an image enhancement method which has a good effect, is suitable for different scenes, is hardware-friendly, and can realize real-time processing.
According to a first aspect of the present disclosure, there is provided an image enhancement method, comprising: acquiring a brightness domain representation of the image, wherein the brightness domain representation is used for representing the pixel value distribution of the image in the brightness domain; calculating a stretch coefficient based on the luminance domain representation; and updating pixel values of the image in a luminance domain based on the stretch coefficient.
Optionally, the step of calculating the stretch coefficient based on the luminance domain representation comprises: calculating a luminance domain pixel mean based on the luminance domain representation; and calculating a tensile coefficient based on the luminance domain pixel average value and the expected luminance value, wherein the magnitude of the tensile coefficient is in negative correlation with the luminance domain pixel average value and in positive correlation with the expected luminance value.
Alternatively, the stretch coefficient k is calculated based on the following formula,
Figure BDA0003673410290000021
or
Figure BDA0003673410290000022
Where b is the mean value of pixels in the luminance domain, y is the expected luminance value, and p (x) represents the ratio of pixels with pixel values of x in the image.
Optionally, the method further comprises: the value of the stretch coefficient is limited to a first value interval.
Optionally, the method further comprises: and calculating a first value interval based on the value range limiting parameter and the pixel mean value in the brightness domain.
Optionally, the first value interval is [ (1- μ) × b, μ + (1- μ) × b ], where μ is a value range limitation parameter, and b is a luminance domain pixel mean value.
Optionally, the step of updating the pixel values of the image in the luminance domain based on the stretch coefficient comprises: and adjusting the pixel value based on the pixel value, the luminance domain pixel mean value and the stretching coefficient.
Optionally, the pixel value x is adjusted based on the following formula i
Figure BDA0003673410290000023
Wherein, y i For the adjusted pixel value, k is the stretch coefficient, b is the mean value of the pixels in the luminance domain, x min Is the minimum pixel value, x, of the image in the luminance domain max Is the maximum pixel value of the image in the luminance domain.
Optionally, the method further comprises: the operation involved in the execution process of the method is quantized into fixed point number operation, and the method is delivered to a field programmable gate array for execution.
Optionally, the field programmable gate array employs a multi-port BRAM memory, the field programmable gate array divides the depth of the multi-port BRAM memory into at least two sub-depths, each sub-depth corresponds to at least 256 storage addresses, adjacent pixel columns in an image are allocated to different sub-depths, the field programmable gate array takes the pixel value of the read image in a brightness domain as an address, reads the sub-depth allocated to the pixel column where the pixel value is located, adds 1 to the read value and writes back the value to the original address, after reading all pixels, the field programmable gate array adds the values stored in the storage addresses corresponding to the same pixel value in different sub-depths, and takes the addition result as the number of the pixel value to obtain the brightness domain representation.
According to a second aspect of the present disclosure, there is also provided an image enhancement apparatus, including: an acquisition module configured to acquire a luminance domain representation of an image, the luminance domain representation being used to characterize a distribution of pixel values of the image in a luminance domain; a calculation module for calculating a stretch coefficient based on the pixel value distribution; and an updating module for updating pixel values of the image in the luminance domain based on the stretch coefficient.
According to a third aspect of the present disclosure, there is also provided a field programmable gate array for performing the method of the first aspect, wherein the operation involved in the method is a fixed-point number operation obtained through quantization.
Optionally, the field programmable gate array employs a multi-port BRAM memory, the field programmable gate array divides the depth of the multi-port BRAM memory into at least two sub-depths, each sub-depth corresponds to at least 256 storage addresses, adjacent pixel columns in an image are allocated to different sub-depths, the field programmable gate array takes the pixel value of the read image in a brightness domain as an address, reads the sub-depth allocated to the pixel column where the pixel value is located, adds 1 to the read value and writes back the value to the original address, after reading all pixels, the field programmable gate array adds the values stored in the storage addresses corresponding to the same pixel value in different sub-depths, and takes the addition result as the number of the pixel value to obtain the brightness domain representation.
According to a fourth aspect of the present disclosure, there is also provided an imaging apparatus including: the method comprises the steps of carrying out the method of the first aspect to enhance the image shot by the imaging device, and carrying out the field programmable logic gate array, wherein the operation involved in the method is a fixed point number operation obtained through quantification.
According to a fifth aspect of the present disclosure, there is provided a computing device comprising: a processor; and a memory having executable code stored thereon, which when executed by the processor, causes the processor to perform the method as described in the first aspect above.
According to a sixth aspect of the present disclosure, there is provided a computer program product comprising executable code which, when executed by a processor of an electronic device, causes the processor to perform the method as described in the first aspect above.
According to a seventh aspect of the present disclosure, there is provided a non-transitory machine-readable storage medium having stored thereon executable code, which when executed by a processor of an electronic device, causes the processor to perform the method as described in the first aspect above.
Therefore, the present disclosure obtains the brightness domain representation of the image, and adjusts the pixel value of the image in the brightness domain by using the stretching coefficient calculated based on the brightness domain representation, so that the contrast enhancement can be realized, the calculation amount can be reduced to a certain extent, and the present disclosure is hardware-friendly while the image color is not affected (i.e., is not distorted).
Drawings
The foregoing and other objects, features and advantages of the disclosure will be apparent from the following more particular descriptions of exemplary embodiments of the disclosure as illustrated in the accompanying drawings wherein like reference numbers generally represent like parts throughout the exemplary embodiments of the disclosure.
Fig. 1 shows a schematic flow diagram of an image enhancement method according to an embodiment of the present disclosure.
Fig. 2 shows the visualization effect of the pixel value adjustment formula.
Fig. 3 shows an overall processing flow diagram of hardware.
Fig. 4 shows a schematic diagram of data stream transmission and BRAM instantiation.
Fig. 5A shows an overall darker image.
FIG. 5B is an image after contrast enhancement of the image shown in FIG. 5A using the present disclosure.
Fig. 6 shows a schematic structural diagram of an image enhancement apparatus according to an embodiment of the present disclosure.
Fig. 7 shows a schematic structural diagram of an imaging device according to an embodiment of the present disclosure.
FIG. 8 shows a schematic structural diagram of a computing device, according to one embodiment of the present disclosure.
Detailed Description
Preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While the preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
For the problems of low contrast and improper exposure (dark or over-bright) of an image, the present disclosure proposes that a component of the image in a luminance domain (i.e., a luminance component or a luminance domain component) may be obtained, and the luminance component is adjusted (stretched) in consideration of the luminance level of the image, so as to adjust the overall luminance of the image while not affecting the color of the image (i.e., without distortion), to achieve contrast enhancement, and to reduce the amount of computation to a certain extent, and to be hardware-friendly.
Fig. 1 shows a schematic flow diagram of an image enhancement method according to an embodiment of the present disclosure.
Referring to fig. 1, in step S110, a luminance domain representation of an image is acquired.
The luminance domain represents a distribution of pixel values used to characterize an image in the luminance domain (i.e., the luminance channel). The pixel value distribution, that is, the distribution of the pixel values, may be the number of pixels per pixel value or the pixel ratio per pixel value, for example.
The pixel values of the image in the luminance domain reflect the luminance values of the image pixels.
The luminance domain representation may be a component of the image in the luminance domain, i.e. a luminance component or a luminance domain component of the image.
The luminance domain representation may also be a statistical result of counting the luminance components of the image.
For example, the luminance domain representation may be a histogram that is statistical of the luminance components of the image. The histogram marks the number of each pixel value of the image in the luminance domain, or the fraction of the number of each pixel value among all the pixel numbers.
The image refers to an image to be subjected to image enhancement, and specifically may be an image with low contrast and improper exposure, such as a relatively dim or excessively bright image.
The image may be an RGB image or an image in other color spaces (such as HSV, YUV, YCbCr). There is no luminance component in the RGB color space, so if the image is an image in a color space without luminance component, such as RGB, it is necessary to convert the image to another color space with luminance component to obtain a luminance domain representation of the image.
For example, HSV (hue, saturation, brightness) is a color space created by a.r.smith in 1978 according to the intuitive characteristics of colors, also called a hexagonal pyramid Model (Hexcone Model), in which a V channel is a luminance channel, and thus an RGB image can be converted into an HSV color space.
For another example, in the YUV color space, "Y" represents brightness (Luma), i.e., a gray scale value; "U" and "V" represent Chrominance (or Chroma), where the Y channel is the luminance channel and "luminance" is derived by linear transformation of the RGB input signal, so that the RGB image can also be converted into YUV color space.
In step S120, a stretch coefficient is calculated based on the luminance domain representation.
The stretching coefficient is used for carrying out contrast enhancement on the image so as to solve the problems of low contrast and improper exposure of the image. The reason why the image contrast is low is mainly that the image pixels are concentrated in some parts of the pixel value range, so that the image is dim or too bright, has no layering sense, and cannot truly reproduce the scene information, i.e. the image contrast is low.
The "stretching" in the stretching coefficient mentioned in the present disclosure refers to stretching the originally concentrated pixel values of the image in the luminance domain to a broader value range of the pixel value domain (e.g. 0 to 255, or 0 to 1 after normalization), and by equalizing the pixel value distribution, the contrast of the image is improved.
In other words, the stretch coefficient is used to adjust the pixel value distribution of the image in the luminance domain. "stretch" in the stretch coefficient is a broad stretch that refers to stretching the distribution of pixel values characterized by the luminance domain representation of the image to enhance contrast, and is not limited to having to stretch each pixel value. That is, when the stretch factor is used to adjust a specific pixel value, not only the pixel value may be turned up (stretched), but also the pixel value may be turned down (scaled).
When contrast enhancement is performed on an image based on a stretch coefficient, it is necessary to consider and protect the original brightness of the image to some extent so as to avoid brightness distortion. To this end, the present disclosure proposes that the stretch coefficient may be calculated based on the luminance domain representation, so that the value of the stretch coefficient is highly correlated with the luminance level of the image, and thus may play a role in protecting luminance.
As an example, the luminance domain pixel mean value may be calculated based on the luminance domain representation, and the stretch coefficient may be calculated based on the luminance domain pixel mean value and the desired luminance value. Wherein the magnitude of the stretch coefficient is inversely related to the average value of the pixels in the brightness domain and positively related to the expected value of the brightness. The luminance domain pixel mean value is used to characterize the average value of all pixel values of the image in the luminance domain. The brightness expected value is used for representing the brightness average value expected after the pixel values of the image in the brightness domain are adjusted. The stretch factor may be used to bring the luminance domain pixel average of the updated image closer to the desired luminance value than before the update.
The expected brightness value is generally 0.5 (the result after numerical value normalization), that is, half of the value range of the image pixel value, and if the average value of the pixels in the brightness range is near the value, the overall visual effect of the image is better from the visual point of view. The expected brightness value generally does not vary, i.e., does not need to change as the image changes. However, the brightness desired value can still be designed as an adjustable parameter, and in practical application, if the image is desired to be darker or lighter, the brightness desired value can be adjusted.
Alternatively, the stretch coefficient may be calculated based on the difference between the luminance domain pixel mean and the luminance desired value. The stretching coefficient can be designed such that the value of the stretching coefficient is larger when the average value of the pixels in the luminance domain is lower than the expected luminance value than when the average value of the pixels in the luminance domain is higher than the expected luminance value, and the value of the stretching coefficient is inversely related to the average value of the pixels in the luminance domain.
For example, when the luminance domain pixel average value is lower than the luminance desired value, the value of the stretch coefficient is larger, and the lower the luminance domain pixel average value is, the larger the value of the stretch coefficient is; and conversely, when the image pixel average value is higher than the expected value, the value of the stretch coefficient is smaller, and the value of the stretch coefficient is smaller when the brightness domain pixel average value is higher.
For example, the stretch coefficient k may be calculated based on the following formula
Figure BDA0003673410290000061
b is the mean value of the pixels in the luminance domain, y is the expected luminance value, and p (x) represents the proportion of the pixels with the pixel value of x in the image. The luminance domain pixel value in the formula is a normalization result, the highest pixel value is 1, and the lowest pixel value is 0.
(1-b) y represents the product of the difference between the highest pixel value and the mean value of the pixels in the luminance domain and the expected value of luminance;
Figure BDA0003673410290000062
a pixel average value representing all pixels whose pixel values are greater than the luminance domain pixel average value;
Figure BDA0003673410290000063
representing the product of the pixel ratio larger than the mean value of the pixels in the luminance domain and the mean value of the pixels in the luminance domain;
Figure BDA0003673410290000064
a pixel average value representing all pixels having pixel values smaller than the luminance domain pixel average value;
Figure BDA0003673410290000065
representing the mean of the pixels in the luminance domain.
For easy understanding, the calculation formula (1) of the stretch coefficient k can be simplified to
Figure BDA0003673410290000066
Wherein
Figure BDA0003673410290000067
Delta is the integral of (x-b) p (x) between b and 1, i.e. the average of the parts of the pixel values above the mean value of the luminance domain pixels of all pixels.
The luminance domain pixel mean value b and the luminance expected value y in the formula are floating point numbers between 0 and 1 and are dimensionless units, so that the finally calculated stretching coefficient k is a dimensionless unit.
As can be seen from equation (2), the magnitude of the stretch coefficient k is calculated based on the ratio of the luminance-domain pixel average value b to the expected luminance value y, and the luminance-domain pixel average value b. When the luminance domain pixel average value b is lower than the luminance expected value y, the k value is larger, and the lower the luminance domain pixel average value b is, the larger the k value is; on the contrary, when the pixel mean value b in the brightness domain is higher than the expected brightness value y, the value k is smaller, and the higher the pixel mean value b in the brightness domain is, the smaller k is; when the luminance domain pixel average value b is equal to the desired luminance value y, the value k is equal to the desired luminance value y.
The calculation principle of the stretch coefficient k can be summarized as that the stretch coefficient k is calculated according to the difference between the image brightness level and the expected brightness level (i.e., the expected brightness value), and the stretch coefficient k is highly correlated with the image brightness level and plays a role in protecting the brightness. In contrast, currently mainstream contrast enhancement algorithms do not take into account the original image brightness level.
Considering that the image is changed too obviously due to too large or too small stretching coefficient, and local overexposure or darkness occurs, the disclosure proposes that the value of the stretching coefficient can be limited in a first value interval, and the first value interval can be regarded as a reasonable value range. The first numerical range may be a range set in advance regardless of the image, or may be a range determined based on the luminance domain representation of the image.
As an example, the first value interval may be calculated based on the value range limiting parameter and the luminance domain pixel mean value. The value range limiting parameter is used for limiting the value range of the stretching coefficient, and in practical application, the value of the value range limiting parameter can be adjusted according to the algorithm effect, for example, if an image processed by the algorithm is too bright, the value of the value range limiting parameter can be properly reduced to inhibit the stretching effect.
As mentioned above, the range limiting parameter sets the range of the stretch coefficient. This value was 0.25 in actual tests. The value will generally not vary. But the value is still adjustable, which in some extreme cases affects the actual stretching effect. The scene is too dim or too bright, and the value can be increased appropriately to enhance the stretching effect; conversely, if the output image is too dim or bright, the value may be reduced, reducing the stretching effect.
The first value interval may be [ (1- μ) × b, μ + (1- μ) × b ], where μ is a value range limiting parameter and b is a luminance domain pixel mean value. Therefore, the limit range of k can be calculated and k can be updated according to the original input image mean value b and the value range limiting parameter mu
Figure BDA0003673410290000071
Wherein, when (1- μ) < k < μ + (1- μ) × b, the value of k is equal to the value calculated according to the above-mentioned formula (1) or (2) for the calculation of the tensile coefficient k.
In step S130, the pixel values of the image in the luminance domain are updated (i.e., adjusted) based on the stretch coefficient.
Adjustments may be made for at least some (e.g., all) of the pixel values of the image in the luminance domain based on the stretch coefficient. For the pixel values with different values, a nonlinear adjustment mode can be used for adjustment so as to uniformly stretch the pixel values of the image in the brightness domain to the pixel value domain. As an example, the pixel values may be adjusted based on the pixel values, the luminance domain pixel mean, and the stretch coefficients. For example, the pixel value x may be adjusted based on the following formula i
Figure BDA0003673410290000072
Wherein, y i For the adjusted pixel value, k is the stretch coefficient, b is the mean value of the pixels in the luminance domain, x min Is the minimum pixel value, x, of the image in the luminance domain max Is the maximum pixel value of the image in the luminance domain. The luminance domain pixel value in the formula is a normalization result, the highest pixel value is 1, and the lowest pixel value is 0.
The pixel value adjustment formula (4) has the effect of uniformly extending the pixel values of the image to the range of [0,1 ]. When the pixel value is smaller than the image pixel value mean value, the formula compares the input pixel value with the mean value b, and distributes the input pixel value to [0, k ]; otherwise, the stretching is carried out to the range of [ k,1 ]. Fig. 2 shows the visualization effect of the formula. As shown in fig. 2, the abscissa is the pixel value before stretching, the ordinate is the pixel value after stretching, and the whole stretching process is nonlinear stretching, where k72_ b11 indicates that the stretching coefficient k is 72, and the luminance domain pixel mean value b is 11.
So far, the image enhancement method of the present disclosure is exemplarily explained with reference to fig. 1.
The present disclosure also provides a hardware adaptation quantization scheme based on the above method.
The method can quantize the operation (such as floating point operation) related in the method execution process into fixed point operation, and transfer the method to hardware for execution, for example, the method quantized into fixed point operation can be transferred to Field Programmable Gate Array (FPGA) for execution.
The FPGA belongs to a semi-custom circuit in an integrated circuit, and is composed of a series of Configurable Logic Blocks (CLBs) and programmable connections. The algorithm can be quickly compiled and burned into a combination of transistor circuits through a hardware description language, so that the deployment and acceleration of the algorithm on a hardware platform can be quickly realized. The method has the characteristics of extremely high calculation energy consumption ratio, rapid iteration, customization and the like, and is particularly suitable for the current artificial intelligence and the related application of a data center.
The method shown in fig. 1 can be implemented by using an FPGA, and the data stream is processed in parallel by using rich configurable resources such as DSP, BRAM, CLB and the like in the FPGA, so that real-time enhancement of video images of types 4K, FPS60 and the like can be implemented.
When the FPGA is used for realizing the method, all parameters related to the method can be limited to nonnegative integers within a pixel value range (such as 0-255), and an integration process is changed into equivalent multiplication and accumulation, so that floating point number operation in the method is quantized into fixed point number operation. The pixel value range refers to a pixel value range of an image to be subjected to image enhancement.
In the method, the image is enhanced by using the above-mentioned calculation formula (1) of the stretch coefficient k, the value limiting formula (3) of the stretch coefficient k, and the pixel value adjusting formula (4), and the scheme after hardware adaptation quantization based on the method is as follows.
1. And (4) giving the expected value y of the image pixel value after histogram equalization and a limiting parameter mu of the value range of the expected value.
2. Counting the number of pixel points corresponding to each pixel value to obtain a pixel value distribution histogram of the image, wherein the value range of the abscissa of the histogram is limited to be 0-255, and if the bit width of the pixel of the image is higher than 8bit, the value range of the abscissa of the histogram can be increased, for example, if the range of the input pixel value is 0-4095, the value range of the abscissa of the histogram is also 0-4095;
3. calculating a pixel value mean value b according to the pixel value of the input image, and taking the high 8 bits b _ofthe bit;
4. calculating an accumulative distribution function by using an accumulative power method according to the following formula;
Figure BDA0003673410290000081
Figure BDA0003673410290000082
Figure BDA0003673410290000083
5. substituting the result of the cumulative distribution function into the following formula, calculating to obtain a histogram stretch coefficient k, and performing fixed point processing on the value after division calculation;
Figure BDA0003673410290000084
6. calculating a limiting range of k according to the original input image mean value b and the value range limiting parameter mu and updating k;
Figure BDA0003673410290000085
7. calculating a new pixel value based on the input pixel value x, the stretching coefficient k and the image mean value b;
Figure BDA0003673410290000086
fig. 3 shows an overall processing flow diagram of hardware.
As shown in fig. 3, the hardware processing flow mainly includes three steps of histogram statistics, k value calculation, and adjustment of new pixel values of low points with respect to pixel values. Taking FPGA as an example, the three steps are specifically implemented as follows.
Step P1, histogram statistics of image data frame by frame
The histogram statistics is to count the number of each pixel value appearing in the luminance domain of one frame of image.
FPGAs may employ multi-port (e.g., dual-port) BRAM memories.
The FPGA can divide the depth of the multiport BRAM memory into at least two sub-depths, each sub-depth corresponds to at least 256 memory addresses, and adjacent pixel columns in the image are allocated to different sub-depths so as to avoid that the BRAM cannot be updated in time when the same data continuously appear.
The FPGA takes the pixel value of the read image in the brightness domain as an address, reads the sub-depth distributed by the pixel column where the pixel value is located, adds 1 to the read value and writes the value back to the original address.
After reading all the pixels, the FPGA adds the numerical values stored in the storage addresses corresponding to the same pixel value in different sub-depths, and takes the addition result as the number of the pixel value, so that the histogram statistical result can be obtained.
The specific implementation steps of the FPGA statistical histogram are as follows:
firstly, taking high 8bit data to obtain a pixel range of 0-255;
instantiating a BRAM with the depth of 512, wherein addresses 0-255 are used for storing the number of even-numbered rows of pixel points, and addresses 256-511 are used for storing the number of odd-numbered rows of pixel points;
thirdly, when even-numbered pixel values are taken, the taken pixel values are taken as addresses to read BRAM addresses 0-255; when the pixel value of the odd column is taken, the taken pixel value is shifted to the left by 1 bit to be used as the address for reading the BRAM address 256-511;
adding 1 to the read data and writing back the corresponding address again;
after storing a whole frame of data, simultaneously reading the data of 0-255 and 256-511 in the BRAM, adding the data of 0 address and the data of 256 addresses in sequence to obtain the number of each pixel value of 0-255;
the data stream transmission and BRAM scaling is shown in fig. 4.
The FPGA adopts the mode that odd-even columns are stored to a true dual-port RAM with the depth of 512 in a crossed mode and front and back 256 addresses, thereby avoiding the use of frequency multiplication clock domain crossing operation, reducing the use of BRAM blocks and saving resources.
As shown in fig. 4, the number of columns of pixel values in the image data may be from the 0 th column, the 0 th column pixel value is read to be 0, and the read pixel value 0 may be read as an address in BRAM addresses 0-255, i.e., address 0 is read, and then the value in address 0 (the initial value is 0) is added by 1 and address 0 is rewritten. Reading the pixel value of the 1 st column as 18, wherein the pixel value belongs to the odd column, shifting the read pixel value 18 by one bit to the left, taking the value 273 obtained by adding 17 and 256 as an address, reading the address 273 in the BRAM address 256-511, adding 1 to the value (the initial value is 0) in the address 273, and rewriting the value into the address 273. And by analogy, the traversal of all pixel values in the image data is completed.
Step P2, calculating the stretch coefficient k according to the histogram statistical result
The FPGA specifically calculates the stretch coefficient k as follows:
the histogram is counted, and meanwhile, the input pixel points are accumulated to obtain an actual pixel sum, and the actual pixel sum is divided by the number of the pixel points of each frame to obtain a pixel mean value b, which is the above mentioned luminance domain pixel mean value.
B is taken as a boundary, the data of the BRAM address 0-b is multiplied by 0-b, and sum0 is obtained through accumulation; accumulating the data of BRAM address b-255 to obtain the sum; and multiplying the data of the BRAM address b-255 by the b-255, and accumulating to obtain sum 1.
Thirdly, according to the formula (5), the divisor and the dividend are obtained separately and sent to a divider to obtain a k value;
fourthly, according to the formula (6), calculating the limit range of k by using the mean value b obtained in the step P2 and updating k;
step P3, calculating the new pixel value of each pixel point of the current frame after stretching according to the formula
The specific steps of the FPGA for calculating the new pixel value are as follows:
firstly, when a histogram is counted, obtaining a maximum pixel xmax and a minimum pixel xmin by comparing the values of input pixels, and registering;
and (II) calculating to obtain a new pixel value according to a formula (7).
Fig. 5A is a relatively dark image as a whole.
For the image shown in fig. 5A, the stretch coefficient k is calculated to be 84, the image pixel mean value b is calculated to be 27, the expected value y of the image pixel value is set to be 127, and the value range limiting parameter μ is set to be 64 according to the above scheme.
The output of the enhancement of the image shown in fig. 5A according to the above scheme is shown in fig. 5B.
It can be seen that for images with lower contrast and overall less brightness, contrast enhancement can be achieved with the present disclosure without resulting in image distortion. In addition, for images with low contrast and excessively bright overall, contrast enhancement can be achieved by the method and the device, and image distortion is not caused.
In summary, the present disclosure provides an image enhancement algorithm based on histogram equalization, which works in a luminance domain, and can protect the luminance of an image while solving the problems of low contrast and improper exposure (e.g., the image is dark or too bright) of the image. The method and the device improve and optimize hardware characteristics, so that real-time processing of the algorithm can be realized on a hardware platform, and particularly, a good histogram equalization effect can still be guaranteed under the application scenes of high resolution and high frame rate.
The image enhancement method of the present disclosure may also be implemented as an image enhancement apparatus. Fig. 6 shows a schematic structural diagram of an image enhancement apparatus according to an embodiment of the present disclosure. The functional elements of the image enhancement apparatus may be implemented by hardware, software, or a combination of hardware and software implementing the principles of the present disclosure. It will be appreciated by those skilled in the art that the functional units described in fig. 6 may be combined or divided into sub-units to implement the inventive principles described above. Thus, the description herein may support any possible combination, or division, or further definition of the functional units described herein.
In the following, brief descriptions are given to functional units that the image enhancement device can have and operations that each functional unit can perform, and details related thereto may be referred to the above description, and are not repeated here.
Referring to fig. 6, the image enhancing apparatus 600 includes an obtaining module 610, a calculating module 620, and an updating module 630.
The obtaining module 610 is configured to obtain a luminance domain representation of the image, where the luminance domain representation is used to characterize a distribution of pixel values of the image in the luminance domain.
The calculation module 620 is configured to calculate a stretch coefficient based on the pixel value distribution.
As an example, the calculation module 620 may calculate a luminance domain pixel mean value based on the luminance domain representation, and calculate a stretch coefficient based on the luminance domain pixel mean value and the expected luminance value, wherein a magnitude of the stretch coefficient is negatively related to the luminance domain pixel mean value and positively related to the expected luminance value. For example, the calculation module may calculate the stretch coefficient using equation (1) mentioned above.
The image enhancement apparatus 600 may further comprise a limiting module for limiting the value of the stretch coefficient to a first value interval.
The image enhancement apparatus 600 may further include a second calculation module, configured to calculate the first value range based on the value range limiting parameter and the luminance domain pixel mean value. For the first value interval, reference is made to the above-mentioned statements.
The updating module 630 is used for updating the pixel values of the image in the luminance domain based on the stretching coefficients.
The update module 630 may adjust the pixel values based on the pixel values, the luminance domain pixel mean, and the stretch coefficients. For example, the update module 630 may adjust the pixel value using the above-mentioned formula (3).
The present disclosure also provides a field programmable gate array, which can quantize the operations involved in the execution process of the above-mentioned method into fixed point number operations, and execute the quantized method.
The present disclosure also provides an image forming apparatus. Fig. 7 shows a schematic structural diagram of an imaging device according to an embodiment of the present disclosure. As shown in fig. 7, the imaging apparatus 700 may include an imaging device 710 and a field programmable gate array 720. The imaging device 710 is used to take an image. The field programmable gate array is used for executing the method shown in the above combined figure 1 to enhance the image obtained by the imaging device, wherein the operation involved in the method is a fixed point number operation obtained through quantification. The specific implementation process of the field programmable gate array can be referred to the above related description.
Fig. 8 shows a schematic structural diagram of a computing device that can be used to implement the image enhancement method described above according to an embodiment of the present disclosure.
Referring to fig. 8, computing device 800 includes memory 810 and processor 820.
The processor 820 may be a multi-core processor or may include multiple processors. In some embodiments, processor 820 may include a general-purpose host processor and one or more special coprocessors such as a Graphics Processor (GPU), a Digital Signal Processor (DSP), or the like. In some embodiments, processor 820 may be implemented using custom circuitry, such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA).
The memory 810 may include various types of storage units, such as system memory, Read Only Memory (ROM), and permanent storage. Wherein the ROM may store static data or instructions for the processor 820 or other modules of the computer. The persistent storage device may be a read-write storage device. The persistent storage may be a non-volatile storage device that does not lose stored instructions and data even after the computer is powered down. In some embodiments, the persistent storage device employs a mass storage device (e.g., a magnetic or optical disk, flash memory) as the persistent storage device. In other embodiments, the permanent storage may be a removable storage device (e.g., floppy disk, optical drive). The system memory may be a read-write memory device or a volatile read-write memory device, such as a dynamic random access memory. The system memory may store instructions and data that some or all of the processors require at runtime. In addition, the memory 810 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), magnetic and/or optical disks, may also be employed. In some embodiments, memory 810 may include a removable storage device that is readable and/or writable, such as a Compact Disc (CD), digital versatile disc read only (e.g., DVD-ROM, dual layer DVD-ROM), Blu-ray disc read only, ultra-compact disc, flash memory card (e.g., SD card, min SD card, Micro-SD card, etc.), magnetic floppy disk, and the like. Computer-readable storage media do not contain carrier waves or transitory electronic signals transmitted by wireless or wired means.
The memory 810 has stored thereon executable code that, when processed by the processor 820, may cause the processor 820 to perform the image enhancement methods described above.
The image enhancement method, apparatus, field programmable gate array, and device according to the present disclosure have been described in detail above with reference to the accompanying drawings.
Furthermore, the method according to the present disclosure may also be implemented as a computer program or computer program product comprising computer program code instructions for performing the above-mentioned steps defined in the above-mentioned method of the present disclosure.
Alternatively, the present disclosure may also be embodied as a non-transitory machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) having stored thereon executable code (or a computer program, or computer instruction code) which, when executed by a processor of an electronic device (or computing device, server, etc.), causes the processor to perform the various steps of the above-described method according to the present disclosure.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems and methods according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or improvements to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims (17)

1. An image enhancement method, comprising:
obtaining a luminance domain representation of an image, the luminance domain representation being used to characterize a distribution of pixel values of the image in a luminance domain;
calculating a stretch coefficient based on the luminance domain representation; and
updating pixel values of the image in the luminance domain based on the stretch coefficient.
2. The method of claim 1, wherein calculating a stretch coefficient based on the luminance domain representation comprises:
calculating a luminance domain pixel mean based on the luminance domain representation;
and calculating the stretching coefficient based on the brightness domain pixel average value and the expected brightness value, wherein the stretching coefficient is in negative correlation with the brightness domain pixel average value and in positive correlation with the expected brightness value.
3. The method according to claim 2, wherein the stretch coefficient k is calculated based on the formula,
Figure FDA0003673410280000011
or
Figure FDA0003673410280000012
Wherein b is the average value of the pixels in the luminance domain, y is the expected luminance value, and p (x) represents the proportion of the pixels with the pixel value of x in the image.
4. The method of claim 1, further comprising:
and limiting the value of the stretching coefficient to a first value interval.
5. The method of claim 4, further comprising:
and calculating the first value interval based on the value range limiting parameter and the pixel mean value in the brightness domain.
6. The method of claim 5, wherein,
the first value interval is [ (1-mu) b, mu + (1-mu) b ], wherein mu is a value range limiting parameter, and b is a brightness domain pixel mean value.
7. The method of claim 1, wherein updating pixel values of the image in the luminance domain based on the stretch coefficient comprises:
and adjusting the pixel value based on the pixel value, the luminance domain pixel mean value and the stretching coefficient.
8. The method of claim 7, wherein the pixel value x is adjusted based on the following formula i
Figure FDA0003673410280000021
Wherein, y i For adjusted pixel valueK is the stretch coefficient, b is the mean value of the pixels in the luminance domain, x min For the minimum pixel value, x, of the image in the luminance domain max Is the maximum pixel value of the image in the luminance domain.
9. The method of any of claims 1 to 8, further comprising:
and quantizing the operation involved in the execution process of the method into fixed point number operation, and handing the method to a field programmable gate array for execution.
10. The method of claim 9, wherein,
the field programmable gate array employs a multiport BRAM memory,
the field programmable gate array divides the depth of the multiport BRAM memory into at least two sub-depths, each sub-depth corresponds to at least 256 memory addresses, adjacent pixel columns in the image are allocated to different sub-depths,
the field programmable gate array takes the pixel value of the read image in the brightness domain as an address, reads the sub-depth distributed by the pixel column where the pixel value is located, adds 1 to the read value and writes back the value to the original address,
after reading all the pixels, the FPGA adds the values stored in the storage addresses corresponding to the same pixel value in different sub-depths, and takes the addition result as the number of the pixel value to obtain the brightness domain representation.
11. An image enhancement apparatus comprising:
an obtaining module, configured to obtain a luminance domain representation of an image, where the luminance domain representation is used to characterize a pixel value distribution of the image in a luminance domain;
a calculation module for calculating a stretch coefficient based on the pixel value distribution; and
an updating module for updating pixel values of the image in the luminance domain based on the stretch coefficient.
12. A field programmable gate array for performing the method of any one of claims 1 to 8, wherein the operations involved in the method are fixed-point number operations obtained by quantization.
13. The field programmable gate array of claim 12,
the field programmable gate array employs a multiport BRAM memory,
the field programmable gate array divides the depth of the multiport BRAM memory into at least two sub-depths, each sub-depth corresponds to at least 256 memory addresses, adjacent pixel columns in the image are allocated to different sub-depths,
the field programmable gate array takes the pixel value of the read image in the brightness domain as an address, reads the sub-depth distributed by the pixel column where the pixel value is located, adds 1 to the read value and writes back the value to the original address,
after reading all the pixels, the field programmable logic gate array adds the numerical values stored in the storage addresses corresponding to the same pixel value in different sub-depths, and takes the addition result as the number of the pixel value to obtain the brightness domain representation.
14. An image forming apparatus comprising: an imaging device and a field programmable gate array,
the field programmable gate array is used for executing the method of any one of claims 1 to 8 to enhance the image obtained by the imaging device, wherein the operation involved in the method is a quantified fixed point number operation.
15. A computing device, comprising:
a processor; and
a memory having executable code stored thereon, which when executed by the processor, causes the processor to perform the method of any of claims 1 to 8.
16. A computer program product comprising executable code which, when executed by a processor of an electronic device, causes the processor to perform the method of any of claims 1 to 8.
17. A non-transitory machine-readable storage medium having stored thereon executable code, which when executed by a processor of an electronic device, causes the processor to perform the method of any of claims 1-8.
CN202210612662.9A 2022-05-31 2022-05-31 Image enhancement method and device, field programmable logic gate array and equipment Pending CN114998135A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210612662.9A CN114998135A (en) 2022-05-31 2022-05-31 Image enhancement method and device, field programmable logic gate array and equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210612662.9A CN114998135A (en) 2022-05-31 2022-05-31 Image enhancement method and device, field programmable logic gate array and equipment

Publications (1)

Publication Number Publication Date
CN114998135A true CN114998135A (en) 2022-09-02

Family

ID=83031064

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210612662.9A Pending CN114998135A (en) 2022-05-31 2022-05-31 Image enhancement method and device, field programmable logic gate array and equipment

Country Status (1)

Country Link
CN (1) CN114998135A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117314795A (en) * 2023-11-30 2023-12-29 成都玖锦科技有限公司 SAR image enhancement method by using background data

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101340510A (en) * 2008-08-07 2009-01-07 中兴通讯股份有限公司 Method for video enhancement and apparatus thereof
CN101674473A (en) * 2008-09-12 2010-03-17 中国科学院沈阳自动化研究所 Image real-time histogram statistical device and implementation method thereof
CN109309826A (en) * 2017-07-27 2019-02-05 Tcl集团股份有限公司 A kind of image color equalization methods and terminal
CN109829860A (en) * 2018-12-26 2019-05-31 武汉高德智感科技有限公司 Linearity dynamic range compression method and system of the full figure in conjunction with Local Phase
CN113222859A (en) * 2021-05-27 2021-08-06 西安电子科技大学 Low-illumination image enhancement system and method based on logarithmic image processing model
CN114219723A (en) * 2021-11-19 2022-03-22 浙江大华技术股份有限公司 Image enhancement method, image enhancement device and computer readable storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101340510A (en) * 2008-08-07 2009-01-07 中兴通讯股份有限公司 Method for video enhancement and apparatus thereof
CN101674473A (en) * 2008-09-12 2010-03-17 中国科学院沈阳自动化研究所 Image real-time histogram statistical device and implementation method thereof
CN109309826A (en) * 2017-07-27 2019-02-05 Tcl集团股份有限公司 A kind of image color equalization methods and terminal
CN109829860A (en) * 2018-12-26 2019-05-31 武汉高德智感科技有限公司 Linearity dynamic range compression method and system of the full figure in conjunction with Local Phase
CN113222859A (en) * 2021-05-27 2021-08-06 西安电子科技大学 Low-illumination image enhancement system and method based on logarithmic image processing model
CN114219723A (en) * 2021-11-19 2022-03-22 浙江大华技术股份有限公司 Image enhancement method, image enhancement device and computer readable storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117314795A (en) * 2023-11-30 2023-12-29 成都玖锦科技有限公司 SAR image enhancement method by using background data
CN117314795B (en) * 2023-11-30 2024-02-27 成都玖锦科技有限公司 SAR image enhancement method by using background data

Similar Documents

Publication Publication Date Title
US9984445B2 (en) Tone mapping
CN112734650B (en) Virtual multi-exposure fusion based uneven illumination image enhancement method
CN100568279C (en) A kind of fast colourful image enchancing method based on the Retinex theory
KR100925419B1 (en) Color Image Enhancement using Laplacian Pyramid and method thereof
RU2400815C2 (en) Method of enhancing digital image quality
KR102317613B1 (en) Systems and methods for localized contrast enhancement
CN100411445C (en) Image processing method and apparatus for correcting image brightness distribution
Khan et al. Tone-mapping using perceptual-quantizer and image histogram
Yu et al. Adaptive inverse hyperbolic tangent algorithm for dynamic contrast adjustment in displaying scenes
US20130287299A1 (en) Image processing apparatus
CN114998135A (en) Image enhancement method and device, field programmable logic gate array and equipment
Muniraj et al. Underwater image enhancement by modified color correction and adaptive Look-Up-Table with edge-preserving filter
Zhang et al. Multi-scale-based joint super-resolution and inverse tone-mapping with data synthesis for UHD HDR video
US8675963B2 (en) Method and apparatus for automatic brightness adjustment of image signal processor
CN110807748A (en) New tone mapping image enhancement method based on high dynamic range
CN116645291A (en) Self-adaptive Gamma correction glare suppression method based on regional brightness perception
CN110599426A (en) CLAHE (class-invariant feature extraction) optimized underwater image enhancement method
KR101418521B1 (en) Image enhancement method and device by brightness-contrast improvement
CN114140348A (en) Contrast enhancement method, device and equipment
Tan et al. A retinex-based local tone mapping algorithm using L 0 smoothing filter
Yang et al. Tone mapping based on multi-scale histogram synthesis
Wang et al. Tone-preserving contrast enhancement in images using rational tone mapping and constrained optimization
CN113487520B (en) High dynamic range image generation method and system based on converter temperature measurement
Yung-Yao et al. Photographic Reproduction and Enhancement Using HVS-Based Modified Histogram Equalization
Thai et al. HDR Image Tone Mapping Histogram Adjustment with Using an Optimized Contrast Parameter

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination