DE202023101378U1 - A System for Improving Poor Visibility Through Image Enhancement Technique Using Contrast Stretched - CLAHE (CS-CLAHE) - Google Patents

Abstract

A system (100) for improving poor visibility through an image enhancement technique using a contrast-stretched contrast-limited adaptive histogram equalization (CS-CLAHE), the system (100) comprising:
an input module (102) for receiving at least one low visibility image as input;
a visibility level prediction module (104) associated with the input module (102) for predicting the visibility level, the visibility prediction module (104) comprising:
a feature calculation module (104a) for calculating a plurality of features from the input image; And
a distance calculation module (104b) for calculating the minimum distance between two objects by a decoding technique;
an image enhancement module (106) associated with the visibility level prediction module (104) for enhancing the output image in predicting the visibility level, the image enhancement module (106) comprising:
a first conversion module (106a) for converting the input image from a first format to a second format to determine the visibility level of the input image, wherein the input image is compensated; and
a second conversion module (106b) associated with a color compensation module (108) and a memory block (110) is associated to convert the compensated image from the second format to the first format to obtain the enhanced image.

Description

FIELD OF THE INVENTION

The present invention relates to the field of image processing systems. More particularly, the present invention relates to a system for low visibility image enhancement technique using contrast stretched - CLAHE (CS-CLAHE).

BACKGROUND OF THE INVENTION

Image processing is a form of signal processing where the input is an image, e.g. a photo or a video image, and the output is either an image or a set of properties or parameters related to the image. Image resolution relates, at least in part, to the detail an image possesses. In satellite imagery, an image is generally considered to have more detail as the area represented by each pixel decreases. As used herein, the term images includes digital images, electronic images, film images, and/or other types of images. Cameras that take pictures from a long distance, such as B. Aerial photos may not receive detailed information about the subject. Consequently, subtle or detailed information may not be present in the images.

Often users simply want to improve the aesthetics of images. However, improving the aesthetics of an image often requires modifying various aspects of the image (e.g. adjusting lighting, color, focus, composition, etc.). As such, a user can manually edit individual aspects of an image. However, the resulting image is often unsatisfactory to the user because an inexperienced user does not have the training or eye to manipulate the image in a way that results in improved aesthetics. In this regard, such a conventional approach insufficiently modifies an image to achieve the desired result of an aesthetically enhanced image. Therefore, creating an image with improved aesthetics is a difficult and tedious task even for professional artists.

In addition, the quality of the image remains the same when changed manually. There are various open source tools used to improve image quality but they are not accurate and there is risk of high data loss due to using open source platforms.

Low-Light Enhancement: In general, low-light image enhancement methods can be divided into three categories, Retinex -based methods, Histogram-based methods and Convolutional Neural Network (CNN)-based methods. According to the Retinex theory, an image can be broken down into two factors, reflection and illumination. The reflection reflects the color characteristics of the object in the picture, and the lighting reflects the brightness of the environment. Single-Scale Retinex (SSR) and Multi-Scale Retinex (MSR) estimate the illuminance from the original image and then calculate the reflectance as an improved result.

• Fuet al. schlugen ein gewichtetes Variationsmodell vor, um Reflexion und Beleuchtung gleichzeitig abzuschätzen. Ein Verfahren schätzt die Beleuchtung mit vorheriger Struktur, was den Lösungsraum verkleinert und die Rechenkosten reduziert.

Various known techniques for improving image quality have been disclosed in the past, which are discussed below:

• Fu et al. proposed a weighted variational model to estimate reflectance and illumination simultaneously. One method estimates the lighting with previous structure, which decreases the solution space and reduces computational costs.

In an existing solution, a robust Retinex model was proposed that can handle low-light image enhancement in the case of intense noise. Histogram Equalization (HE) extends the dynamic range of the image by making the probability distribution function (PDF) of the image grayscale approximation to the uniform distribution. The variations of HE, such as Brightness Preserving Bi-Histogram (BBHE) and Dualistic Sub-Image Histogram Equalization (DSIHE), each take the mean brightness and mean gray value of the original image as thresholds, divide the original image into two non-overlapping sub-images, and then performs performs a histogram equalization for the two sub-images. Contrast Stretch and Contrast Limited Adaptive Histogram Equalization (CLAHE) both construct the cumulative distribution function (CDF) according to the local histogram, while CLAHE limits the slope of CDF by clipping the portion of the histogram that exceeds the set threshold. Therefore, CLAHE avoids excessive amplification of noise-like artifacts in homogeneous regions.

US11069030B2 discloses a neural network system that is trained, the training comprising training a first neural network that generates enhanced images that are dependent on the content of an enhanced image, and training a second neural network that denotes the realism of the enhanced images, generated by the first neural network. The neural network system is trained by determining the loss and adjusting the appropriate neural network(s) accordingly. That trained A neural network system is used to generate an enhanced aesthetic image from a selected image, where the output enhanced aesthetic image has enhanced aesthetics compared to the selected image.

US9727959B2 discloses a system and method for image enhancement, comprising: receiving a plurality of frame images of a given region of interest, the frame images consisting of a plurality of pixels; determining, based on a quantum property of the frames, a normalized pixel intensity value for each pixel of each of the plurality of frames; and generating an enhanced image of the given region of interest based on the plurality of frames and the corresponding normalized pixel intensity values for the frames, the order of the image being two.

US11115565B2 discloses that a smartphone can be moved freely in three dimensions when capturing a stream of images of an object. Multiple individual images can be captured at different orientations and distances from the object and combined into a composite image that represents an image of the object. The image frames may be formed into the composite image based on representing features of each image frame as a set of points in a three-dimensional point cloud. Inconsistencies between image frames can be adjusted when respective points in the point cloud are projected into the composite image. The quality of the picture frames can be improved by processing the picture frames to correct errors. Reflections and shadows can be detected and removed. Optical character recognition can also be used. As the scan progresses, an instruction to capture subsequent frames is provided to a user as real-time feedback.

According to a researcher, image enhancement is one of the most important and complex techniques in image processing technology. The main goal of image enhancement is to improve the visual appearance of an image and provide a better representation of the image for computer vision algorithms.

The success of CNNs in image classification has inspired researchers to use CNNs to improve low-light conditions.

In an existing solution, a variant of the Stacked Sparse Denoising Trained Autoencoder (LLNet) on a synthetic dataset has been proposed.

In another existing solution, a hybrid neural network was designed with a content stream for scene information prediction and an edge stream for edge detail learning.

In another existing solution, a linear transformation for embedding both the transmission map and the atmospheric light in one variable and performing end-to-end learning from the blurry image to the clean image was disclosed. To improve the algorithm's generalization ability, a semi-supervised learning procedure was proposed, which trained the supervised branch with the synthetic data with ground-truth labels, and trained the unsupervised branch with real data and unlabeled loss functions. Furthermore, Golts et al. used only real outdoor images and realized unsupervised learning via minimization of the DCP energy function.

In addition to the above image restoration and CNN-based methods, image enhancement methods including retinex theory and the histogram can also be used in defogging tasks.

The prior art mentioned above discloses a complex technique for improving image quality, which either focuses on improving the parameters of the image, which ultimately changes the properties of the image, or is an expensive technique.

In order to overcome the above limitations, there is a need to develop a system for a low-visibility image enhancement technique using Contrast Stretched-CLAHE (CS-CLAHE) that is simple and inexpensive.

The technical advances disclosed by the present invention overcome the limitations and disadvantages of existing and conventional systems and methods.

SUMMARY OF THE INVENTION

The present invention relates generally to a system for improving poor visibility by an image enhancement technique using contrast stretched-CLAHE (CS-CLAHE).

An object of the present invention is to develop a system for image processing.

Another object of the present invention is to provide a system for improving image quality.

Yet another object of the present invention is to improve real-time low visibility with high image quality for both blurry and dimly lit video sequences.

Yet another object of the present invention is to provide an inexpensive and simple system.

• ein Eingabemodul zum Aufnehmen mindestens eines Bildes mit geringer Sichtbarkeit als Eingabe;
• ein Modul zur Vorhersage des Sichtbarkeitsgrads, das dem Eingabemodul zugeordnet ist, um den Grad der Sichtbarkeit vorherzusagen, wobei das Modul zur Vorhersage des Sichtbarkeitsgrads Folgendes umfasst:
  • ein Merkmalsberechnungsmodul zum Berechnen einer Vielzahl von Merkmalen aus dem Eingabebild; Und
  • ein Abstandsberechnungsmodul zum Berechnen des Mindestabstands zwischen zwei Objekten durch Decodiertechnik;
  • ein Bildverbesserungsmodul, das dem Modul zur Vorhersage des Sichtbarkeitsgrads zugeordnet ist, um das Ausgabebild beim Vorhersagen des Sichtbarkeitsgrads zu verbessern, wobei das Bildverbesserungsmodul Folgendes umfasst:
    • ein erstes Konvertierungsmodul, das dem Eingabemodul zugeordnet ist, zum Konvertieren des Eingabebilds von einem ersten Format in ein zweites Format, um den Sichtbarkeitsgrad des Eingabebilds zu bestimmen, wobei das Eingabebild kompensiert wird; Und
    • ein zweites Umwandlungsmodul, das einem Farbkompensationsmodul und einem Speicherblock zugeordnet ist, um das kompensierte Bild von dem zweiten Format in das erste Format umzuwandeln, um das verbesserte Bild zu erhalten.

In one embodiment, a system for improving low visibility through an image enhancement technique using contrast-stretched, contrast-limited adaptive histogram equalization (CS-CLAHE), the system comprising:

• an input module for receiving at least one low visibility image as input;
• a visibility level prediction module associated with the input module for predicting the visibility level, the visibility level prediction module comprising:
  • a feature calculation module for calculating a plurality of features from the input image; And
  • a distance calculation module for calculating the minimum distance between two objects by decoding technique;
  • an image enhancement module associated with the visibility level prediction module to enhance the output image in predicting the visibility level, the image enhancement module comprising:
    • a first conversion module, associated with the input module, for converting the input image from a first format to a second format to determine the visibility level of the input image, wherein the input image is compensated; And
    • a second conversion module associated with a color compensation module and a memory block to convert the compensated image from the second format to the first format to obtain the enhanced image.

In one embodiment, a plurality of features include degree of intensity, saturation, and contrast energy.

In one embodiment, the input image and the obtained output image are in Red-Green-Blue (RGB) format.

In one embodiment, the distance calculation module decodes concatenated blocks to obtain the visibility level of the image.

In one embodiment, the first format is an RGB format and the second format is based on luma, blue projection and red projection (YUV).

In one embodiment, the luma (Y) is modified to a new Y to obtain the enhanced image by the second conversion module.

In one embodiment, the color compensation module receives the converted blue projection (U) and red projection (V) from the first conversion module and the modified new luma gain (Y) from the memory block.

In one embodiment, the memory block is a compact block that receives the Y from the first conversion block and the visibility level predicted by the visibility level prediction module.

In order to further clarify the advantages and features of the present invention, a more detailed description of the invention will be given with reference to specific embodiments thereof that are illustrated in the accompanying drawings. It should be understood that these drawings represent only typical embodiments of the invention and are therefore not to be considered as limiting its scope. The invention will be described and illustrated with additional specificity and detail with reference to the accompanying drawings.

character list

• 1 veranschaulicht ein Blockdiagramm eines Systems zum Verbessern schlechter Sichtbarkeit durch eine Bildverbesserungstechnik unter Verwendung von kontrastgestrecktem, kontrastbegrenztem, adaptivem Histogrammausgleich (CS-CLAHE), und
• 2 veranschaulicht eine schematische Darstellung der Hardwarearchitektur für CS-CLAHE mit Vorhersage der Sichtbarkeitsbewertung.
• 3 veranschaulicht einen algorithmischen Rahmen des vorgeschlagenen CS-CLAHE mit Sichtbarkeitsbewertungsmodell.

These and other features, aspects and advantages of the present invention will be better understood when the following detailed description is read with reference to the accompanying drawings, in which like numerals represent like parts throughout the drawings, wherein:

• 1 Figure 12 illustrates a block diagram of a system for improving poor visibility through an image enhancement technique using contrast-stretched, contrast-limited, adaptive histogram equalization (CS-CLAHE), and
• 2 illustrates a schematic of the hardware architecture for CS-CLAHE with visibility rating prediction.
• 3 illustrates an algorithmic framework of the proposed CS-CLAHE with visibility rating model.

Furthermore, those skilled in the art will appreciate that elements in the drawings are presented for simplicity and may not necessarily be drawn to scale. For example, the flowcharts illustrate the method in terms of the salient steps involved to help improve understanding of aspects of the present disclosure. Moreover, with respect to the construction of the device, one or more components of the device may have been represented in the drawings by conventional symbols, and the drawings may only show such specific details as are relevant to an understanding of the embodiments of the present disclosure around the drawings not to be obscured with details that would be readily apparent to one of ordinary skill in the art benefiting from the description contained herein.

DETAILED DESCRIPTION:

For the purposes of promoting an understanding of the principles of the invention, reference will now be made to the embodiment illustrated in the drawings and specific language will be used to describe the same. It should be understood, however, that no limitation on the scope of the invention is intended thereby, contemplating such changes and further modifications to the illustrated system, and such further applications of the principles of the invention illustrated therein, as would normally occur to one skilled in the art to which the invention relates.

Those skilled in the art will understand that the foregoing general description and the following detailed description are exemplary and explanatory of the invention and are not intended to be limiting.

Reference throughout this specification to "an aspect," "another aspect," or similar language indicates that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the phrases "in one embodiment," "in another embodiment," and similar phrases throughout this specification may, but need not, all refer to the same embodiment.

The terms "comprising," "comprising," or other variations thereof are intended to cover non-exclusive inclusion such that a process or method that includes a list of steps includes not only those steps but may include other steps not expressly listed or any inherent in such process or procedure. Similarly, without further limitation, one or more devices or subsystems or elements or structures or components preceded by "comprises...a" does not exclude the existence of other devices or other subsystems or other elements or other structures or other components or additional devices or additional subsystems or additional elements or additional structures or additional components.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The system, methods, and examples provided herein are illustrative only and are not intended to be limiting.

Embodiments of the present invention are described below in detail with reference to the accompanying drawings.

1 Figure 11 illustrates a block diagram of a system (100) for improving poor visibility through an image enhancement technique using contrast-stretched, contrast-limited adaptive histogram equalization (CS-CLAHE), the system (100) comprising: an input module (102), a visibility level prediction module (104) , a feature calculation module (104a), a distance calculation module (104b), an image enhancement module (106), a first conversion module (106a), a second conversion module (106b), color compensation module (108) and a memory block (110).

The input module (102) for receiving at least one low visibility image as input.

• ein Merkmalsberechnungsmodul (104a) zum Berechnen einer Vielzahl von Merkmalen aus dem Eingabebild; Und
• ein Abstandsberechnungsmodul (104b) zum Berechnen des minimalen Abstands zwischen zwei Objekten durch eine Decodiertechnik. Die Vielzahl von Merkmalen umfasst Tenengrad, Sättigung und Kontrastenergie. Das Entfernungsberechnungsmodul (104b) decodiert verkettete Blöcke, um den Sichtbarkeitsgrad des Bildes zu erhalten.

The visibility level prediction module (104) is associated with the input module (102) for predicting the visibility level, the visibility prediction module (104) comprising:

• a feature calculation module (104a) for calculating a plurality of features from the input image; And
• a distance calculation module (104b) for calculating the minimum distance between two objects by a decoding technique. The Variety of characteristics includes Tenengrad, Saturation and Contrast Energy. The distance calculation module (104b) decodes concatenated blocks to obtain the visibility level of the image.

• ein erstes Umwandlungsmodul (106a) zum Umwandeln des Eingangsbildes von einem ersten Format in ein zweites Format, um den Sichtbarkeitsgrad des Eingangsbildes zu bestimmen, wobei das Eingangsbild kompensiert wird; Und
• ein zweites Umwandlungsmodul (106b), das einem Farbkompensationsmodul (108) und einem Speicherblock (110) zugeordnet ist, um das kompensierte Bild von dem zweiten Format in das erste Format umzuwandeln, um das verbesserte Bild zu erhalten. Das erste Format ist das RGB-Format und das zweite Format basiert auf Luma, Blauprojektion und Rotprojektion (YUV). Das Luma (Y) wird zu einem neuen Y modifiziert, um das verbesserte Bild durch das zweite Konvertierungsmodul zu erhalten. Das Farbkompensationsmodul (108) empfängt die umgewandelte Blauprojektion (U) und Rotprojektion (V) von dem ersten Umwandlungsmodul (106a) und die Verstärkung des modifizierten neuen Luma (Y) von dem Speicherblock (110). Der Speicherblock (110) ist ein kompakter Block, der das Y von dem ersten Konvertierungsblock und die Sichtbarkeitsstufe, die von dem Sichtbarkeitsstufenvorhersagemodul (104) vorhergesagt wird, empfängt. Das Bildverbesserungsmodul (106) verbessert das Bild basierend auf einer Histogrammtechnik, die kontrastgestreckt ist - kontrastbegrenzte adaptive Histogrammentzerrung (CS-CLAHE).

The image enhancement module (106) is associated with the visibility level prediction module (104) to enhance the output image in predicting the visibility level, the image enhancement module (106) comprising:

• a first conversion module (106a) for converting the input image from a first format to a second format to determine the visibility level of the input image, wherein the input image is compensated; And
• a second conversion module (106b) associated with a color compensation module (108) and a memory block (110) for converting the compensated image from the second format to the first format to obtain the enhanced image. The first format is the RGB format and the second format is based on luma, blue projection and red projection (YUV). The luma (Y) is modified to a new Y to get the enhanced image by the second conversion module. The color compensation module (108) receives the converted blue projection (U) and red projection (V) from the first conversion module (106a) and the modified new luma gain (Y) from the memory block (110). The storage block (110) is a compact block that receives the Y from the first conversion block and the visibility level predicted by the visibility level prediction module (104). The image enhancement module (106) enhances the image based on a histogram technique that is contrast stretched - Contrast Limited Adaptive Histogram Equalization (CS-CLAHE).

The input image and the obtained output image are in red-green-blue (RGB) format.

2 illustrates a schematic of the hardware architecture for CS-CLAHE with visibility rating prediction.

The system takes an input RGB image that undergoes a first conversion that converts the RGB image to a YUV image. At the same time, the features are calculated from the RGB image and a minimum distance search is carried out. The input from the MVG model memory is sent to the minimum range search. The UV components obtained from the first conversion undergo color compensation. The Y component is fed to the AHE core and the CDF memory block simultaneously. The gain obtained from the CDF memory block is sent to the color compensation to perform color compensation. The color-compensated components undergo a second conversion before receiving the final output, in which the modified YUV component is converted back to an RGB component.

3 illustrates an algorithmic framework of the proposed CS-CLAHE with visibility rating model.

The drawings and the above description provide exemplary embodiments. Those skilled in the art will recognize that one or more of the elements described may well be combined into a single functional element. Alternatively, certain elements can be broken down into multiple functional elements. Elements from one embodiment may be added to another embodiment. For example, orders of processes described herein may be changed and are not limited to the manner described herein. Furthermore, the actions of any flowchart need not be implemented in the order shown; also, not all actions have to be performed. Actions that are not dependent on other actions can also be performed in parallel with the other actions. The scope of the embodiments is in no way limited by these specific examples. Numerous variations, whether or not expressly stated in the description, such as differences in structure, dimensions and use of materials, are possible. The scope of the embodiments is at least as broad as indicated by the following claims.

Advantages, other benefits, and solutions to problems have been described above with respect to specific embodiments. However, the benefits, benefits, problem solutions, and components that may cause benefits, benefits, or solutions to occur or become more pronounced are not to be construed as critical, required, or essential features or components of any or all claims.

Reference List

100

A system (100) for improving poor visibility by an image enhancement technique using Contrast Stretched - Contrast Limited Adaptive Histogram Equalization (CS-CLAHE).

102

input module

104

Visibility level prediction module

104a

Feature calculation module

104b

distance calculation module

106

image enhancement module

106a

first conversion module

106b

second conversion module

108

color compensation module

110

memory block

202

Enter RGB

204

RGB To YUV

206

Y channel

208

AH core

210

CDF memory blocks

212

win

214

New Y

216

YUV To RGB

218

Output RGB

220

color compensation

222

Find minimum LD distance

224

Calculate special feature

226

MVG model memory

228

visibility

302

Visibility level prediction

304

Image enhancement with contrast stretch followed by CLAH

306

Low visible image

308

Tenengrad

310

contrast search

312

CLAH

314

contrast energy

316

saturation

QUOTES INCLUDED IN DESCRIPTION

This list of documents cited by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent Literature Cited

• US 11069030 B2 [0008]
• US 9727959 B2 [0009]
• US 11115565 B2 [0010]

Claims (9)

A system (100) for improving poor visibility through an image enhancement technique using a contrast-stretched contrast-limited adaptive histogram equalization (CS-CLAHE), the system (100) comprising: an input module (102) for receiving at least one low visibility image as input; a visibility level prediction module (104) associated with the input module (102) for predicting the visibility level, the visibility prediction module (104) comprising: a feature calculation module (104a) for calculating a plurality of features from the input image; And a distance calculation module (104b) for calculating the minimum distance between two objects by a decoding technique; an image enhancement module (106) associated with the visibility level prediction module (104) for enhancing the output image in predicting the visibility level, the image enhancement module (106) comprising: a first conversion module (106a) for converting the input image from a first format to a second format to determine the visibility level of the input image, wherein the input image is compensated; and a second conversion module (106b) associated with a color compensation module (108) and a memory block (110) for converting the compensated image from the second format to the first format to obtain the enhanced image. system after claim 1 , where several features include degree of tenene, saturation, and contrast energy. system after claim 1 , where the input image and the obtained output image are in Red-Green-Blue (RGB) format. system after claim 1 , wherein the distance calculation module (104b) decodes concatenated blocks to obtain the visibility level of the image. system after claim 1 , where the first format is an RGB format and the second format is based on luma, blue projection and red projection (YUV). system after claim 1 , where the luma (Y) is modified to a new Y to obtain the enhanced image by the second conversion module. system after claim 1 wherein the color compensation module (108) receives the converted blue projection (U) and red projection (V) from the first conversion module (106a) and the modified new luma gain (Y) from the memory block (110). system after claim 1 , wherein the memory block (110) is a compact block receiving the Y from the first conversion block and the visibility level predicted by the visibility level prediction module (104). system after claim 1 , wherein the image enhancement module (106) enhances the image based on a histogram technique that is contrast stretched. Contrast-Limited Adaptive Histogram Equalization (CS-CLAHE).

Images