TWI512681B - Device and method for rain removal - Google Patents

Description

Image rain pattern removing device and method

The invention relates to an image in-vivo rain pattern removing device and a method, in particular to decomposing an image into a high frequency image and a low frequency image, and respectively strengthening the high frequency image and the low frequency image, and finally using the high frequency to enhance the image and the low frequency. Enhance the image to obtain the image of the rain pattern removal device and method.

Due to advances in technology, the use of motor vehicles has become increasingly popular, and the safety of the driving process is becoming more and more important. In many image security warning systems, the obstacles in the image are judged to alert the user to a safe distance. However, in bad weather conditions, such as heavy rain, it may cause misjudgment of images, which may lead to increased risk of driving. Therefore, how to effectively remove the rain pattern from the image to maintain the sharpness of the image is currently a problem to be solved.

In order to solve the above problem, an embodiment of the present invention provides a method for removing rain marks in an image, the method comprising: decomposing the original image into a high frequency image and a low frequency image; and filtering out the information corresponding to the predetermined depth of field according to the predetermined depth of field range. Acquire processed images; obtain from original images, low-frequency images, and processed images The low frequency enhances the image; the high frequency enhanced image corresponding to the high frequency image is obtained according to the dictionary learning algorithm; and the output image is obtained according to the high frequency enhanced image and the low frequency enhanced image.

Another embodiment of the present invention provides an image internal rain pattern removing device. The invention comprises an image capturing unit and a processing unit. The image capturing unit captures an original image. The processing unit receives the original image, and decomposes the original image into a high frequency image and a low frequency image, wherein the original image corresponding to the predetermined depth of field range is filtered according to a predetermined depth of field range to obtain a processed image according to the original image. And obtaining, by the low frequency image and the processed image, a low frequency enhanced image, obtaining a high frequency enhanced image corresponding to the high frequency image according to a dictionary learning algorithm, and obtaining an output image according to the high frequency enhanced image and the low frequency enhanced image .

100‧‧‧Image internal rain pattern removal device

110‧‧‧Image capture unit

120‧‧‧ raindrop detection unit

130‧‧‧Processing unit

S201-S212‧‧‧Step procedure

FIG. 1 is a schematic view showing an image internal rain pattern removing device according to an embodiment of the invention.

FIG. 2 is a flow chart showing a method for removing rain marks in an image according to an embodiment of the invention.

Other ranges applicable to the system and method of the present invention will be The details provided below are clearly visible. It is to be understood that the following detailed description, as well as specific embodiments, are intended to be illustrative of the embodiments of the present invention, and are not intended to limit the scope of the invention.

Please refer to Figure 1. Figure 1 shows an embodiment in accordance with the present invention. A schematic diagram of the rain pattern removal device in the image. As shown in FIG. 1 , the image internal rain pattern removing apparatus 100 includes an image capturing unit 110 , a raindrop detecting unit 120 , and a processing unit 130 . The image capturing unit 110 is configured to capture an original image. The raindrop detecting unit 120 is configured to sense whether it is currently raining and output a sensing result. The processing unit 130 is configured to receive the original image and determine whether it is raining according to the sensing result. If the processing unit 130 determines that it is currently raining, the method of removing the rain pattern in the image is performed to obtain a clear original image.

According to an embodiment of the present invention, if it is currently raining, the processing unit 130 decomposes the original image I into a high frequency image I _HF and a low frequency image I _LF using a guided filter, and uses the depth of field information based on photography. The significant area detection technique of (Depth of Field, DoF) obtains the processed image I _DoF corresponding to the original image I. It should be noted that other types of filters capable of decomposing images into high frequency images and low frequency images, such as left and right symmetric filters, may also be used in other embodiments. Then, the processing unit 130 obtains the processed image I _DoF corresponding to the original image I by using the salient region detection technology based on the depth information of the photography, and performs image enhancement on the high frequency image I _HF and the low frequency image I _{LF ,} respectively.

Regarding the part of the low frequency image I _LF , since the low frequency image obtained by the guiding filter may filter out the excessive non-rainprint information of the original image, the processing unit 130 first obtains the processed image I' after processing by setting a threshold value. _DoF , and then obtain a matrix α according to the processed image _I'DoF after processing to enhance the low frequency part of the original image. Next, the processing unit 130 further obtains the low-frequency enhanced image I′ _LF corresponding to the low-frequency image I _LF according to the original image I, the low-frequency image I _{LF ,} and the matrix α .

Regarding the portion of the high frequency image I _HF , the processing unit 130 obtains the dictionary D _HF corresponding to the high frequency image I _HF according to a dictionary learning algorithm. Since the dictionary D _HF corresponding to the high frequency image I _HF may still contain part of the rain pattern, and the part of the rain pattern is closer to the gray scale value, the processing unit 130 performs the Eigen Color feature extraction algorithm based on the color feature. The rain pattern is filtered to obtain a first non- _rainprint area high frequency image I _{HF — NR1} corresponding to the dictionary D _HF .

Further, the processing unit 130 further dictionary D _HF according to the gradient histogram (Histogram of oriented gradient, HoG) to obtain the corresponding dictionary D _HF rain pattern dictionary containing free D _R and rain pattern dictionary D _NR. Among them, the correlation algorithm using the gradient histogram is a conventional technique, so it will not be described in detail here to simplify the description. Then, the rain-free dictionary D _NR is reconstructed according to the significant region detection technology based on the depth information to obtain the second non- _rainprint region high frequency image I HF — _NR2 corresponding to the rainless dictionary D _NR . Rain on the part of the pattern dictionary containing D _R, since the details may still contain the original image I, so that the processing unit 130 are reconstructed into an image containing a high frequency through the rain groove _I HF_R dictionary learning algorithms rain pattern dictionary containing D _R. Then, the processing unit 130 obtains the rain- _fed high-frequency depth image DoF _{HF_R} corresponding to the rain-like high-frequency image I _{HF_R} according to the significant region detection technology based on the depth information, and the rain- _fed high-frequency depth image DoF _{HF_R} and The processed image I _{DoF is} multiplied to leave a corrected image DoF' _{HF_R} containing the details of the false positive original image I. After obtaining the corrected image DoF' _{HF_R} , the processing unit 130 multiplies the corrected image DoF' _{HF_R by the} rain-like high-frequency image I _{HF_R} to obtain the rain-grain region high-frequency image I _{HF_R} .

Finally, the processing unit 130 according to the first high-frequency region of the non-image pattern rain _I HF_NR1, the second non-rain-out area and a high-frequency image _I HF_NR2 rain-out area to obtain a high-frequency image _I HF_R high frequency emphasis image I _'HF.

After the processing unit 130 completes the image enhancement process of all the high frequency image I _HF and the low frequency image I _LF , the high frequency enhanced image I′ _HF and the low frequency enhanced image I′ _LF are added to obtain the final output image I _RN .

Then see Figure 2. FIG. 2 is a flow chart showing a method for removing rain marks in an image according to an embodiment of the invention. In this embodiment, the method for removing raindrops in the image according to the present invention is analyzed and processed in units of a single image. First, in step S201, the original image I is decomposed into a high frequency image I _HF and a low frequency image I _LF using a steering filter. It should be noted that other types of filters capable of decomposing images into high frequency images and low frequency images, such as bilateral filters, etc., may also be used in other embodiments. In step S202, the processed image I _DoF corresponding to the original image I is obtained by the salient region detection technology based on the depth of field information of the photography using the following function: In other words, the processed image I _DoF is obtained by filtering out the rain pattern portion of the original image according to a predetermined depth of field range. Subsequently, reinforcing lines were made corresponding to the image I _'HF and enhanced image I corresponding to a low frequency _LF of the low frequency image I' at high frequencies _LF frequency of _{the HF} video I.

Since the low frequency image obtained by the guiding filter may filter out the non-rainprint information of the original image, in step S203, the original image I, the low frequency image I _LF and the matrix α are used to obtain the corresponding low frequency image I according to the following function. _LF of the low frequency image strengthening _{I 'LF: I' LF =} αI + (1-α) I LF where, α (i, j) [0,1], according to the processed image I' _DoF after processing (by setting a threshold, the value of the processing image I _DoF is greater than or equal to the threshold value, and the value less than the threshold is set to 0 For example, the threshold is set to 0.2, the value of greater than or equal to 0.2 in the processed image I _DoF is retained, and the value less than 0.2 is set to 0). The purpose is to derive the low frequency portion of the original image. Next, proceeding to step S204, a dictionary D _HF corresponding to the high frequency image I _HF is obtained based on the dictionary learning algorithm. _Proceeding to step S205, the first non- _rainprint region high frequency image I _{HF_NR1} corresponding to the dictionary D _{HF is} obtained according to the feature extraction algorithm based on the color feature. Since the color information of the rainweed information is closer to the grayscale value, the above-mentioned characteristics can be used to re-filter the missing rainweave information.

In step S206, the dictionary D _HF according to the gradient histogram (Histogram of oriented gradient, HoG) to obtain the corresponding dictionary D _HF rain pattern dictionary containing free D _R and rain pattern dictionary D _NR. Among them, the correlation algorithm using the gradient histogram is a conventional technique, so it will not be described in detail here to simplify the description. In step S207, the rain-free dictionary D _NR is reconstructed according to the significant area detection technology based on the depth information to obtain the second non- _rainprint area high frequency image I HF — _NR2 corresponding to the rain-free dictionary D _NR .

Next, proceeding to step S208, the rain- _stained dictionary D _{R is} reconstructed into a rain- _stained high-frequency image I _{HF_R by a} dictionary learning algorithm. . In step S209, the rain- _fed high-frequency depth image DoF _{HF_R} corresponding to the rain-like high-frequency image I _{HF_R is obtained according to the salient} depth region detection technology based on the depth of field information. In step S210, according to the image containing the high frequency pattern _I HF_R rain, rain grain containing the high-frequency and depth of the image-processed image _DoF HF_R I _DoF high frequency region to obtain the image pattern rain _I HF_R. Next, proceeding to step S211, the high-frequency enhanced image I' _{HF is} obtained from the first non- _rainprint area high frequency image I _{HF_NR1} , the second non-rain area high frequency image I _{HF_NR2,} and the rain pattern high frequency image I _{HF_R} . Finally, in step S212, the final output image I _{RN is} obtained from the high frequency enhanced image I' _HF and the low frequency enhanced image I' _LF .

In summary, the image internal rain according to an embodiment of the present invention The method and device for removing the image are obtained by dividing the image into a high-frequency image and a low-frequency image, and respectively performing image enhancement on the high-frequency image and the low-frequency image. In the details of the omission, the enhanced high-frequency image and low-frequency image are combined to obtain the image after the rain pattern is removed. In addition to retaining the image details of the original high-frequency image, the false positive part can be recovered from the low-frequency image. In order to get a clearer image.

The features of many of the embodiments are described above in the art. Those who have ordinary knowledge can clearly understand the form of this specification. Those having ordinary skill in the art will appreciate that the objectives of the above-described embodiments and/or advantages consistent with the above-described embodiments can be accomplished by designing or modifying other processes and structures based on the present disclosure. It is also to be understood by those skilled in the art that <RTIgt; </ RTI> <RTIgt; </ RTI> <RTIgt; </ RTI> <RTIgt; </ RTI> <RTIgt;

100‧‧‧Image internal rain pattern removal device

110‧‧‧Image capture unit

120‧‧‧ raindrop detection unit

130‧‧‧Processing unit

Claims (10)

A method for removing rain marks in an image, comprising: decomposing an original image into a high frequency image and a low frequency image; filtering out information corresponding to the predetermined depth of field range according to a predetermined depth of field range to obtain a processed image; Obtaining a low frequency enhanced image from the original image, the low frequency image, and the processed image; acquiring a first non-rainprint region high frequency image, a second non-rainprint region high frequency image, and a rain pattern region according to a dictionary learning algorithm a high frequency image, and obtaining a high frequency enhanced image corresponding to the high frequency image according to the first non-rainprint region high frequency image, the second non-rainprint region high frequency image, and the rain region high frequency image; And obtaining an output image according to the high frequency enhanced image and the low frequency enhanced image. The method for removing raindrops in an image according to claim 1, further comprising: obtaining a dictionary corresponding to the high frequency image according to a dictionary learning algorithm; and obtaining a correspondence according to a feature extraction algorithm based on the color feature And the first non-rainprint region high frequency image in the dictionary; obtaining a rainy weather dictionary corresponding to the dictionary and a rainless dictionary according to a gradient histogram corresponding to the dictionary; Reconstructing the above-mentioned non-rainprint dictionary according to a significant area detection technique based on depth of field information to obtain the second non-rainprint region high frequency image corresponding to the above-mentioned rainless dictionary; obtaining a rainy image according to the rainy weather dictionary a high frequency image and a rainy high frequency depth image; the high frequency image of the rain region is obtained according to the rainy high frequency image, the rainy high frequency depth image and the processed image; and according to the first non The high frequency image of the rain pattern region, the high frequency image of the second non-rainprint region, and the high frequency image of the rain region are used to obtain the high frequency enhanced image. The method for removing the rain pattern in the image according to the second aspect of the patent application, further comprising: obtaining the high-frequency image containing the rain pattern according to the rain-free dictionary; and obtaining the corresponding region based on the significant region detection technology based on the depth information The above-mentioned rainy-grain high-frequency depth image containing the rain-grain high-frequency image. The method for removing rain marks in an image according to claim 1, further comprising: obtaining the processed image corresponding to the original image according to a salient region detection technology based on depth of field information. The method for removing rain marks in an image according to claim 1, further comprising: decomposing the original image into the high frequency image by using a guiding filter. And the above low frequency image. An image internal rain pattern removing device includes: an image capturing unit that captures an original image; and a processing unit that receives the original image and decomposes the original image into a high frequency image and a low frequency image, according to one The predetermined depth of field range filters out the information corresponding to the predetermined depth of field range to obtain a processed image, and obtains a low frequency enhanced image according to the original image, the low frequency image, and the processed image, and obtains a first image according to a dictionary learning algorithm. a non-rainprint region high frequency image, a second non-rainprint region high frequency image, and a rain pattern region high frequency image, and the first non-rainprint region high frequency image and the second non-rainprint region high frequency image And the high frequency image of the rain region is obtained by acquiring a high frequency enhanced image corresponding to the high frequency image, and obtaining an output image according to the high frequency enhanced image and the low frequency enhanced image. The intra-image rain pattern removing device according to claim 6, wherein the processing unit further obtains a dictionary corresponding to the high-frequency image according to a dictionary learning algorithm, according to a feature extraction algorithm based on color features. Obtaining the first non-rainprint region high frequency image corresponding to the dictionary, and obtaining a rainy weather dictionary corresponding to the dictionary and a rainless dictionary according to a gradient histogram corresponding to the dictionary, according to a depth of field based on The significant area detection technology of the information reconstructs the above-mentioned rain-free dictionary to obtain the second non-rainprint area high-frequency image corresponding to the above-mentioned rain-free dictionary, and obtains a rain-like high-frequency image according to the rain-containing dictionary. And a rainy-grain high-frequency depth image, according to The high-frequency image containing rainprint, the high-definition depth image including the rain pattern, and the processed image acquire the high-frequency image of the rain region, and the high-frequency image of the first non-rainprint region and the second non-rainned region The high frequency enhanced image is obtained from the frequency image and the high frequency image of the rain region. The image in-vivo rain removing device according to claim 7, wherein the processing unit further obtains the rain-like high-frequency image according to the rain-free dictionary, and the salient region detecting technology based on the depth information based The above-mentioned rain-fed high-frequency depth image corresponding to the rain-like high-frequency image is obtained. The image in-vivo rain pattern removing device of claim 6, wherein the processing unit further obtains the processed image corresponding to the original image according to a salient region detection technology based on depth of field information. The image in-vivo rain removing apparatus according to claim 6, wherein the processing unit further decomposes the original image into the high frequency image and the low frequency image by using a guiding filter.