CN116645289A - Method and device for removing raindrops in image, storage medium and electronic equipment - Google Patents
Method and device for removing raindrops in image, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN116645289A CN116645289A CN202310638637.2A CN202310638637A CN116645289A CN 116645289 A CN116645289 A CN 116645289A CN 202310638637 A CN202310638637 A CN 202310638637A CN 116645289 A CN116645289 A CN 116645289A
- Authority
- CN
- China
- Prior art keywords
- image
- raindrop
- raindrops
- characteristic information
- training sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 238000012545 processing Methods 0.000 claims abstract description 23
- 238000012549 training Methods 0.000 claims description 67
- 238000003062 neural network model Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 13
- 238000001914 filtration Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 description 13
- 238000013473 artificial intelligence Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 238000013135 deep learning Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000010801 machine learning Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000012535 impurity Substances 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000036413 temperature sense Effects 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The application discloses a method and a device for removing raindrops in an image, a storage medium and electronic equipment, wherein the method comprises the following steps: acquiring a raindrop image to be processed, inputting the raindrop image into a trained raindrop feature recognition model to obtain raindrop feature information in the raindrop image, and performing raindrop removal processing on raindrops in the raindrop image according to the raindrop feature information to obtain a target raindrop-free image corresponding to the raindrop image. According to the raindrop-free image corresponding to the raindrop-containing image, the raindrop characteristic information in the raindrop-containing image can be accurately identified through the raindrop identification model, and the raindrops in the raindrop-containing image are removed according to the accurately identified raindrop characteristic information, so that the raindrop-free image corresponding to the raindrop-containing image can be obtained.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to a method and an apparatus for removing raindrops in an image, a storage medium, and an electronic device.
Background
At present, an auxiliary driving technology for assisting driving of a vehicle by shooting images of the vehicle in the running process of the vehicle by a vehicle-mounted camera is widely applied, and in the process of shooting the images by the vehicle-mounted camera, raindrops and raindrops appear in image pictures acquired by the camera when the vehicle-mounted camera encounters rainy days, so that the image quality of the acquired images is affected to different degrees. In the related art, in order to eliminate the influence of raindrops and rainwater on the image quality of an image, the raindrops in the image are usually removed, and then the vehicle is driven in an assisted manner through the image after the raindrops are removed, however, the accuracy is low when the raindrops in the image are removed in the related art, so that the detection and extraction of useful information by an intelligent driving algorithm are greatly influenced, and the safety of intelligent driving is influenced.
Disclosure of Invention
The embodiment of the application provides a method and a device for removing raindrops in an image, a storage medium and electronic equipment, which can improve the accuracy of removing the raindrops in the image.
In a first aspect, an embodiment of the present application provides a method for removing raindrops in an image, including:
acquiring a raindrop image to be processed;
inputting the raindrop image into a trained raindrop feature recognition model to obtain raindrop feature information in the raindrop image;
and removing raindrops in the raindrop image according to the raindrop characteristic information to obtain a target raindrop-free image corresponding to the raindrop image.
In a second aspect, an embodiment of the present application further provides an apparatus for removing a raindrop in an image, including:
the acquisition unit is used for acquiring the image with the raindrops to be processed;
the raindrop characteristic recognition unit is used for inputting the raindrop image into a trained raindrop characteristic recognition model to obtain raindrop characteristic information in the raindrop image;
and the raindrop removing unit is used for removing raindrops in the raindrop image according to the raindrop characteristic information to obtain a target raindrop-free image corresponding to the raindrop image.
In a third aspect, embodiments of the present application also provide a computer-readable storage medium having stored thereon a computer program which, when run on a computer, causes the computer to perform a method of raindrop removal in an image as provided by any of the embodiments of the present application.
In a fourth aspect, an embodiment of the present application further provides an electronic device, including a processor and a memory, where the memory has a computer program, and the processor is configured to execute the method for removing raindrops in an image provided in any embodiment of the present application by calling the computer program.
According to the technical scheme provided by the embodiment of the application, the to-be-processed raindrop image is obtained, the raindrop image is input into the trained raindrop feature recognition model to obtain the raindrop feature information in the raindrop image, and the raindrop removal processing is carried out on the raindrops in the raindrop image according to the raindrop feature information to obtain the target raindrop-free image corresponding to the raindrop image. According to the raindrop-free image corresponding to the raindrop-containing image, the raindrop characteristic information in the raindrop-containing image can be accurately identified through the raindrop identification model, and the raindrops in the raindrop-containing image are removed according to the accurately identified raindrop characteristic information, so that the raindrop identification model is high in accuracy of raindrop characteristic identification, the accuracy of raindrop removal in the image can be improved, detection and extraction of useful information by an intelligent driving algorithm by the raindrops are reduced, and accordingly safety of intelligent driving is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is an application scenario schematic diagram of a method for removing raindrops in an image according to an embodiment of the present application.
Fig. 2 is a schematic flow chart of a first method for removing raindrops in an image according to an embodiment of the present application.
Fig. 3 is a schematic structural diagram of an apparatus for removing raindrops in an image according to an embodiment of the present application.
Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It will be apparent that the described embodiments are only some, but not all, embodiments of the application. All other embodiments, which can be made by a person skilled in the art without any inventive effort, are intended to be within the scope of the present application based on the embodiments of the present application.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
Artificial intelligence (Artificial Intelligence, AI) is a theory, method, technique, and application system that utilizes a digital computer or a machine controlled by a digital computer to simulate, extend and extend human intelligence, sense the environment, acquire knowledge, and use knowledge to obtain optimal results. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence, i.e. research on design principles and implementation methods of various intelligent machines, enables the machines to have functions of sensing, reasoning and decision.
The artificial intelligence technology is a comprehensive subject, and relates to the technology with wide fields, namely the technology with a hardware level and the technology with a software level. Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. Artificial intelligence software technology mainly includes Machine Learning (ML) technology, wherein Deep Learning (DL) is a new research direction in Machine Learning, which is introduced into Machine Learning to make it closer to an original target, i.e., artificial intelligence. At present, deep learning is mainly applied to the fields of computer vision, natural language processing and the like.
Deep learning is the inherent regularity and presentation hierarchy of learning sample data, and information obtained during such learning processes greatly aids in interpretation of data such as text, image and sound. The deep learning technology and the corresponding training data set are utilized to train and obtain network models realizing different functions, for example, a gender classification model for gender classification can be obtained based on one training data set, an image optimization model for image optimization can be obtained based on another training data set, and the like.
In order to improve the accuracy of raindrop removal, the application introduces a deep learning technology into raindrop characteristic information identification, and correspondingly provides a method for removing raindrops in an image, a device for removing raindrops in the image, electronic equipment and a storage medium. The method for removing the raindrops in the image can be performed by a device for removing the raindrops in the image or an electronic device integrated with the device for removing the raindrops in the image. The device for removing the raindrops in the image can be realized in a hardware or software mode. The electronic device may be any device with a processor and having a processing capability, such as a mobile electronic device with a processor, such as a smart phone, a tablet computer, a palm computer, a notebook computer, or a stationary electronic device with a processor, such as a desktop computer, a television, or a server.
For example, referring to fig. 1, the present application further provides a system for removing raindrops in an image, as shown in fig. 1, where the system for removing raindrops in an image includes an electronic device 10, and the electronic device 10 has an integrated device for removing raindrops in an image provided by the present application, and the system for removing raindrops in an image provided by the present application may further include an information collecting component, where the information collecting component is used for collecting various information, and the information collecting component may be an image sensor, for example, used for collecting an image. For example for acquiring images with raindrops. The information acquisition component may be an information acquisition component configured by the electronic device 10 itself, or may be a separate information acquisition component in communication with the electronic device, through which the raindrop image may be directly or indirectly acquired. The raindrop image may be an image in which raindrops exist in a screen captured in rainy weather, or may be an image in which water splashes exist in a captured scene, thereby capturing a waterdrop image. The electronic device 10 may obtain the raindrop characteristic information in the raindrop image by acquiring a raindrop image to be processed, inputting the raindrop image into a trained raindrop characteristic recognition model, and performing raindrop removal processing on the raindrops in the raindrop image according to the raindrop characteristic information to obtain a target raindrop-free image corresponding to the raindrop image. In addition, as shown in fig. 1, the system for removing raindrops in the image may further include a storage device 20 for storing data, for example, the electronic device 10 stores the acquired image with raindrops to be processed in the storage device 20.
It should be noted that, the schematic view of the system for removing the raindrops in the image shown in fig. 1 is only an example, and the system for removing the raindrops in the image and the scene described in the embodiment of the present application are for more clearly describing the technical solution of the embodiment of the present application, and do not constitute a limitation to the technical solution provided by the embodiment of the present application, and as a person of ordinary skill in the art can know that the technical solution provided by the embodiment of the present application is applicable to similar technical problems with evolution of the system for removing the raindrops in the image and occurrence of new service scenes.
Referring to fig. 2, fig. 2 is a schematic flow chart of a method for removing raindrops in an image according to an embodiment of the application. The specific flow of the method for removing the raindrops in the image provided by the embodiment of the application can be as follows:
101. and acquiring a raindrop image to be processed.
The raindrop image is an image in which raindrops exist in an image frame, for example, the raindrop image may be an image in which raindrops exist in a frame photographed in rainy weather, or may be an image in which water splashes exist in a photographed scene, thereby photographing the image. In an actual scene, the to-be-processed raindrop image can be an image which is photographed by a vehicle-mounted camera in an auxiliary driving scene and used for auxiliary driving, or can be a raindrop image which needs to be enhanced in image quality in a repairing scene. Whether in-vehicle scene or other images with raindrops are acquired in other scenes, since the image quality is reduced due to the raindrops in the images, the subsequent processing may be affected by the raindrops in the images, and therefore, it is necessary to perform the processing of removing the raindrops in the images to eliminate the influence of the raindrops in the images on the image quality as much as possible.
The electronic device can acquire the image with the rain drops to be processed through an image sensor which is configured by the electronic device, and can also acquire the image with the rain drops to be processed through a separate external image sensor which is in communication connection with the electronic device.
102. And inputting the raindrop image into a trained raindrop feature recognition model to obtain raindrop feature information in the raindrop image.
The raindrop characteristic recognition model is obtained through pre-training and is configured to perform raindrop recognition processing on an input raindrop image and output raindrop characteristic information in the raindrop image. The model structure and training mode of the raindrop feature recognition model are not particularly limited, and can be selected by a person skilled in the art according to actual needs. For example, the raindrop feature recognition model can be obtained by training a Long-and-short-term memory neural network model (Long-short time memory, LSTM) as a basic model.
The raindrop characteristic information may be information such as contour information, shape information, transparency, and the like.
In some embodiments, the raindrop feature recognition model may be trained as follows:
the method comprises the steps of obtaining a training sample set, wherein the training sample set comprises a plurality of training sample sets, and each training sample set comprises a group of visible light images and invisible light images which are obtained under the same condition when the rainy condition exists;
training the neural network model according to the training sample set until a preset training stopping condition is met;
and taking the neural network model meeting the preset training stopping condition as the raindrop characteristic recognition model.
The visible light image is an image acquired by a visible light camera. The invisible light image is an image collected by an invisible light camera, the invisible light camera can be an infrared camera, and the invisible light image can be an infrared light image collected by the infrared camera.
In some embodiments, the same condition may be the same shooting scene at the same time and the same place. The visible light camera and the infrared light camera can be used, and simultaneously, the visible light image and the infrared light image of the same shooting scene at the same time and the same place can be collected under the condition of rain to be used as a training sample group.
In the related art, two visible light images are used as training sample sets, one image is provided with raindrops, and the other image is not provided with raindrops, so that time synchronization is difficult, and other difference information except for raindrop information is likely to be shot into the images, thereby influencing the extraction precision of the raindrop characteristic information. The method provided by the application can overcome the defects in the related art by using the infrared light image as a training sample to train the raindrop characteristic recognition model. The application provides the method for simultaneously capturing the visible light image and the infrared light image in raining, and the effective image information on the two images is more synchronous. Therefore, when difference comparison is carried out on the raindrop characteristic information, the impurity information is less, and the raindrop characteristic recognition model is more accurate. It can be understood that, because the imaging of the infrared camera depends on temperature sense, whether raindrops exist or not has no influence on the shot image, and under the rainy condition, no raindrops exist in the shot infrared light image.
As described above, after the training sample set is obtained, the embodiment may further train the long-short-term memory neural network model according to the obtained training sample set until a preset training stop condition is satisfied. The preset training stop conditions are not particularly limited, and can be configured by a person skilled in the art according to actual needs, for example, the raindrop characteristic recognition model can be considered successful when the raindrop recognition rate in the image reaches more than 80%, and training of the model can be stopped.
For example, the training sample set may be divided into three parts proportionally, such as a training set, a validation set, and a test set in a 3:1:1 ratio. It should be noted that, the training set, the verification set and the test set all include a plurality of training sample sets, where the training set is used to adjust network parameters of the model, the verification set is used to adjust super parameters of the model and to perform preliminary evaluation on the capacity of the model, and the test set is used to evaluate the generalization capacity of the final model.
In some embodiments, before training the neural network model according to the training sample set, further comprising:
acquiring first image characteristic information corresponding to the visible light image and second image characteristic information corresponding to the infrared light image in each training sample group;
acquiring image difference information according to the first image characteristic information and the second image characteristic information;
taking the image characteristic difference information as target raindrop characteristic information;
and acquiring target raindrop characteristic information in all training sample groups to obtain a raindrop characteristic information set.
It can be understood that, since the visible light image and the infrared light image in the training sample set are images of the same shooting scene at the same time and the same place acquired under the condition of raining, raindrops exist in the visible light image, and raindrops do not exist in the infrared light image, the image characteristic difference information reflects the raindrop characteristic information.
In order to improve the accuracy of the raindrop feature recognition model, images with different times, different places and different rainfall can be collected to serve as sample images in a training sample set. The shooting scene can be designed according to actual needs, for example, if the raindrop feature recognition model is used for driving scene, the shooting scene can be a common scene in driving situation, such as expressway scene, urban road scene, and the like.
The raindrop characteristic information set is a set of raindrop characteristics extracted from sample images in a training sample set.
In some embodiments, the training the neural network model according to the training sample set includes:
and training the neural network model according to the raindrop characteristic information set.
It can be understood that the raindrop feature recognition model provided by the application helps to recognize raindrops by acquiring a large amount of image feature information related to the raindrops and taking the image feature information as reference information.
In some embodiments, the acquiring the first image feature information corresponding to the visible light image in each training sample set includes:
carrying out gray scale processing on the visible light image to obtain a gray scale image;
and acquiring the first image characteristic information according to the gray level image.
The visible light image is converted into a gray image by gray processing, that is, brightness information is extracted by gray processing, colors are removed, and the gray image is compared with the infrared light image, wherein most of difference information is information of raindrops.
103. And removing raindrops in the raindrop image according to the raindrop characteristic information to obtain a target raindrop-free image corresponding to the raindrop image.
In some embodiments, a raindrop may be used as a noise point in an image by using an image denoising algorithm, for example, image filtering processing is performed based on raindrop feature information, the raindrop feature information in a raindrop image is removed, a filtered image is obtained, and then image restoration processing is performed on the filtered image, so as to obtain a target raindrop-free image corresponding to the raindrop image. Wherein the pixel characteristics of the damaged or lost part of the image are repaired or filled through image repair processing.
In some embodiments, a distribution area of the raindrops in the raindrop image may be determined according to the raindrop characteristic information in the raindrop image, and the raindrop removal processing may be performed only on the distribution area when the raindrop removal processing is performed, so that the loss of image quality of the whole image may be reduced to some extent.
In particular, the application is not limited by the order of execution of the steps described, as some of the steps may be performed in other orders or concurrently without conflict.
From the above, according to the method for removing the raindrops in the image provided by the embodiment of the application, the raindrop characteristic information in the image with the raindrops can be accurately identified through the raindrop identification model, and the raindrops in the image with the raindrops are removed according to the accurately identified raindrop characteristic information, so that the raindrop-free image corresponding to the image with the raindrops is obtained.
In some embodiments, an apparatus for removing raindrops from an image is also provided. Referring to fig. 3, fig. 3 is a schematic structural diagram of an apparatus 200 for removing raindrops in an image according to an embodiment of the application. The apparatus 200 for removing a raindrop in an image is applied to an electronic device, and the apparatus 200 for removing a raindrop in an image includes an acquisition unit 201, a raindrop feature identification unit 202, a raindrop removal unit 203, and an image restoration unit 204, as follows:
an acquisition unit 201 for acquiring a raindrop image to be processed;
a raindrop feature recognition unit 202, configured to input the raindrop image into a trained raindrop feature recognition model, to obtain raindrop feature information in the raindrop image;
and the raindrop removing unit 203 is configured to perform a raindrop removing process on the raindrops in the raindrop image according to the raindrop characteristic information, so as to obtain a target raindrop-free image corresponding to the raindrop image.
In some embodiments, the apparatus for removing raindrops in an image further comprises a model training unit for:
the method comprises the steps of obtaining a training sample set, wherein the training sample set comprises a plurality of training sample sets, and each training sample set comprises a group of visible light images and invisible light images which are obtained under the same condition when the rainy condition exists;
training the neural network model according to the training sample set until a preset training stopping condition is met;
and taking the neural network model meeting the preset training stopping condition as the raindrop characteristic recognition model.
In some embodiments, the same condition refers to the same shooting scene at the same time and the same place.
In some embodiments, the invisible light image is an infrared light image.
In some embodiments, the raindrop feature identification unit 202 may also be configured to:
acquiring first image characteristic information corresponding to the visible light image and second image characteristic information corresponding to the infrared light image in each training sample group;
acquiring image difference information according to the first image characteristic information and the second image characteristic information;
taking the image characteristic difference information as target raindrop characteristic information;
and acquiring target raindrop characteristic information in all training sample groups to obtain a raindrop characteristic information set.
In some embodiments, the model training unit is further to:
and training the neural network model according to the raindrop characteristic information set.
In some embodiments, the raindrop feature identification unit 202 may also be configured to:
carrying out gray scale processing on the visible light image to obtain a gray scale image;
and acquiring the first image characteristic information according to the gray level image.
In some embodiments, the raindrop removal unit 203 is configured to:
filtering the image with the raindrops according to the raindrop characteristic information to obtain a filtered image;
and performing image restoration processing on the filtered image to obtain a target raindrop-free image corresponding to the raindrop-containing image.
It should be noted that, the device for removing a raindrop in an image provided by the embodiment of the present application belongs to the same concept as the method for removing a raindrop in an image in the above embodiment, and any method provided in the method embodiment for removing a raindrop in an image may be implemented by using the device for removing a raindrop in an image, and a specific implementation process of the method embodiment for removing a raindrop in an image is detailed in the method embodiment for removing a raindrop in an image, which is not described herein.
As can be seen from the foregoing, in the device for removing a raindrop in an image provided by the embodiment of the present application, a raindrop image to be processed may be obtained by the obtaining unit 201, the raindrop image is input into a trained raindrop feature recognition model by the raindrop feature recognition unit 202, the raindrop feature information in the raindrop image is obtained, and the raindrop removal unit 203 performs a raindrop removal process on a raindrop in the raindrop image according to the raindrop feature information, so as to obtain a target raindrop-free image corresponding to the raindrop image. According to the raindrop-free image corresponding to the raindrop-containing image, the raindrop characteristic information in the raindrop-containing image can be accurately identified through the raindrop identification model, and the raindrops in the raindrop-containing image are removed according to the accurately identified raindrop characteristic information, so that the raindrop-free image corresponding to the raindrop-containing image can be obtained.
In addition, in order to better implement the method for removing the raindrops in the image according to the embodiment of the present application, the present application further provides an electronic device, please refer to fig. 4, fig. 4 shows a schematic structural diagram of the electronic device 300 provided by the present application, as shown in fig. 4, the electronic device 300 provided by the present application includes a processor 301 and a memory 302, where the processor 301 is configured to implement steps of the method for removing the raindrops in the image according to the above embodiment of the present application when executing a computer program stored in the memory 302, for example:
acquiring a raindrop image to be processed;
inputting the raindrop image into a trained raindrop feature recognition model to obtain raindrop feature information in the raindrop image;
and removing raindrops in the raindrop image according to the raindrop characteristic information to obtain a target raindrop-free image corresponding to the raindrop image.
By way of example, a computer program may be partitioned into one or more modules/units that are stored in memory 302 and executed by processor 301 to accomplish an embodiment of the application. One or more of the modules/units may be a series of computer program instruction segments capable of performing particular functions to describe the execution of the computer program in a computer device.
Electronic device 300 may include, but is not limited to, a processor 301, a memory 302. It will be appreciated by those skilled in the art that the illustration is merely an example of the electronic device 300 and is not limiting of the electronic device 300, and may include more or fewer components than shown, or may combine some of the components, or different components, e.g., the electronic device 300 may further include an input-output device, a network access device, a bus, etc., through which the processor 301, the memory 302, the input-output device, the network access device, etc., are connected.
The processor 301 may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or any conventional processor or the like that is a control center of the electronic device 300 that interfaces and lines to various portions of the overall electronic device 300.
The memory 302 may be used to store computer programs and/or modules, and the processor 301 implements various functions of the computer device by running or executing the computer programs and/or modules stored in the memory 302 and invoking data stored in the memory 302. The memory 302 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data (such as audio data, video data, etc.) created according to the use of the electronic device 300, and the like. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, smart Media Card (SMC), secure Digital (SD) Card, flash Card (Flash Card), at least one disk storage device, flash memory device, or other volatile solid-state storage device.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, the above-described apparatus for removing raindrops in an image, the specific working process of the electronic device 300 and the corresponding units thereof may refer to the description of the method for removing raindrops in an image in the above embodiment of the present application, which is not repeated herein.
Those of ordinary skill in the art will appreciate that all or a portion of the steps of the various methods of the above embodiments may be performed by instructions, or by instructions controlling associated hardware, which may be stored in a computer-readable storage medium and loaded and executed by a processor.
To this end, an embodiment of the present application provides a computer readable storage medium having stored therein a plurality of instructions capable of being loaded by a processor to perform steps in the method for removing raindrops in an image in the above embodiment of the present application, for example:
acquiring a raindrop image to be processed;
inputting the raindrop image into a trained raindrop feature recognition model to obtain raindrop feature information in the raindrop image;
and removing raindrops in the raindrop image according to the raindrop characteristic information to obtain a target raindrop-free image corresponding to the raindrop image.
The specific operation may refer to the description of the method for removing the raindrops in the image in the above embodiment of the present application, which is not repeated here.
Wherein the computer-readable storage medium may comprise: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like.
Because the instructions stored in the computer readable storage medium may execute the steps in the method for removing raindrops in an image in the above embodiment of the present application, the beneficial effects that can be achieved by the method for removing raindrops in an image in the above embodiment of the present application can be achieved, which are detailed in the foregoing description and will not be repeated here.
Furthermore, the terms "first," "second," and "third," and the like, herein, are used for distinguishing between different objects and not for describing a particular sequential order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or modules is not limited to the particular steps or modules listed and certain embodiments may include additional steps or modules not listed or inherent to such process, method, article, or apparatus.
The method, the device, the electronic equipment and the storage medium for removing the raindrops in the image provided by the application are described in detail, and specific examples are applied to the principle and the implementation of the application, and the description of the above examples is only used for helping to understand the method and the core idea of the application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in light of the ideas of the present application, the present description should not be construed as limiting the present application.
Claims (10)
1. A method of raindrop removal in an image, comprising:
acquiring a raindrop image to be processed;
inputting the raindrop image into a trained raindrop feature recognition model to obtain raindrop feature information in the raindrop image;
and removing raindrops in the raindrop image according to the raindrop characteristic information to obtain a target raindrop-free image corresponding to the raindrop image.
2. The method for removing raindrops in an image according to claim 1, wherein the raindrop feature recognition model is trained according to the following steps:
the method comprises the steps of obtaining a training sample set, wherein the training sample set comprises a plurality of training sample sets, and each training sample set comprises a group of visible light images and invisible light images which are obtained under the same condition when the rainy condition exists;
training the neural network model according to the training sample set until a preset training stopping condition is met;
and taking the neural network model meeting the preset training stop condition as the raindrop characteristic recognition model.
3. The method for removing raindrops in an image according to claim 2, wherein the same condition refers to the same shooting scene at the same time and the same place, and the invisible light image is an infrared light image.
4. A method of raindrop removal in an image according to claim 3, further comprising, prior to training the neural network model according to the training sample set:
acquiring first image characteristic information corresponding to the visible light image and second image characteristic information corresponding to the infrared light image in each training sample group;
acquiring image difference information according to the first image characteristic information and the second image characteristic information;
taking the image characteristic difference information as target raindrop characteristic information;
and acquiring target raindrop characteristic information in all training sample groups to obtain a raindrop characteristic information set.
5. A method of raindrop removal in an image according to claim 3, wherein said training a neural network model from said training sample set comprises:
and training the neural network model according to the raindrop characteristic information set.
6. The method for removing raindrops in an image according to claim 4, wherein the obtaining the first image feature information corresponding to the visible light image in each training sample set includes:
carrying out gray scale processing on the visible light image to obtain a gray scale image;
and acquiring the first image characteristic information according to the gray level image.
7. The method for removing raindrops in an image according to claim 1, wherein the performing a raindrop removal process on raindrops in the raindrop image according to the raindrop feature information to obtain a target raindrop-free image corresponding to the raindrop image comprises:
filtering the image with the raindrops according to the raindrop characteristic information to obtain a filtered image;
and performing image restoration processing on the filtered image to obtain a target raindrop-free image corresponding to the raindrop-containing image.
8. A device for removing raindrops in an image, characterized by comprising means for performing the method for removing raindrops in an image according to any one of claims 1 to 7.
9. A computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when run on a computer, causes the computer to perform the method of raindrop removal in an image according to any one of claims 1 to 7.
10. An electronic device comprising a processor and a memory, the memory storing a computer program, characterized in that the processor is adapted to perform the method of raindrop removal in an image according to any one of claims 1 to 7 by invoking the computer program.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310638637.2A CN116645289A (en) | 2023-05-31 | 2023-05-31 | Method and device for removing raindrops in image, storage medium and electronic equipment |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310638637.2A CN116645289A (en) | 2023-05-31 | 2023-05-31 | Method and device for removing raindrops in image, storage medium and electronic equipment |
Publications (1)
Publication Number | Publication Date |
---|---|
CN116645289A true CN116645289A (en) | 2023-08-25 |
Family
ID=87643144
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202310638637.2A Pending CN116645289A (en) | 2023-05-31 | 2023-05-31 | Method and device for removing raindrops in image, storage medium and electronic equipment |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN116645289A (en) |
-
2023
- 2023-05-31 CN CN202310638637.2A patent/CN116645289A/en active Pending
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN112132156B (en) | Image saliency target detection method and system based on multi-depth feature fusion | |
CN110414507B (en) | License plate recognition method and device, computer equipment and storage medium | |
CN109840883B (en) | Method and device for training object recognition neural network and computing equipment | |
CN111079764B (en) | Low-illumination license plate image recognition method and device based on deep learning | |
CN107704797B (en) | Real-time detection method, system and equipment based on pedestrians and vehicles in security video | |
CN113808098A (en) | Road disease identification method and device, electronic equipment and readable storage medium | |
CN111695609A (en) | Target damage degree determination method, target damage degree determination device, electronic device, and storage medium | |
CN112712005B (en) | Training method of recognition model, target recognition method and terminal equipment | |
AG | Development of portable automatic number plate recognition (ANPR) system on Raspberry Pi | |
CN112052907A (en) | Target detection method and device based on image edge information and storage medium | |
CN113505682A (en) | Living body detection method and device | |
CN112613508A (en) | Object identification method, device and equipment | |
CN109523570B (en) | Motion parameter calculation method and device | |
CN116958687A (en) | Unmanned aerial vehicle-oriented small target detection method and device based on improved DETR | |
CN111652242B (en) | Image processing method, device, electronic equipment and storage medium | |
CN112085680B (en) | Image processing method and device, electronic equipment and storage medium | |
CN116051373A (en) | Image stitching method, image stitching device and terminal equipment | |
CN116645289A (en) | Method and device for removing raindrops in image, storage medium and electronic equipment | |
CN112906679B (en) | Pedestrian re-identification method, system and related equipment based on human shape semantic segmentation | |
CN113239738B (en) | Image blurring detection method and blurring detection device | |
CN115346209A (en) | Motor vehicle three-dimensional target detection method and device and computer readable storage medium | |
CN116206229A (en) | Target detection method and device | |
CN114359572A (en) | Training method and device of multi-task detection model and terminal equipment | |
CN111368579B (en) | Image recognition method, apparatus and system, and computer-readable medium | |
CN116189132B (en) | Training method for target detection model of road information, target detection method and device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication |