CN108898592A - Prompt method and device, the electronic equipment of camera lens degree of fouling - Google Patents

Prompt method and device, the electronic equipment of camera lens degree of fouling Download PDF

Info

Publication number
CN108898592A
CN108898592A CN201810654629.6A CN201810654629A CN108898592A CN 108898592 A CN108898592 A CN 108898592A CN 201810654629 A CN201810654629 A CN 201810654629A CN 108898592 A CN108898592 A CN 108898592A
Authority
CN
China
Prior art keywords
image
fouling
characteristic
degree
coding characteristic
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201810654629.6A
Other languages
Chinese (zh)
Inventor
杨松
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Xiaomi Mobile Software Co Ltd
Original Assignee
Beijing Xiaomi Mobile Software Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Xiaomi Mobile Software Co Ltd filed Critical Beijing Xiaomi Mobile Software Co Ltd
Priority to CN201810654629.6A priority Critical patent/CN108898592A/en
Publication of CN108898592A publication Critical patent/CN108898592A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N17/00Diagnosis, testing or measuring for television systems or their details
    • H04N17/002Diagnosis, testing or measuring for television systems or their details for television cameras
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10016Video; Image sequence
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)

Abstract

The disclosure is directed to a kind of method and devices for prompting camera lens degree of fouling, electronic equipment.It is a kind of prompt camera lens degree of fouling method include:Obtain the coding characteristic that number of frames image is continuously set in video flowing;The degree of fouling of the camera lens is determined based on the coding characteristic;Determine whether that user pushes prompting message according to the degree of fouling.It can be using setting number of frames image as detection sample in the present embodiment, make to detect sample to include more identification features, i.e. feature image-related is more in coding characteristic, can be improved the accuracy of detector lens degree of fouling, and then promotes the accuracy to user's camera prompting message.It also, in the present embodiment after camera lens is dirty, may remind the user that by pushing prompting message, promote user to clean camera lens in time to keep the cleaning of camera lens, so as to take the image of better quality, be conducive to the shooting experience for promoting user.

Description

Prompt method and device, the electronic equipment of camera lens degree of fouling
Technical field
This disclosure relates to technical field of image processing more particularly to it is a kind of prompt camera lens degree of fouling method and device, Electronic equipment.
Background technique
Currently, most of electronic equipment is provided with camera.In use, user usually require adjustment grip position or Moving finger finds fingerprint sensor, and such finger can often touch camera mirror surface, be formed on camera mirror surface dirty. After mirror surface is dirty, the quality of captured image can decline, to reduce the shooting experience of user.
Summary of the invention
The disclosure provides a kind of method and device of prompt camera lens degree of fouling, electronic equipment, to solve in the related technology The problem of user has found that mirror surface is dirty not in time and leads to captured image quality decrease.
According to the first aspect of the embodiments of the present disclosure, a kind of method prompting camera lens degree of fouling is provided, including:
Obtain the coding characteristic that number of frames image is continuously set in video flowing;
The degree of fouling of the camera lens is determined based on the coding characteristic;
Determine whether that user pushes prompting message according to the degree of fouling.
Optionally, it obtains and continuously sets the coding characteristic of number of frames image in video flowing and include:
The characteristics of image for successively obtaining every frame image in video flowing, obtains the characteristics of image of setting number of frames image;
Based on the characteristics of image of the setting number of frames image, the characteristic sequence of the setting number of frames image is determined;
The coding characteristic of the setting number of frames image is obtained based on the characteristic sequence.
Optionally, it obtains and continuously sets the coding characteristic of number of frames image in video flowing and include:
It obtains and continuously sets number of frames image in the video flowing, obtain image recognition collection;
The characteristics of image that every frame image is concentrated in described image identification is obtained, the characteristic sequence of described image identification collection is obtained;
The coding characteristic of described image identification collection is obtained according to the characteristic sequence.
Optionally, the characteristics of image of every frame image is obtained using convolutional neural networks;The coding characteristic utilizes circulation mind It is obtained through network.
Optionally, before the characteristics of image for obtaining every frame image, the method also includes:
According to the quantity of input terminal in the input layer of the neural network by the size adjusting of every frame image to being sized.
According to the second aspect of an embodiment of the present disclosure, a kind of device prompting camera lens degree of fouling is provided, including:
Coding characteristic obtains module, for obtaining the coding characteristic for continuously setting number of frames image in video flowing;
Degree of fouling determining module, for determining the degree of fouling of the camera lens based on the coding characteristic;
Prompting message pushing module, for determining whether that user pushes prompting message according to the degree of fouling.
Optionally, the coding characteristic acquisition module includes:
Characteristics of image acquiring unit obtains setting quantity for successively obtaining the characteristics of image of every frame image in video flowing The characteristics of image of frame image;
Characteristic sequence acquiring unit determines the setting number for the characteristics of image based on the setting number of frames image Measure the characteristic sequence of frame image;
Coding characteristic acquiring unit, the coding for obtaining the setting number of frames image based on the characteristic sequence are special Sign.
Optionally, the coding characteristic acquisition module includes:
Identification collection acquiring unit, continuously sets number of frames image for obtaining, obtains image recognition in the video flowing Collection;
Characteristic sequence acquiring unit concentrates the characteristics of image of every frame image for obtaining described image identification, obtains described The characteristic sequence of image recognition collection;
Coding characteristic acquiring unit, for obtaining the coding characteristic of described image identification collection according to the characteristic sequence.
Optionally, described image feature acquiring unit obtains the characteristics of image of every frame image using convolutional neural networks, and The coding characteristic acquiring unit obtains the coding characteristic of setting number of frames image using Recognition with Recurrent Neural Network;
Alternatively,
The characteristic sequence acquiring unit obtains the characteristics of image of every frame image, and the coding using convolutional neural networks Feature acquiring unit obtains the coding characteristic using Recognition with Recurrent Neural Network.
Optionally, the coding characteristic acquisition module further includes:
Image scaling unit, the quantity for input terminal in the input layer according to the neural network is by every frame image Size adjusting to being sized, and be sent to described image feature acquiring unit or the characteristic sequence acquiring unit.
According to the third aspect of an embodiment of the present disclosure, a kind of electronic equipment is provided, including:
Processor;
For storing the memory of the processor-executable instruction;
Wherein, the processor is configured to executing the executable instruction in the memory to realize described in first aspect The step of method.
According to a fourth aspect of embodiments of the present disclosure, a kind of computer readable storage medium is provided, calculating is stored thereon with Machine program, when which is executed by processor the step of realization first aspect the method.
The technical scheme provided by this disclosed embodiment can include the following benefits:
As can be seen from the above embodiments, number of frames image is continuously set by obtaining in video flowing in the embodiment of the present disclosure Coding characteristic;It is then based on the degree of fouling that coding characteristic determines camera lens;Finally determine whether to use according to the degree of fouling Family pushes prompting message.It can make to detect sample to include more using setting number of frames image as detection sample in the present embodiment More identification features, i.e., feature image-related is more in coding characteristic, can be improved the accuracy of detector lens degree of fouling, The accuracy to user's camera prompting message is promoted in turn.Also, in the present embodiment after camera lens is dirty, pass through push Prompting message may remind the user that, user is promoted to clean camera lens in time to keep the cleaning of camera lens, higher so as to take The image of quality is conducive to the shooting experience for promoting user.
It should be understood that above general description and following detailed description be only it is exemplary and explanatory, not The disclosure can be limited.
Detailed description of the invention
The drawings herein are incorporated into the specification and forms part of this specification, and shows the implementation for meeting the disclosure Example, and together with specification for explaining the principles of this disclosure.
Fig. 1 is a kind of flow diagram of method for prompting camera lens degree of fouling shown according to an exemplary embodiment;
Fig. 2 is a kind of process signal of the method for the prompt camera lens degree of fouling shown according to another exemplary embodiment Figure;
Fig. 3 is the result schematic diagram of each step in embodiment illustrated in fig. 2;
Fig. 4 is a kind of application scenarios schematic diagram shown according to an exemplary embodiment;
Fig. 5 is a kind of process signal of the method for the prompt camera lens degree of fouling shown according to a further exemplary embodiment Figure;
Fig. 6 is the result schematic diagram of each step in embodiment illustrated in fig. 5;
A kind of Fig. 7~Figure 11 block diagram for the device for prompting camera lens degree of fouling shown according to an exemplary embodiment;
Figure 12 is the block diagram of a kind of electronic equipment shown according to an exemplary embodiment.
Specific embodiment
Example embodiments are described in detail here, and the example is illustrated in the accompanying drawings.Following description is related to When attached drawing, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements.Following exemplary embodiment Described in embodiment do not represent all implementations consistent with this disclosure.On the contrary, they be only with it is such as appended The consistent device example of some aspects be described in detail in claims, the disclosure.
Currently, most of electronic equipment is provided with camera.In use, user usually require adjustment grip position or Moving finger finds fingerprint sensor, and such finger can often touch camera mirror surface, be formed on camera mirror surface dirty. After mirror surface is dirty, the quality of captured image can decline, to reduce the shooting experience of user.
In order to solve the above technical problems, the embodiment of the invention provides a kind of method for prompting camera lens degree of fouling, Fig. 1 is A kind of flow diagram of method prompting camera lens degree of fouling shown according to an exemplary embodiment.It will be appreciated that this The method for the prompt camera lens degree of fouling that inventive embodiments provide can be applied to the electronics such as computer, mobile phone, tablet computer and set It is standby.For convenience of description, it is described by taking smart phone as an example in subsequent embodiment.
Referring to Fig. 1, it is a kind of prompt camera lens degree of fouling method include step 101~step 103:
101, obtain the coding characteristic that number of frames image is continuously set in video flowing.
In the present embodiment, after the camera module of smart phone is opened, image can be acquired in real time or according to the setting period, To form video flowing.Also, camera module can store video flowing to designated position, such as memory or caching, To form offline video stream.Certainly, video flowing can also be sent in real time caching or processor by camera module, thus Form live video stream.
In the present embodiment, the processor of smart phone can read offline video stream from designated position or receive camera The live video stream that mould group is sent.Then, processor obtains coding characteristic based on the video flowing of acquisition, may include with lower section Formula:
Mode one
In the method, processor successively obtains every frame image in video flowing, then obtains the characteristics of image of every frame image, from And obtain the characteristics of image of setting number of frames image.Later, processor is determined based on the characteristics of image of setting number of frames image Set the characteristic sequence of number of frames image.Finally, processor obtains the coding characteristic of setting number of frames image based on characteristic sequence. This programme will be described in detail in subsequent embodiment, be first not illustrated herein.
It should be noted that setting quantity can be configured according to concrete scene, such as 10~30.In an embodiment In, setting quantity is set as 20.
It should be noted that mode one can be applied to the scene that video flowing is Online Video stream.For example, camera module The every frame image that can be will acquire is timely transmitted to processor, so that convenient processor is handled in time.
Mode two
In the method, video flowing is divided into multiple images identification collection by processor, wherein each image recognition collection includes setting Determine number of frames image.Then, for each image recognition collection, processor obtains image recognition and concentrates the image of every frame image special Sign, the characteristics of image of every frame image can form the characteristic sequence of image recognition collection.Finally, processor is obtained based on characteristic sequence Set the coding characteristic of number of frames image.This programme will be described in detail in subsequent embodiment, be first not illustrated herein.
It should be noted that processor divides image recognition collection after processor marks off first image recognition collection The process of subsequent process and processor identification coding characteristic can carry out side by side, to improve treatment effeciency, be conducive to improve real Shi Xing.
It should be noted that mode two can be applied to the scene that video flowing is offline video stream.
102, the degree of fouling of the camera lens is determined based on the coding characteristic.
In the present embodiment, degree of fouling may include dirty and not dirty.Certainly, in some scenes, technology people Member can also continue to refine to degree of fouling, such as dirty can be subdivided into:Dirty serious (can not shoot), dirty one As (shooting quality being influenced heavier), dirty relatively light (shooting quality being influenced lighter).For another example, not dirty to be subdivided into:It is dirty Dirty (needing professional personnel that can just feel to have an impact to shooting quality) very light, it is not dirty (to have no shadow to shooting quality It rings).Technical staff can select suitable mode classification, be not limited thereto according to concrete scene.
Due to camera lens it is dirty after, will form one layer " white haze " on the image of shooting, to influence picture quality.This reality It applies in example, processor can determine the degree of fouling after whether camera lens is dirty and dirty according to " white haze " on image, It determines that the process of degree of fouling elaborates in subsequent implementation regular meeting, does not describe first herein.
In the present embodiment, processor can call pre-set recognizer, then using coding characteristic as the identification The input parameter of algorithm, the recognizer identify the corresponding classification of setting number of frames image.The classification with it is " white on image Mist " is closely related, i.e. the classification is corresponding with the degree of fouling of camera lens.In other words, processor can be determined according to coding characteristic The degree of fouling of the camera lens of shooting setting number of frames image out.
In the present embodiment, pre-set recognizer can be neural network algorithm, such as convolutional neural networks CNN, Recognition with Recurrent Neural Network RNN, long short-term memory artificial neural network LSTM etc..Certainly, pre-set recognizer can also be Logistic regression algorithm, such as clustering algorithm, deep learning algorithm, support vector machines, decision tree, linear regression etc..Technical staff Suitable recognizer can be selected according to concrete scene, be not limited thereto.
103, determine whether that user pushes prompting message according to the degree of fouling.
In the present embodiment, processor determines whether to push prompting message according to degree of fouling.Wherein prompting message can be pre- First it is arranged, such as " camera lens is dirty, influences whether the quality of shooting image, please cleans in time ", the content of prompting message is not made It limits.
In one embodiment, when degree of fouling indicates that camera lens is not dirty, processor is determined to be prompted without pushing to user Message.
In another embodiment, when degree of fouling indicates that camera lens is dirty, the true directional user's push of processor is preparatory The prompting message of setting.
So far, can make to detect sample to include more using setting number of frames image as detection sample in the present embodiment Identification feature, i.e., feature image-related is more in coding characteristic, can be improved the accuracy of detector lens degree of fouling, into And promote the accuracy to user's camera prompting message.Also, in the present embodiment after camera lens is dirty, mentioned by push Show that message may remind the user that, user is promoted to clean camera lens in time to keep the cleaning of camera lens, it is more high-quality so as to take The image of amount is conducive to the shooting experience for promoting user.
Fig. 2 is a kind of flow diagram of method for prompting camera lens degree of fouling shown according to an exemplary embodiment. Referring to fig. 2, a method of prompt camera lens degree of fouling, the field of the degree of fouling suitable for the real-time detector lens of smart phone Scape, including:
After the camera module of smart phone is triggered, video flowing is formed in real time or according to setting period acquisition image, By the video stream to the processor of smart phone.
Referring to Fig. 3, after getting every frame image, processor calls image feature recognition algorithms to get the frame image Characteristics of image.Wherein, characteristics of image recognizer can be preparatory trained neural network, such as convolutional neural networks CNN Or deep-neural-network.Every frame image is input in neural network by processor as input parameter can be obtained the frame image Characteristics of image fm, fm be a feature vector, and m indicates the serial number of image in video streaming.
In the present embodiment, it is contemplated that the computing resource of smart phone, above-mentioned neural network can use MobileNet net The light-weighted networks such as network, ShuffleNet network, SqueezeNet network, so as to save the calculating money of smart phone Source.It certainly, can be using other minds when a kind of method for prompting camera lens degree of fouling is applied to such as PC, server equipment Through network.Technical staff can select according to concrete scene, be not limited thereto.
In practical application, since the size of every frame image of camera module shooting is different, and after training CNN network it is defeated It is then fixed for entering input terminal quantity in layer, therefore processor is before the characteristics of image for obtaining every frame image, it is also necessary to according to Input terminal quantity is sized the number with input terminal by the size adjusting of every frame image to being sized in CNN network input layer It measures identical.For example, by every frame Image Adjusting to 224*224 (pixel).By adjusting the size of image, Neng Gouti in the present embodiment Rise the efficiency that neural network extracts characteristics of image.
While obtaining every frame image features, or after getting the characteristics of image of a frame image, processor can be with The frame number N of the characteristics of image got is counted, and judging whether frame number N reaches setting quantity n, n and N is all natural number. If N is less than n, processor continues to obtain next frame image, and obtains the characteristics of image of next frame image.If N is equal to n, Processor is by the characteristics of image of n frame image according to sequence composition characteristic sequence F, the F={ f of image1,f2,f3,f4,f5,f6,…, fn-1,fn}。
Then, processor can call pre-set encryption algorithm, using characteristic sequence F as the input of the encryption algorithm Parameter, since encryption algorithm calculates the corresponding coding characteristic of characteristic sequence F.Wherein, encryption algorithm can be trains in advance Neural network, such as Recognition with Recurrent Neural Network RNN.In this step, available this combination of setting number of frames image of processor Feature, that is, coding characteristic Fencoder
Later, processor can call pre-set Logic Regression Models, by Logic Regression Models according to coding characteristic FencoderDetermine the degree of fouling of camera lens.In the present embodiment, Logic Regression Models can be multi-layer perception (MLP) (MLP) or BP Neural network etc., main function are that its corresponding classification is determined according to coding characteristic, and classification is the degree of fouling of camera lens.Skill Art personnel can select suitable model to identify degree of fouling according to concrete scene, be not limited thereto.
It should be noted that the degree of fouling that will acquire camera lens in the present embodiment is divided into two steps to realize, main mesh Be can in advance individually training obtain characteristic sequence neural network and obtain degree of fouling Logic Regression Models.This Sample, neural network and logistic regression mould group can be separated and be trained, and two training process can use different sample images It trains, can promote the utilization rate of sample image as far as possible.In addition, neural network or Logic Regression Models carry out individually Training, accumulated error when can reduce while train are extracted the accuracy of characteristic sequence and are mentioned to improve neural network The accuracy of high Logic Regression Models identification degree of fouling.In other words, characteristic procedure and identification feature will be extracted in the present embodiment Process decoupling makes the training of the two without directly connection, can not only promote the utilization rate of sample image, and each mistake can be improved The accuracy of journey.
Finally, processor determines whether camera prompting message according to degree of fouling.If it is determined that push prompting message, then locate Reason device can trigger the display of smart phone, then can be played in display frame, bubbling, variation icon, in voice One or more kinds of setting means push prompting message to user.Fig. 4 is a kind of application shown according to an exemplary embodiment Scene figure.Referring to fig. 4, for processor after determining push prompting message, processor triggers the display 402 of smart phone 401, Superposition plays frame 404, and the display reminding message 405 in playing frame in the present image 403 of display 402, to keep user timely Read prompting message 405.For another example, processor can trigger the loudspeaker 406 of smart phone, will be mentioned in a manner of voice broadcast Show that message is presented to user, achievees the purpose that remind user in time.
So far, processor passes through the characteristics of image of real-time every frame image in the present embodiment, then to set number of frames image It can be improved detector lens degree of fouling by obtaining feature, that is, coding characteristic of setting number of frames image as detection sample Accuracy, and then promote the accuracy to user's camera prompting message.Also, it is dirty in camera lens in the present embodiment Afterwards, it can achieve the purpose of prompting user in real time by pushing prompting message in real time, user promoted to clean camera lens in time to keep The cleaning of camera lens is conducive to the shooting experience for promoting user so as to take the image of better quality.
Fig. 5 is a kind of flow diagram of method for prompting camera lens degree of fouling shown according to an exemplary embodiment. Referring to Fig. 5, it is a kind of prompt camera lens degree of fouling method include
After the offline video that user triggers designated position, processor can read the video flowing and be handled.Referring to figure 6, processor first obtains preset setting quantity, and according to the setting quantity by video flowing be divided into multiple images identification collection 1~ n。
For one of image recognition collection x, with continued reference to Fig. 6, processor gets every frame of image recognition concentration Then image calls image feature recognition algorithms to get characteristics of image f1.x~fn.x of the frame image, until having handled Image recognition collection, combine each frame image characteristics of image obtain the characteristic sequence Fx, Fx=of the image recognition collection f1.1, F2.1 ..., fn.x }.Wherein fn.x indicates that x-th of image recognition concentrates the characteristics of image of n-th frame image.
Then, processor can call pre-set encryption algorithm, using characteristic sequence Fx as the defeated of the encryption algorithm Enter parameter, since encryption algorithm calculates the corresponding coding characteristic F of characteristic sequence Fxencoderx.Encryption algorithm may refer to Fig. 2 institute Show the set-up mode of encryption algorithm in embodiment, details are not described herein.
Later, processor can call pre-set Logic Regression Models, by Logic Regression Models according to coding characteristic FencoderxDetermine the degree of fouling x of camera lens.Logic Regression Models may refer to Logic Regression Models in embodiment illustrated in fig. 2 Set-up mode, details are not described herein.
In the present embodiment, the training method of encryption algorithm and Logic Regression Models may refer to compile in embodiment illustrated in fig. 2 The training method of code algorithm and Logic Regression Models, details are not described herein.
Finally, processor determines whether camera prompting message according to degree of fouling.The mode for pushing prompting message can be with The mode of push prompting message shown in Figure 2, details are not described herein.
So far, processor passes through the characteristics of image for analyzing every frame image in offline video stream in the present embodiment, then to set Number of frames image, which is determined, as detection sample then passes through coding by obtaining feature, that is, coding characteristic of setting number of frames image Feature can determine the degree of fouling of the camera lens of shooting offline video stream.Dirty journey is determined using multiple image in the present embodiment The accuracy of detector lens degree of fouling can be improved in degree, and then promotes the accuracy to user's camera prompting message.Also, In the present embodiment after camera lens is dirty, it may remind the user that by pushing prompting message, user promoted to clean camera lens in time To keep the cleaning of camera lens, the image of better quality can be taken when shooting video next time in this way, is conducive to promote user Shooting experience.
Fig. 7 is a kind of block diagram of device for prompting camera lens degree of fouling shown according to an exemplary embodiment.Referring to figure 7, it is a kind of prompt camera lens degree of fouling device 700 include:
Coding characteristic obtains module 701, for obtaining the coding characteristic for continuously setting number of frames image in video flowing;
Degree of fouling determining module 702, for determining the degree of fouling of the camera lens based on the coding characteristic;
Prompting message pushing module 703, for determining whether that user pushes prompting message according to the degree of fouling.
So far, coding characteristic obtains in the acquisition video flowing of module 701 and continuously sets number of frames image in the present embodiment Then coding characteristic is determined the degree of fouling of camera lens, finally by prompting by degree of fouling determining module 702 based on the coding characteristic Message pushing module 703 determines whether that user pushes prompting message according to degree of fouling.In this way, can be used in the present embodiment Set number of frames image as detection sample, due to detection sample include more identification feature, that is, coding characteristics in it is image-related Feature it is more, can be improved the accuracy of detector lens degree of fouling, and then promote the standard to user's camera prompting message Exactness.Also, in the present embodiment after camera lens is dirty, by push prompting message may remind the user that, promote user and Shi Qingxi camera lens is to keep the cleaning of camera lens to be conducive to the shooting for promoting user so as to take the image of better quality Experience.
Fig. 8 is a kind of block diagram of device for prompting camera lens degree of fouling shown according to an exemplary embodiment.Referring to figure 8, on the basis of a kind of device 700 of prompt camera lens degree of fouling shown in Fig. 7, coding characteristic obtains module 701 and can wrap It includes:
Characteristics of image acquiring unit 801 obtains setting number for successively obtaining the characteristics of image of every frame image in video flowing Measure the characteristics of image of frame image;
Characteristic sequence acquiring unit 802 determines the setting for the characteristics of image based on the setting number of frames image The characteristic sequence of number of frames image;
Coding characteristic acquiring unit 803, for obtaining the coding of the setting number of frames image based on the characteristic sequence Feature.
In one embodiment, the image that characteristics of image acquiring unit 801 obtains every frame image using convolutional neural networks is special Sign, and coding characteristic acquiring unit 803 obtains the coding characteristic of setting number of frames image using Recognition with Recurrent Neural Network.
So far, the characteristics of image for obtaining every frame image in the present embodiment by characteristics of image acquiring unit 801, then by feature Retrieval unit 802 determines the characteristic sequence of setting number of frames image based on setting number of frames image, finally by coding characteristic Acquiring unit 803 obtains the coding characteristic of setting number of frames image based on particular sequence.In this way, coding characteristic obtains module 701 It is adapted to the scene of live video stream, it is convenient to receive the dirty journey for setting number of frames pictures subsequent and determining camera lens Degree, can be improved the accuracy and real-time of the definitive result of degree of fouling.
Fig. 9 is a kind of block diagram of device for prompting camera lens degree of fouling shown according to an exemplary embodiment.Referring to figure 9, on the basis of a kind of device 700 of prompt camera lens degree of fouling shown in Fig. 8, coding characteristic obtains module 701 can be with Including:
Image scaling unit 901, the quantity for input terminal in the input layer according to the neural network is by every frame The size adjusting of image to being sized, and be sent to described image feature acquiring unit or the characteristic sequence obtain it is single Member.
So far, nerve can be improved by the size that image scaling unit 901 adjusts every frame image in the present embodiment Network obtains the speed of characteristics of image, promotes the efficiency of subsequent degree of fouling.
Figure 10 is a kind of block diagram of device for prompting camera lens degree of fouling shown according to an exemplary embodiment.Referring to figure 10, on the basis of a kind of device 700 of prompt camera lens degree of fouling shown in Fig. 7, coding characteristic obtains module 701 and can wrap It includes:
Identification collection acquiring unit 1001, continuously sets number of frames image for obtaining, obtains image in the video flowing Identification collection;
Characteristic sequence acquiring unit 1002 is concentrated the characteristics of image of every frame image for obtaining described image identification, is obtained The characteristic sequence of described image identification collection;
Coding characteristic acquiring unit 1003, the coding for obtaining described image identification collection according to the characteristic sequence are special Sign.
In one embodiment, the image that characteristic sequence acquiring unit 1002 obtains every frame image using convolutional neural networks is special Sign, and coding characteristic acquiring unit 1003 obtains the coding characteristic using Recognition with Recurrent Neural Network.
So far, video flowing is divided by identification collection acquiring unit 1001 in the present embodiment and obtains image recognition collection, then by spy It levies retrieval unit 1002 and obtains the characteristics of image that every frame image is concentrated in described image identification, obtain described image identification collection Characteristic sequence, the coding for finally obtaining described image identification collection according to the characteristic sequence by coding characteristic acquiring unit 1003 are special Sign.In this way, coding characteristic obtains the scene that module 701 is adapted to offline video stream, the convenient process in viewing video flowing The degree of fouling of the camera lens of middle determining shooting image, can be improved the accuracy for the degree of fouling determined.
Figure 11 is a kind of block diagram of device for prompting camera lens degree of fouling shown according to an exemplary embodiment.Referring to figure 11, on the basis of a kind of device 700 for prompting camera lens degree of fouling shown in Fig. 10, coding characteristic obtains module 701 and may be used also To include:
Image scaling unit 1101, the quantity for input terminal in the input layer according to the neural network is by every frame The size adjusting of image to being sized, and be sent to described image feature acquiring unit or the characteristic sequence obtain it is single Member.
So far, nerve can be improved by the size that image scaling unit 1101 adjusts every frame image in the present embodiment Network obtains the speed of characteristics of image, promotes the efficiency of subsequent degree of fouling.
Figure 12 is the block diagram of a kind of electronic equipment shown according to an exemplary embodiment.For example, electronic equipment 1200 can To be mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, Medical Devices are good for Body equipment, personal digital assistant etc..
Referring to Fig.1 2, electronic equipment 1200 may include following one or more components:Processing component 1202, memory 1204, power supply module 1206, multimedia component 1208, audio component 1210, the interface 1212 of input/output (I/O), sensor Component 1214 and communication component 1216.Wherein, memory 1204 is used to store the executable instruction of processing component 1202.Place It manages component 1202 and reads instruction from memory 1204 to realize:
Obtain the coding characteristic that number of frames image is continuously set in video flowing;
The degree of fouling of the camera lens is determined based on the coding characteristic;
Determine whether that user pushes prompting message according to the degree of fouling.
The integrated operation of the usual controlling electronic devices 1200 of processing component 1202, such as with display, call, data are logical Letter, camera operation and record operate associated operation.Processing component 1202 may include one or more processors 1220 It executes instruction.In addition, processing component 1202 may include one or more modules, convenient for processing component 1202 and other assemblies it Between interaction.For example, processing component 1202 may include multi-media module, to facilitate multimedia component 1208 and processing component Interaction between 1202.
Memory 1204 is configured as storing various types of data to support the operation in electronic equipment 1200.These numbers According to example include any application or method for being operated on electronic equipment 1200 instruction, contact data, electricity Talk about book data, message, picture, video etc..Memory 1204 can be by any kind of volatibility or non-volatile memory device Or their combination is realized, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), Erasable Programmable Read Only Memory EPROM (EPROM), programmable read only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, disk or CD.
Power supply module 1206 provides electric power for the various assemblies of electronic equipment 1200.Power supply module 1206 may include power supply Management system, one or more power supplys and other with for electronic equipment 1200 generate, manage, and distribute associated group of electric power Part.
Multimedia component 1208 includes the screen of one output interface of offer between the electronic equipment 1200 and user Curtain.In some embodiments, screen may include liquid crystal display (LCD) and touch panel (TP).If screen includes touching Panel, screen may be implemented as touch screen, to receive input signal from the user.Touch panel includes one or more touchings Sensor is touched to sense the gesture on touch, slide, and touch panel.The touch sensor can not only sense touch or cunning The boundary of movement, but also detect duration and pressure associated with the touch or slide operation.In some embodiments In, multimedia component 1208 includes a front camera and/or rear camera.When electronic equipment 1200 is in operation mould Formula, such as in a shooting mode or a video mode, front camera and/or rear camera can receive external multi-medium data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom energy Power.
Audio component 1210 is configured as output and/or input audio signal.For example, audio component 1210 includes a wheat Gram wind (MIC), when electronic equipment 1200 is in operation mode, when such as call mode, recording mode, and voice recognition mode, Mike Wind is configured as receiving external audio signal.The received audio signal can be further stored in memory 1204 or via Communication component 1216 is sent.In some embodiments, audio component 1210 further includes a loudspeaker, for exporting audio letter Number.
I/O interface 1212 provides interface, above-mentioned peripheral interface module between processing component 1202 and peripheral interface module It can be keyboard, click wheel, button etc..These buttons may include, but are not limited to:Home button, volume button, start button and Locking press button.
Sensor module 1214 includes one or more sensors, for providing the shape of various aspects for electronic equipment 1200 State assessment.For example, sensor module 1214 can detecte the state that opens/closes of electronic equipment 1200, component it is relatively fixed Position, such as the component are the display and keypad of electronic equipment 1200, and sensor module 1214 can also detect electronics and set For 1200 or the position change of 1,200 1 components of electronic equipment, the existence or non-existence that user contacts with electronic equipment 1200, The temperature change in 1200 orientation of electronic equipment or acceleration/deceleration and electronic equipment 1200.Sensor module 1214 may include connecing Nearly sensor is configured to detect the presence of nearby objects without any physical contact.Sensor module 1214 is also It may include optical sensor, such as CMOS or ccd image sensor, for being used in imaging applications.In some embodiments, should Sensor module 1214 can also include acceleration transducer, and gyro sensor, Magnetic Sensor, pressure sensor or temperature pass Sensor.
Communication component 1216 is configured to facilitate the logical of wired or wireless way between electronic equipment 1200 and other equipment Letter.Electronic equipment 1200 can access the wireless network according to communication standard, such as WiFi, 2G or 3G or their combination.One In a exemplary embodiment, communication component 1216 via broadcast channel receive broadcast singal from external broadcasting management system or Broadcast related information.In one exemplary embodiment, the communication component 1216 further includes near-field communication (NFC) module, with Promote short range communication.For example, can be surpassed according to radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology in NFC module Broadband (UWB) technology, bluetooth (BT) technology and other technologies are realized.
In the exemplary embodiment, electronic equipment 1200 can by one or more application specific integrated circuit (ASIC), Digital signal processor (DSP), digital signal processing appts (DSPD), programmable logic device (PLD), field-programmable gate array (FPGA), controller, microcontroller, microprocessor or other electronic components are arranged to realize.
In the exemplary embodiment, a kind of computer readable storage medium is additionally provided, can be ROM, arbitrary access is deposited Reservoir (RAM), CD-ROM, tape, floppy disk and optical data storage devices etc., are stored thereon with computer program, and processor can be with The reading program from computer readable storage medium, the step of realizing embodiment of the method shown in FIG. 1 to FIG. 6, particular content can be with Referring to the content of FIG. 1 to FIG. 6 illustrated embodiment, details are not described herein.
Those skilled in the art will readily occur to its of the disclosure after considering specification and practicing disclosure disclosed herein Its embodiment.This application is intended to cover any variations, uses, or adaptations of the disclosure, these modifications, purposes or Person's adaptive change follows the general principles of this disclosure and including the undocumented common knowledge in the art of the disclosure Or conventional techniques.The description and examples are only to be considered as illustrative, and the true scope and spirit of the disclosure are by following Claim is pointed out.
It should be understood that the present disclosure is not limited to the precise structures that have been described above and shown in the drawings, and And various modifications and changes may be made without departing from the scope thereof.The scope of the present disclosure is only limited by the accompanying claims.

Claims (12)

1. a kind of method for prompting camera lens degree of fouling, which is characterized in that the method includes:
Obtain the coding characteristic that number of frames image is continuously set in video flowing;
The degree of fouling of the camera lens is determined based on the coding characteristic;
Determine whether that user pushes prompting message according to the degree of fouling.
2. the method according to claim 1, wherein obtaining the volume for continuously setting number of frames image in video flowing Code feature include:
The characteristics of image for successively obtaining every frame image in video flowing, obtains the characteristics of image of setting number of frames image;
Based on the characteristics of image of the setting number of frames image, the characteristic sequence of the setting number of frames image is determined;
The coding characteristic of the setting number of frames image is obtained based on the characteristic sequence.
3. the method according to claim 1, wherein obtaining the volume for continuously setting number of frames image in video flowing Code feature include:
It obtains and continuously sets number of frames image in the video flowing, obtain image recognition collection;
The characteristics of image that every frame image is concentrated in described image identification is obtained, the characteristic sequence of described image identification collection is obtained;
The coding characteristic of described image identification collection is obtained according to the characteristic sequence.
4. according to the method in claim 2 or 3, which is characterized in that the characteristics of image of every frame image utilizes convolutional Neural net Network obtains;The coding characteristic is obtained using Recognition with Recurrent Neural Network.
5. according to the method described in claim 4, it is characterized in that, before the characteristics of image for obtaining every frame image, the method Further include:
According to the quantity of input terminal in the input layer of the neural network by the size adjusting of every frame image to being sized.
6. a kind of device for prompting camera lens degree of fouling, which is characterized in that described device includes:
Coding characteristic obtains module, for obtaining the coding characteristic for continuously setting number of frames image in video flowing;
Degree of fouling determining module, for determining the degree of fouling of the camera lens based on the coding characteristic;
Prompting message pushing module, for determining whether that user pushes prompting message according to the degree of fouling.
7. device according to claim 6, which is characterized in that the coding characteristic obtains module and includes:
Characteristics of image acquiring unit obtains setting number of frames figure for successively obtaining the characteristics of image of every frame image in video flowing The characteristics of image of picture;
Characteristic sequence acquiring unit determines the setting number of frames for the characteristics of image based on the setting number of frames image The characteristic sequence of image;
Coding characteristic acquiring unit, for obtaining the coding characteristic of the setting number of frames image based on the characteristic sequence.
8. device according to claim 6, which is characterized in that the coding characteristic obtains module and includes:
Identification collection acquiring unit, continuously sets number of frames image for obtaining, obtains image recognition collection in the video flowing;
Characteristic sequence acquiring unit concentrates the characteristics of image of every frame image for obtaining described image identification, obtains described image Identify the characteristic sequence of collection;
Coding characteristic acquiring unit, for obtaining the coding characteristic of described image identification collection according to the characteristic sequence.
9. device according to claim 7 or 8, which is characterized in that described image feature acquiring unit utilizes convolutional Neural Network obtains the characteristics of image of every frame image, and the coding characteristic acquiring unit obtains setting quantity using Recognition with Recurrent Neural Network The coding characteristic of frame image;
Alternatively,
The characteristic sequence acquiring unit obtains the characteristics of image of every frame image, and the coding characteristic using convolutional neural networks Acquiring unit obtains the coding characteristic using Recognition with Recurrent Neural Network.
10. device according to claim 9, which is characterized in that the coding characteristic obtains module and further includes:
Image scaling unit, the quantity for input terminal in the input layer according to the neural network is by the ruler of every frame image It is very little to adjust to being sized, and it is sent to described image feature acquiring unit or the characteristic sequence acquiring unit.
11. a kind of electronic equipment, which is characterized in that the electronic equipment includes:
Processor;
For storing the memory of the processor-executable instruction;
Wherein, the processor is configured to executing the executable instruction in the memory to realize that Claims 1 to 5 is any The step of item the method.
12. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is by processor The step of any one of Claims 1 to 5 the method is realized when execution.
CN201810654629.6A 2018-06-22 2018-06-22 Prompt method and device, the electronic equipment of camera lens degree of fouling Pending CN108898592A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810654629.6A CN108898592A (en) 2018-06-22 2018-06-22 Prompt method and device, the electronic equipment of camera lens degree of fouling

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810654629.6A CN108898592A (en) 2018-06-22 2018-06-22 Prompt method and device, the electronic equipment of camera lens degree of fouling

Publications (1)

Publication Number Publication Date
CN108898592A true CN108898592A (en) 2018-11-27

Family

ID=64345920

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810654629.6A Pending CN108898592A (en) 2018-06-22 2018-06-22 Prompt method and device, the electronic equipment of camera lens degree of fouling

Country Status (1)

Country Link
CN (1) CN108898592A (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109978139A (en) * 2019-03-20 2019-07-05 深圳大学 Picture automatically generates method, system, electronic device and the storage medium of description
CN110266914A (en) * 2019-07-23 2019-09-20 北京小米移动软件有限公司 Image capturing method, device and computer readable storage medium
CN110572636A (en) * 2019-07-23 2019-12-13 RealMe重庆移动通信有限公司 camera contamination detection method and device, storage medium and electronic equipment
CN111666840A (en) * 2020-05-25 2020-09-15 维沃移动通信有限公司 Information prompting method and device and electronic equipment
CN111845647A (en) * 2019-04-28 2020-10-30 上海汽车集团股份有限公司 Automobile camera cleaning system and method
CN112333441A (en) * 2020-10-28 2021-02-05 维沃移动通信有限公司 Camera detection method and device and electronic equipment
CN112804522A (en) * 2021-04-15 2021-05-14 魔门塔(苏州)科技有限公司 Method and device for detecting abnormal conditions of camera
CN113200022A (en) * 2021-04-28 2021-08-03 广州文远知行科技有限公司 Cleaning method, device and equipment of sensor and storage medium
CN113496172A (en) * 2020-04-03 2021-10-12 深圳爱根斯通科技有限公司 Fingerprint foreign matter detection method and device, electronic equipment and storage medium
CN113705790A (en) * 2021-08-31 2021-11-26 湖北航天技术研究院总体设计所 Window mirror cleaning method and device

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104539939A (en) * 2014-12-17 2015-04-22 惠州Tcl移动通信有限公司 Lens cleanliness detection method and system based on mobile terminal
CN105163110A (en) * 2015-09-02 2015-12-16 厦门美图之家科技有限公司 Camera cleanliness detection method and system and shooting terminal
CN105827914A (en) * 2016-04-20 2016-08-03 北京小米移动软件有限公司 Lens cleaning prompting method and device
CN106446782A (en) * 2016-08-29 2017-02-22 北京小米移动软件有限公司 Image identification method and device
CN107527059A (en) * 2017-08-07 2017-12-29 北京小米移动软件有限公司 Character recognition method, device and terminal
CN107743224A (en) * 2017-09-30 2018-02-27 奇酷互联网络科技(深圳)有限公司 The dirty based reminding method of camera lens, system, readable storage medium storing program for executing and mobile terminal

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104539939A (en) * 2014-12-17 2015-04-22 惠州Tcl移动通信有限公司 Lens cleanliness detection method and system based on mobile terminal
CN105163110A (en) * 2015-09-02 2015-12-16 厦门美图之家科技有限公司 Camera cleanliness detection method and system and shooting terminal
CN105827914A (en) * 2016-04-20 2016-08-03 北京小米移动软件有限公司 Lens cleaning prompting method and device
CN106446782A (en) * 2016-08-29 2017-02-22 北京小米移动软件有限公司 Image identification method and device
CN107527059A (en) * 2017-08-07 2017-12-29 北京小米移动软件有限公司 Character recognition method, device and terminal
CN107743224A (en) * 2017-09-30 2018-02-27 奇酷互联网络科技(深圳)有限公司 The dirty based reminding method of camera lens, system, readable storage medium storing program for executing and mobile terminal

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109978139A (en) * 2019-03-20 2019-07-05 深圳大学 Picture automatically generates method, system, electronic device and the storage medium of description
CN109978139B (en) * 2019-03-20 2021-06-04 深圳大学 Method, system, electronic device and storage medium for automatically generating description of picture
CN111845647A (en) * 2019-04-28 2020-10-30 上海汽车集团股份有限公司 Automobile camera cleaning system and method
CN110266914B (en) * 2019-07-23 2021-08-24 北京小米移动软件有限公司 Image shooting method, device and computer readable storage medium
CN110266914A (en) * 2019-07-23 2019-09-20 北京小米移动软件有限公司 Image capturing method, device and computer readable storage medium
CN110572636A (en) * 2019-07-23 2019-12-13 RealMe重庆移动通信有限公司 camera contamination detection method and device, storage medium and electronic equipment
EP3770849A1 (en) * 2019-07-23 2021-01-27 Beijing Xiaomi Mobile Software Co., Ltd. Image capturing method and device, computer readable storage medium
US11176648B2 (en) 2019-07-23 2021-11-16 Beijing Xiaomi Mobile Software Co., Ltd. Image capturing method and device, computer readable storage medium
CN113496172A (en) * 2020-04-03 2021-10-12 深圳爱根斯通科技有限公司 Fingerprint foreign matter detection method and device, electronic equipment and storage medium
CN111666840A (en) * 2020-05-25 2020-09-15 维沃移动通信有限公司 Information prompting method and device and electronic equipment
CN112333441A (en) * 2020-10-28 2021-02-05 维沃移动通信有限公司 Camera detection method and device and electronic equipment
CN112804522A (en) * 2021-04-15 2021-05-14 魔门塔(苏州)科技有限公司 Method and device for detecting abnormal conditions of camera
CN113200022A (en) * 2021-04-28 2021-08-03 广州文远知行科技有限公司 Cleaning method, device and equipment of sensor and storage medium
CN113705790A (en) * 2021-08-31 2021-11-26 湖北航天技术研究院总体设计所 Window mirror cleaning method and device

Similar Documents

Publication Publication Date Title
CN108898592A (en) Prompt method and device, the electronic equipment of camera lens degree of fouling
CN106339680B (en) Face key independent positioning method and device
CN105608425B (en) The method and device of classification storage is carried out to photo
CN106572299A (en) Camera switching-on method and device
US10115019B2 (en) Video categorization method and apparatus, and storage medium
CN106651955A (en) Method and device for positioning object in picture
CN106791893A (en) Net cast method and device
CN107018333A (en) Shoot template and recommend method, device and capture apparatus
CN108921134A (en) fingerprint collecting method, device, electronic equipment and storage medium
CN105809704A (en) Method and device for identifying image definition
CN109872297A (en) Image processing method and device, electronic equipment and storage medium
CN108668080B (en) Method and device for prompting degree of dirt of lens and electronic equipment
CN105205479A (en) Human face value evaluation method, device and terminal device
CN107123081A (en) image processing method, device and terminal
CN106408603A (en) Camera method and device
CN106682736A (en) Image identification method and apparatus
CN105357425B (en) Image capturing method and device
CN105447150B (en) Method for playing music, device and terminal device based on face photograph album
CN106600530B (en) Picture synthesis method and device
CN106778531A (en) Face detection method and device
CN110636315B (en) Multi-user virtual live broadcast method and device, electronic equipment and storage medium
CN105528078B (en) The method and device of controlling electronic devices
CN107679533A (en) Character recognition method and device
CN109168062A (en) Methods of exhibiting, device, terminal device and the storage medium of video playing
CN109145679A (en) A kind of method, apparatus and system issuing warning information

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20181127