CN113436137A - Image definition recognition method, device, equipment and medium - Google Patents
Image definition recognition method, device, equipment and medium Download PDFInfo
- Publication number
- CN113436137A CN113436137A CN202110271927.9A CN202110271927A CN113436137A CN 113436137 A CN113436137 A CN 113436137A CN 202110271927 A CN202110271927 A CN 202110271927A CN 113436137 A CN113436137 A CN 113436137A
- Authority
- CN
- China
- Prior art keywords
- image
- definition
- encoder
- sharpness
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 230000006870 function Effects 0.000 claims description 26
- 238000012549 training Methods 0.000 claims description 22
- 238000004590 computer program Methods 0.000 claims description 8
- 238000007499 fusion processing Methods 0.000 claims description 8
- 238000013528 artificial neural network Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 abstract description 8
- 238000010586 diagram Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 230000009471 action Effects 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 3
- 230000008447 perception Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30168—Image quality inspection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the disclosure relates to an image definition identification method, an image definition identification device, image definition identification equipment and an image definition identification medium, wherein the method comprises the following steps: acquiring an image to be identified; respectively inputting an image to be identified into a first encoder and a second encoder, and determining a first image characteristic and a second image characteristic; determining a target characteristic based on the first image characteristic and the second image characteristic, inputting the target characteristic into a definition recognition model, and determining the definition level of the image to be recognized; wherein the number of sharpness levels is at least two. By adopting the technical scheme, the image definition can be identified through the discrete classification definition identification model, the image definition can be directly identified without comparing images, and the continuous numerical calculation problem is converted into the discrete classification problem, so that the result robustness is higher, and the practical application requirements are met better.
Description
Technical Field
The present disclosure relates to the field of image processing technologies, and in particular, to a method, an apparatus, a device, and a medium for identifying image sharpness.
Background
With the continuous development of image processing technology, image definition recognition gradually becomes an essential link in many scenes, for example, in the application of photographing questions, due to the fact that when a user photographs and uploads images, focusing problems are very easy to occur due to the shaking of a mobile phone and other reasons, the uploaded images are fuzzy, and the definition recognition of the images needs to be carried out.
At present, methods based on errors, perception and the like can be adopted for identifying the image definition, but the identification methods all need a reference image, the application range of the reference image is limited to a certain extent, the reference image cannot be processed when the image is too fuzzy, and the efficiency of identifying the image definition is reduced.
Disclosure of Invention
To solve the above technical problems or to at least partially solve the above technical problems, the present disclosure provides an image sharpness recognition method, apparatus, device, and medium.
The embodiment of the disclosure provides an image definition identification method, which comprises the following steps:
acquiring an image to be identified;
respectively inputting the image to be identified into a first encoder and a second encoder, and determining a first image characteristic and a second image characteristic;
determining a target feature based on the first image feature and the second image feature, inputting the target feature into a definition recognition model, and determining the definition level of the image to be recognized; wherein the number of sharpness levels is at least two.
The disclosed embodiment also provides an image definition recognition device, which includes:
the image acquisition module is used for acquiring an image to be identified;
the characteristic module is used for respectively inputting the image to be identified into a first encoder and a second encoder and determining a first image characteristic and a second image characteristic;
the definition recognition module is used for determining a target feature based on the first image feature and the second image feature, inputting the target feature into a definition recognition model and determining the definition level of the image to be recognized; wherein the number of sharpness levels is at least two.
An embodiment of the present disclosure further provides an electronic device, which includes: a processor; a memory for storing the processor-executable instructions; the processor is used for reading the executable instructions from the memory and executing the instructions to realize the image definition identification method provided by the embodiment of the disclosure.
The embodiment of the disclosure also provides a computer-readable storage medium, which stores a computer program for executing the image definition recognition method provided by the embodiment of the disclosure.
Compared with the prior art, the technical scheme provided by the embodiment of the disclosure has the following advantages: according to the image definition identification scheme provided by the embodiment of the disclosure, an image to be identified is obtained; respectively inputting an image to be identified into a first encoder and a second encoder, and determining a first image characteristic and a second image characteristic; determining a target feature based on the first image feature and the second image feature, inputting the target feature into a definition recognition model, and determining the definition level of the image to be recognized; wherein the number of sharpness levels is at least two. By adopting the technical scheme, the image definition can be identified through the discrete classification definition identification model, the image definition can be directly identified without comparing images, and the continuous numerical calculation problem is converted into the discrete classification problem, so that the result robustness is higher, and the practical application requirements are met better.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present disclosure, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a schematic flowchart of an image sharpness identification method according to an embodiment of the present disclosure;
fig. 2 is a schematic diagram of a variational self-coder model provided in an embodiment of the present disclosure;
fig. 3 is a flow chart illustrating another image sharpness recognition method provided in the embodiment of the present disclosure;
fig. 4 is a schematic diagram of image sharpness identification provided in an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of an image sharpness identifying apparatus according to an embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
Detailed Description
In order that the above objects, features and advantages of the present disclosure may be more clearly understood, aspects of the present disclosure will be further described below. It should be noted that the embodiments and features of the embodiments of the present disclosure may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure may be practiced in other ways than those described herein; it is to be understood that the embodiments disclosed in the specification are only a few embodiments of the present disclosure, and not all embodiments.
Currently, two methods, namely error-based and perception-based, can be adopted for identifying the image definition. The error-based method is typically a Peak signal-to-noise Ratio (PSNR) with a mathematical expression such as
And
in the above formula y represents the real tag information,
representing the model output, both are m-dimensional vectors, n represents the number of samples, MaxValue represents the maximum value among the pixel values, and bits represents how many bits each pixel value is represented by. PSNR values above 40 indicate excellent image quality, between 30 and 40 indicate good image quality, between 20 and 30 indicate poor image quality, and below 20 indicate poor image quality. The perception-based method considers information such as spatial structure, semantic features and the like of an image, considers image quality at a higher level, and has Structural Similarity (SSIM) in a representation method, wherein the mathematical expression is shown in the formula
And SSIM (x, y) ═ l (x, y)]α·[c(x，y)]β·[s(x，y)]γIn the above formula, x and y represent two images, μ _ x and μ _ y represent the average of x and y, σ _ x, σ _ y, and σ _ xy represent the variance of x and y and the covariance of the two, c _1, c _2, and c _3 are constants, and are generally 1, and α, β, and γ are generally 1.
Although the evaluation effect of the image definition evaluation method is good, the method has a great problem that a reference image is required to be provided, and then two images are calculated to obtain a final evaluation value, which limits the application range to a certain extent. In applications such as photo-taking and question-judging, focusing is very easy to occur due to shaking of a mobile phone and the like when a user takes photos and uploads the photos, so that uploaded images (which are dense text images) are relatively fuzzy. Because the conventional text detection and recognition model cannot process the blurred image, the user is required to upload the blurred image again, and more importantly, the conventional text detection and recognition model cannot judge whether the image can be processed or not, all the processes are performed once, and the result cannot be returned after the user waits, so that the user experience is greatly influenced, and even more, the program crash may be caused. In order to solve the above problem, embodiments of the present disclosure provide an image sharpness recognition method, which is described below with reference to specific embodiments.
Fig. 1 is a schematic flowchart of an image sharpness identifying method provided in an embodiment of the present disclosure, which may be executed by an image sharpness identifying apparatus, where the apparatus may be implemented by software and/or hardware, and may be generally integrated in an electronic device. As shown in fig. 1, the method includes:
The image to be recognized refers to any image needing definition recognition. The specific source of the image to be recognized is not limited in the embodiment of the disclosure.
In the embodiment of the disclosure, the image definition recognition device may acquire, in real time, an image to be recognized acquired by the image acquisition device, and may also acquire an image to be recognized in the internet, where the image acquisition device is a device having an image acquisition function, such as a mobile phone, a tablet computer, and the like.
And 102, respectively inputting the image to be identified into a first encoder and a second encoder, and determining a first image characteristic and a second image characteristic.
The first Encoder and the second Encoder are both encoders in a Variational Auto-Encoder (VAE) model. The VAE model is an important generative model, and is composed of two parts, namely an encoder and a decoder, and usually the infimum bound of log likelihood is taken as an optimization target, so that the loss function of the VAE model is generally composed of two parts, namely reconstruction loss and cross entropy loss. The training of the VAE model is more stable and faster than other generative models.
For example, fig. 2 is a schematic diagram of a variational self-encoder model provided in an embodiment of the present disclosure, as shown in fig. 2, a VAE model may encode an input through an encoder, and then input the encoded input into a decoder for restoring the input, where in most cases, the restored image is very similar to the original image. The code that the VAE model converts the input into may be a parameter of some distribution, or may be a feature map, etc.
In the embodiment of the present disclosure, before acquiring the image to be recognized, the method may further include: and obtaining a fuzzy variational self-encoder model corresponding to the first encoder and a clear variational self-encoder model corresponding to the second encoder through training based on the sample image marked with the definition level. Optionally, the sample images include images representing different levels of sharpness from blur to sharpness, the input and the output of the blur variation self-encoder model are sample images representing blur, the input of the sharpness variation self-encoder model is the sample image representing blur, and the output of the sharpness variation self-encoder model is the sample image representing sharpness.
The sample image may be an image obtained by artificially labeling the image with a level of sharpness, the sample image may cover a plurality of different images from blurred to sharp, and the number of the sample images is not limited. The number of sharpness levels may include a plurality, for example, the sharpness levels may be divided into 5 levels, from 1 to 5, from blur to sharpness in sequence. In the embodiment of the disclosure, a large number of blurred images can be collected, the blurred degree is different, some blurred images can be seen completely, some blurred images can be seen clearly, the corresponding clear images can be collected simultaneously, and then the blurred images are artificially graded, that is, the definition grade is labeled.
Specifically, the image definition recognition device can construct two VAE models, the input of the first VAE model is a sample image representing fuzzy, the output of the first VAE model is also a sample image representing fuzzy, the input of the second VAE model is a sample image representing fuzzy, the output of the second VAE model is a sample image representing clear, and through training, a fuzzy variational self-encoder model and a clear variational self-encoder model with loss functions meeting requirements can be obtained, so that a first encoder in the fuzzy variational self-encoder model and a second encoder in the clear variational self-encoder model can be obtained.
Optionally, the loss functions of the fuzzy variational self-encoder model and the clear variational self-encoder model are receptive field loss functionsAnd the L1 loss function. The receptive field can be the size of the area of the original image mapped by the pixel points on the characteristic diagram output by each layer of the convolutional neural network. The loss of receptive field function is
Where ρ is Ex[||x+e||1]It is a derivable form of the L1 loss function, N is the number of training samples per batch, L represents the number of levels of the image pyramid, ysRepresenting real images on image pyramids, different y on image pyramidssThe original high-resolution image is obtained by down-sampling in a bicubic interpolation mode,
representing images, x, obtained via a networksRepresents input, rsRepresenting the residual error. The L1 penalty function, also called L1 norm penalty function, is also called Least Absolute Deviation (LAD) or Least Absolute Deviation (LAE), which is the target value YiAnd an estimated value f (x)i) The sum of absolute differences S of (a) is minimized,
in the embodiment of the disclosure, after the image sharpness identifying apparatus obtains the image to be identified, the image to be identified may be respectively input into the first encoder and the second encoder, so as to determine the first image feature and the second image feature. The first image feature may be used to characterize a relevant feature of the image to be recognized, the second image feature may be used to characterize a relevant feature after improving the definition of the image to be recognized, and the first image feature and the second image feature may be encoding results converted from an input in the VAE model, may be a certain distributed parameter, may also be a feature diagram, and the like, and are not limited specifically.
103, determining target characteristics based on the first image characteristics and the second image characteristics, inputting the target characteristics into a definition recognition model, and determining the definition level of an image to be recognized; wherein the number of sharpness levels is at least two.
Wherein, the definition level is a discretized definition result obtained based on human observation results, and the number of the definition levels can include a plurality of levels, for example, the definition level can be divided into 5 levels, and the definition levels are characterized from fuzzy to clear from 1 to 5. The definition recognition model is a depth learning model which is trained in advance and used for carrying out definition recognition on the image.
For a human, it is intuitive to see whether an image is clear without a reference (contrast image), and when observing an image, the observation result of the human is very simple, such as blurred, less blurred, clearly visible, and very clear, and is roughly discretized values rather than a continuous precise value calculated, and the discretization results obtained by the human can provide sufficient information related to the sharpness for processing the image, that is, in terms of processing the sharpness, a specific value of the sharpness is not actually required, but an approximate value is required. Based on this, the problem of image sharpness recognition in the embodiments of the present disclosure is converted into a classification problem, that is, several levels of discretization are described which describe the sharpness of an image and are no longer specific values.
In the embodiment of the present disclosure, before executing step 103, the method may further include: and obtaining a definition recognition model through training based on the sample image marked with the definition level. Optionally, the obtaining of the sharpness recognition model through training based on the sample image labeled with the sharpness level includes: taking the sample characteristics determined based on the sample image as input, taking the definition level of the sample image as output, and training a basic neural network to obtain a definition recognition model; wherein the sample features are determined by two image features obtained by inputting the sample image into the first encoder and the second encoder, respectively. Optionally, the number of nodes of the full-link layer in the definition recognition model is the same as the number of definition levels.
The neural network adopted by the definition recognition model can be various, for example, the neural network only consisting of the last two blocks (blocks) of the Resnet18 network can be adopted, the calculation speed is high, and the fitting effect on data is good. And, the number of nodes of the last full connection layer in the definition recognition model may correspond to the number of definition levels, for example, when the definition levels include 5, the number of nodes of the full connection layer in the definition recognition model may be set to 5. Wherein, the sample characteristic determination process may be: and respectively inputting the sample image into the first encoder and the second encoder to obtain two image characteristics, and performing characteristic fusion processing (Concat) on the two image characteristics to obtain the sample characteristics. The feature fusion processing mode may be a serial superposition, for example, if the two image features are both 5 × 5 features, the sample feature after fusion processing is 2 × 5 features, and one dimension is added
Specifically, in the training stage, the sample characteristics determined based on the sample image can be used as input, the definition level of the sample image is output, the loss function is a multi-classification cross entropy loss function, and the basic neural network is trained to obtain a definition recognition model which is trained successfully. The above-described loss functions are merely examples.
Optionally, determining the target feature based on the first image feature and the second image feature may include: and performing feature fusion processing on the first image feature and the second image feature to determine a target feature. For example, assuming that the first image feature and the second image feature are both 10 × 10 features, the target feature after the fusion process is 2 × 10 features, which is increased by one dimension.
In the embodiment of the disclosure, after the image definition recognition model determines the first image feature and the second image feature, the target feature may be determined, and the target feature is input into the previously trained definition recognition model to obtain the definition level of the image to be recognized.
According to the image definition identification scheme provided by the embodiment of the disclosure, an image to be identified is obtained; respectively inputting an image to be identified into a first encoder and a second encoder, and determining a first image characteristic and a second image characteristic; determining a target characteristic based on the first image characteristic and the second image characteristic, inputting the target characteristic into a definition recognition model, and determining the definition level of the image to be recognized; wherein the number of sharpness levels is at least two. By adopting the technical scheme, the image definition can be identified through the discrete classification definition identification model, the image definition can be directly identified without comparing images, and the continuous numerical calculation problem is converted into the discrete classification problem, so that the result robustness is higher, and the practical application requirements are met better.
Fig. 3 is a flow chart illustrating another image sharpness identifying method according to an embodiment of the present disclosure, and the embodiment further specifically describes the image sharpness identifying method on the basis of the foregoing embodiment. As shown in fig. 3, the method includes:
Optionally, the sample images include images representing different levels of sharpness from blur to sharpness, the input and the output of the blur variation self-encoder model are sample images representing blur, and the input of the sharpness variation self-encoder model is the sample image representing blur and the output is the sample image representing sharpness. Optionally, the loss functions of the fuzzy variational self-encoder model and the clear variational self-encoder model are a receptive field loss function and an L1 loss function.
Optionally, the obtaining of the sharpness recognition model through training based on the sample image labeled with the sharpness level includes: taking the sample characteristics determined based on the sample image as input, taking the definition level of the sample image as output, and training a basic neural network to obtain a definition recognition model; wherein the sample features are determined by two image features obtained by inputting the sample image into the first encoder and the second encoder respectively. Optionally, the number of nodes of the full connection layer in the definition recognition model is the same as the number of definition levels.
And 204, performing feature fusion processing on the first image feature and the second image feature to determine a target feature.
And step 205, inputting the target characteristics into the definition recognition model, and determining the definition level of the image to be recognized.
Wherein the number of sharpness levels is at least two.
Next, the image sharpness identifying method in the embodiment of the present disclosure is further explained by a specific example. Exemplarily, fig. 4 is a schematic diagram of image sharpness recognition provided in an embodiment of the present disclosure, and as shown in fig. 4, a process of image sharpness recognition may include: firstly, manually collecting a large number of blurred images, wherein the blurred images are different in blurring degree, some blurred images are completely seen, some blurred images can be seen clearly, and the others can be seen clearly, and meanwhile, clear images corresponding to the blurred images are collected; secondly, according to the first step, grading the blurred image (according to different degrees of definition, grades 1-5), and converting the image definition problem into a classification problem to be considered, namely describing the image definition, wherein the image definition is not a specific numerical value any more, but several grades, such as 5 grades, and the grades are sequentially from blurring to sharpness from 1 to 5; thirdly, two VAE models are constructed, wherein the input of the first VAE model is a fuzzy image, the output of the first VAE model is also a fuzzy image, the input of the second VAE model is a fuzzy image, and the output of the second VAE model is a clear image; fourthly, according to the third step, the loss functions of the two VAE models in the training stage are the loss of the receptive field and the loss function of L1 used by the LAPSRN; fifthly, constructing a definition discrimination network (or other network models) only composed of the last two block blocks of the resnet18 network, and setting the number of nodes of the last full-connection layer to be 5 (corresponding to the previous level information); sixthly, according to the fifth step, in the training stage, the input of the network is the output of the two VAE model encoder parts which are superposed in series, the output is a class response value, and a multi-class cross entropy loss function can be used for training; and seventhly, in a forward reasoning stage, according to the third step, only using the encoder parts of the two trained VAE models, respectively inputting the input images into the two encoders to obtain two codes, then connecting the two codes in series, according to the fifth step, inputting the serial codes into a definition discrimination model, and then obtaining definition categories.
Compared with the existing scheme, the image definition recognition method provided by the embodiment of the disclosure can directly judge the image definition without comparing images for calculation, better conforms to the observation rule of human eyes on the definition problem, converts the continuous numerical calculation problem into the discrete classification problem, has higher result robustness, and better meets the actual application requirements.
The image definition recognition scheme provided by the embodiment of the disclosure obtains a fuzzy variational self-encoder model corresponding to a first encoder, a definition variational self-encoder model corresponding to a second encoder and a definition recognition model through training based on a sample image marked with a definition level; acquiring an image to be identified; respectively inputting an image to be identified into a first encoder and a second encoder, and determining a first image characteristic and a second image characteristic; determining a target feature based on the first image feature and the second image feature, inputting the target feature into a definition recognition model, and determining the definition level of an image to be recognized; wherein the number of sharpness levels is at least two. By adopting the technical scheme, the image definition can be identified through the discrete classification definition identification model, the image definition can be directly identified without comparing the image, and the continuous numerical calculation problem is converted into the discrete classification problem, so that the result robustness is higher, and the practical application requirements are better met.
Fig. 5 is a schematic structural diagram of an image sharpness identifying apparatus provided in an embodiment of the present disclosure, where the apparatus may be implemented by software and/or hardware, and may be generally integrated in an electronic device. As shown in fig. 5, the apparatus includes:
an image acquisition module 301, configured to acquire an image to be identified;
a feature module 302, configured to input the image to be identified into a first encoder and a second encoder respectively, and determine a first image feature and a second image feature;
a definition recognition module 303, configured to determine a target feature based on the first image feature and the second image feature, input the target feature into a definition recognition model, and determine a definition level of the image to be recognized; wherein the number of sharpness levels is at least two.
According to the image definition identification scheme provided by the embodiment of the disclosure, an image to be identified is obtained; respectively inputting an image to be identified into a first encoder and a second encoder, and determining a first image characteristic and a second image characteristic; determining a target characteristic based on the first image characteristic and the second image characteristic, inputting the target characteristic into a definition recognition model, and determining the definition level of the image to be recognized; wherein the number of sharpness levels is at least two. By adopting the technical scheme, the image definition can be identified through the discrete classification definition identification model, the image definition can be directly identified without comparing images, and the continuous numerical calculation problem is converted into the discrete classification problem, so that the result robustness is higher, and the practical application requirements are met better.
Optionally, the apparatus further includes a model training module, configured to: prior to the acquisition of the image to be identified,
and obtaining a fuzzy variational self-encoder model corresponding to the first encoder, a clearness variational self-encoder model corresponding to the second encoder and the clearness recognition model through training based on the sample image marked with the clearness level.
Optionally, the sample images include images representing different levels of sharpness from blur to sharpness, the input and the output of the blur variation self-encoder model are sample images representing blur, the input of the sharpness variation self-encoder model is a sample image representing blur, and the output of the sharpness variation self-encoder model is a sample image representing sharpness.
Optionally, the loss functions of the fuzzy variational self-encoder model and the clear variational self-encoder model are a receptive field loss function and an L1 loss function.
Optionally, the model training module is specifically configured to:
taking the sample characteristics determined based on the sample image as input, taking the definition level of the sample image as output, and training a basic neural network to obtain the definition identification model; wherein the sample feature is determined by two image features obtained by inputting the sample image into the first encoder and the second encoder, respectively.
Optionally, the number of nodes of the full-link layer in the sharpness recognition model is the same as the number of sharpness levels.
Optionally, the definition identifying module 303 is specifically configured to:
and performing feature fusion processing on the first image feature and the second image feature to determine the target feature.
The image definition recognition device provided by the embodiment of the disclosure can execute the image definition recognition method provided by any embodiment of the disclosure, and has corresponding functional modules and beneficial effects of the execution method.
Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. As shown in fig. 6, the electronic device 400 includes one or more processors 401 and memory 402.
The processor 401 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.
In one example, the electronic device 400 may further include: an input device 403 and an output device 404, which are interconnected by a bus system and/or other form of connection mechanism (not shown).
The input device 403 may also include, for example, a keyboard, a mouse, and the like.
The output device 404 may output various information to the outside, including the determined distance information, direction information, and the like. The output devices 404 may include, for example, a display, speakers, a printer, and a communication network and its connected remote output devices, among others.
Of course, for simplicity, only some of the components of the electronic device 400 relevant to the present disclosure are shown in fig. 6, omitting components such as buses, input/output interfaces, and the like. In addition, electronic device 400 may include any other suitable components depending on the particular application.
In addition to the above methods and apparatuses, embodiments of the present disclosure may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the image sharpness recognition method provided by embodiments of the present disclosure.
The computer program product may write program code for carrying out operations for embodiments of the present disclosure in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.
Furthermore, embodiments of the present disclosure may also be a computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, cause the processor to perform the image sharpness recognition method provided by the embodiments of the present disclosure.
The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
It is noted that, herein, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other elements in the process, method, article, or apparatus that comprise the element.
The foregoing are merely exemplary embodiments of the present disclosure, which enable those skilled in the art to understand or practice the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. An image sharpness recognition method, comprising:
acquiring an image to be identified;
respectively inputting the image to be identified into a first encoder and a second encoder, and determining a first image characteristic and a second image characteristic;
determining a target feature based on the first image feature and the second image feature, inputting the target feature into a definition recognition model, and determining the definition level of the image to be recognized; wherein the number of sharpness levels is at least two.
2. The method of claim 1, wherein before the obtaining the image to be identified, further comprising:
and obtaining a fuzzy variational self-encoder model corresponding to the first encoder, a clearness variational self-encoder model corresponding to the second encoder and the clearness recognition model through training based on the sample image marked with the clearness level.
3. The method of claim 2, wherein the sample images comprise images representing different levels of sharpness from blur to sharpness, wherein the input and output of the blur variational self-coder model are sample images representing blur, and wherein the input of the sharpness variational self-coder model is a sample image representing blur and the output is a sample image representing sharpness.
4. The method of claim 2, wherein the loss functions of the fuzzy variational self-encoder model and the sharp variational self-encoder model are a receptive field loss function and an L1 loss function.
5. The method of claim 2, wherein the sharpness recognition model is derived by training based on the sample images labeled with sharpness levels, and comprises:
taking the sample characteristics determined based on the sample image as input, taking the definition level of the sample image as output, and training a basic neural network to obtain the definition recognition model; wherein the sample feature is determined by two image features obtained by inputting the sample image into the first encoder and the second encoder, respectively.
6. The method of claim 5, wherein the number of nodes of a fully-connected layer in the sharpness recognition model is the same as the number of sharpness levels.
7. The method of claim 1, wherein determining a target feature based on the first image feature and the second image feature comprises:
and performing feature fusion processing on the first image feature and the second image feature to determine the target feature.
8. An image sharpness recognition apparatus, comprising:
the image acquisition module is used for acquiring an image to be identified;
the characteristic module is used for respectively inputting the image to be identified into a first encoder and a second encoder and determining a first image characteristic and a second image characteristic;
the definition recognition module is used for determining a target feature based on the first image feature and the second image feature, inputting the target feature into a definition recognition model and determining the definition level of the image to be recognized; wherein the number of sharpness levels is at least two.
9. An electronic device, characterized in that the electronic device comprises:
a processor;
a memory for storing the processor-executable instructions;
the processor is used for reading the executable instructions from the memory and executing the instructions to realize the image definition identification method of any one of the claims 1 to 7.
10. A computer-readable storage medium, characterized in that the storage medium stores a computer program for executing the image sharpness recognition method according to any one of the preceding claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110271927.9A CN113436137A (en) | 2021-03-12 | 2021-03-12 | Image definition recognition method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110271927.9A CN113436137A (en) | 2021-03-12 | 2021-03-12 | Image definition recognition method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113436137A true CN113436137A (en) | 2021-09-24 |
Family
ID=77752844
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110271927.9A Pending CN113436137A (en) | 2021-03-12 | 2021-03-12 | Image definition recognition method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113436137A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113837965A (en) * | 2021-09-26 | 2021-12-24 | 北京百度网讯科技有限公司 | Image definition recognition method and device, electronic equipment and storage medium |
| CN114022442A (en) * | 2021-11-03 | 2022-02-08 | 武汉智目智能技术合伙企业(有限合伙) | Unsupervised learning-based fabric defect detection algorithm |
| CN114187593A (en) * | 2021-12-14 | 2022-03-15 | 北京有竹居网络技术有限公司 | Image processing method and device |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107833193A (en) * | 2017-11-20 | 2018-03-23 | 长沙全度影像科技有限公司 | A kind of simple lens global image restored method based on refinement network deep learning models |
| CN108898579A (en) * | 2018-05-30 | 2018-11-27 | 腾讯科技（深圳）有限公司 | A kind of image definition recognition methods, device and storage medium |
| CN108921806A (en) * | 2018-08-07 | 2018-11-30 | Oppo广东移动通信有限公司 | A kind of image processing method, image processing apparatus and terminal device |
| CN109360171A (en) * | 2018-10-26 | 2019-02-19 | 北京理工大学 | A kind of real-time deblurring method of video image neural network based |
| CN109598695A (en) * | 2017-09-29 | 2019-04-09 | 南京大学 | A kind of non-reference picture fog-level estimation method based on deep learning network |
| CN109903236A (en) * | 2019-01-21 | 2019-06-18 | 南京邮电大学 | Facial image restorative procedure and device based on VAE-GAN to similar block search |
| CN109949234A (en) * | 2019-02-25 | 2019-06-28 | 华中科技大学 | Video restoration model training method and video restoration method based on depth network |
| CN110223245A (en) * | 2019-05-16 | 2019-09-10 | 华南理工大学 | Blurred picture clearness processing method and system based on deep neural network |
| CN111815536A (en) * | 2020-07-15 | 2020-10-23 | 电子科技大学 | Motion blur restoration method based on contour enhancement strategy |
| CN112102185A (en) * | 2020-09-04 | 2020-12-18 | 腾讯科技（深圳）有限公司 | Image deblurring method and device based on deep learning and electronic equipment |
| WO2020252917A1 (en) * | 2019-06-20 | 2020-12-24 | 平安科技（深圳）有限公司 | Fuzzy face image recognition method and apparatus, terminal device, and medium |
| CN112135140A (en) * | 2020-09-17 | 2020-12-25 | 上海连尚网络科技有限公司 | Video definition recognition method, electronic device and storage medium |
| CN112435185A (en) * | 2020-11-19 | 2021-03-02 | 北京工业大学 | Fuzzy pollen picture processing method and device and electronic equipment |
-
2021
- 2021-03-12 CN CN202110271927.9A patent/CN113436137A/en active Pending
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109598695A (en) * | 2017-09-29 | 2019-04-09 | 南京大学 | A kind of non-reference picture fog-level estimation method based on deep learning network |
| CN107833193A (en) * | 2017-11-20 | 2018-03-23 | 长沙全度影像科技有限公司 | A kind of simple lens global image restored method based on refinement network deep learning models |
| CN108898579A (en) * | 2018-05-30 | 2018-11-27 | 腾讯科技（深圳）有限公司 | A kind of image definition recognition methods, device and storage medium |
| CN108921806A (en) * | 2018-08-07 | 2018-11-30 | Oppo广东移动通信有限公司 | A kind of image processing method, image processing apparatus and terminal device |
| CN109360171A (en) * | 2018-10-26 | 2019-02-19 | 北京理工大学 | A kind of real-time deblurring method of video image neural network based |
| CN109903236A (en) * | 2019-01-21 | 2019-06-18 | 南京邮电大学 | Facial image restorative procedure and device based on VAE-GAN to similar block search |
| CN109949234A (en) * | 2019-02-25 | 2019-06-28 | 华中科技大学 | Video restoration model training method and video restoration method based on depth network |
| CN110223245A (en) * | 2019-05-16 | 2019-09-10 | 华南理工大学 | Blurred picture clearness processing method and system based on deep neural network |
| WO2020252917A1 (en) * | 2019-06-20 | 2020-12-24 | 平安科技（深圳）有限公司 | Fuzzy face image recognition method and apparatus, terminal device, and medium |
| CN111815536A (en) * | 2020-07-15 | 2020-10-23 | 电子科技大学 | Motion blur restoration method based on contour enhancement strategy |
| CN112102185A (en) * | 2020-09-04 | 2020-12-18 | 腾讯科技（深圳）有限公司 | Image deblurring method and device based on deep learning and electronic equipment |
| CN112135140A (en) * | 2020-09-17 | 2020-12-25 | 上海连尚网络科技有限公司 | Video definition recognition method, electronic device and storage medium |
| CN112435185A (en) * | 2020-11-19 | 2021-03-02 | 北京工业大学 | Fuzzy pollen picture processing method and device and electronic equipment |
Non-Patent Citations (2)
| Title |
|---|
| 余孝源;谢巍;: "基于深度编码?解码器的图像模糊核估计", 控制理论与应用, no. 04 * |
| 武玉伟: "《深度学习基础与应用》", 北京理工大学出版社, pages: 13 - 17 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113837965A (en) * | 2021-09-26 | 2021-12-24 | 北京百度网讯科技有限公司 | Image definition recognition method and device, electronic equipment and storage medium |
| CN114022442A (en) * | 2021-11-03 | 2022-02-08 | 武汉智目智能技术合伙企业(有限合伙) | Unsupervised learning-based fabric defect detection algorithm |
| CN114187593A (en) * | 2021-12-14 | 2022-03-15 | 北京有竹居网络技术有限公司 | Image processing method and device |
| CN114187593B (en) * | 2021-12-14 | 2024-01-30 | 北京有竹居网络技术有限公司 | Image processing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021093468A1 (en) | Video classification method and apparatus, model training method and apparatus, device and storage medium | |
| CN113436137A (en) | Image definition recognition method, device, equipment and medium | |
| CN110210513B (en) | Data classification method and device and terminal equipment | |
| CN110033023B (en) | Image data processing method and system based on picture book recognition | |
| US9330336B2 (en) | Systems, methods, and media for on-line boosting of a classifier | |
| CN111291819A (en) | Image recognition method and device, electronic equipment and storage medium | |
| CN114067119B (en) | Training method of panorama segmentation model, panorama segmentation method and device | |
| CN111428805A (en) | Method and device for detecting salient object, storage medium and electronic equipment | |
| CN115222566A (en) | Learning method and system for international finance and finance metrology teaching | |
| CN111507288A (en) | Image detection method, image detection device, computer equipment and storage medium | |
| DE102022003003A1 (en) | Automatic photo editing using spoken instructions | |
| CN111522979B (en) | Picture sorting recommendation method and device, electronic equipment and storage medium | |
| CN114529890A (en) | State detection method and device, electronic equipment and storage medium | |
| CN113240699B (en) | Image processing method and device, model training method and device, and electronic equipment | |
| CN113313179A (en) | Noise image classification method based on l2p norm robust least square method | |
| CN113393385A (en) | Unsupervised rain removal method, system, device and medium based on multi-scale fusion | |
| CN112465805A (en) | Neural network training method for quality detection of steel bar stamping and bending | |
| Li et al. | A mixed noise removal algorithm based on multi-fidelity modeling with nonsmooth and nonconvex regularization | |
| CN116994049A (en) | Full-automatic flat knitting machine and method thereof | |
| CN115661803A (en) | Image definition detection method, electronic device, and computer-readable storage medium | |
| CN115439179A (en) | Method for training fitting model, virtual fitting method and related device | |
| CN114359587A (en) | Class activation graph generation method, interpretable method, device, equipment and storage medium | |
| CN111798376A (en) | Image recognition method and device, electronic equipment and storage medium | |
| CN112699898A (en) | Image direction identification method based on multi-layer feature fusion | |
| CN116168398B (en) | Examination paper approval method, device and equipment based on image identification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |