CN115222635A - No-reference image quality evaluation method and device based on image feature fusion and computer readable storage medium - Google Patents
No-reference image quality evaluation method and device based on image feature fusion and computer readable storage medium Download PDFInfo
- Publication number
- CN115222635A CN115222635A CN202210839006.2A CN202210839006A CN115222635A CN 115222635 A CN115222635 A CN 115222635A CN 202210839006 A CN202210839006 A CN 202210839006A CN 115222635 A CN115222635 A CN 115222635A
- Authority
- CN
- China
- Prior art keywords
- image
- domain
- fusion
- natural distortion
- quality evaluation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000004927 fusion Effects 0.000 title claims abstract description 108
- 238000013441 quality evaluation Methods 0.000 title claims abstract description 74
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000012549 training Methods 0.000 claims description 16
- 230000006870 function Effects 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 13
- 238000010586 diagram Methods 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 8
- 238000012417 linear regression Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000001303 quality assessment method Methods 0.000 claims description 2
- 238000011156 evaluation Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000000137 annealing Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30168—Image quality inspection
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the field of computer vision, and particularly relates to a no-reference image quality evaluation method and a no-reference image quality evaluation device based on image feature fusion and a computer readable storage medium, wherein the no-reference image quality evaluation method comprises the following steps of: acquiring a natural distortion image to be evaluated, generating a gradient image according to the natural distortion image, and inputting the natural distortion image and the gradient image into a trained non-reference image quality evaluation model based on image feature fusion to obtain a quality evaluation score; the no-reference image quality evaluation model based on image feature fusion comprises the following steps: the invention not only evaluates the image, but also fully considers the global semantic information and the local semantic information of the natural distortion image domain, thereby being capable of evaluating the quality of the natural distortion image more accurately.
Description
Technical Field
The invention belongs to the field of computer vision, and particularly relates to a no-reference image quality evaluation method and device based on image feature fusion and a computer readable storage medium.
Background art:
the No-reference image quality evaluation (No-referenceimagequality evaluation) refers to evaluating the quality of itself only by a distortion map, and unlike the full-reference image quality evaluation method, the No-reference image quality evaluation does not require a reference image.
With the intensive research of the backbone network, the extraction method of the image features is very abundant, however, how to apply the extracted features to the image quality evaluation is still a challenging task, which is also one of the development factors hindering the image quality evaluation.
The existing image quality evaluation technology only adopts feature information of an image to evaluate the quality of the image, for example, in patent cn201810759247.X, a generation countermeasure network is mainly used to construct an image training model, then a high-definition lossless image is sent to the image training model as a training data set for training and learning to obtain a no-reference image quality evaluation model with a training complete identification network, finally an image to be evaluated is sent to the training complete identification network, and a final evaluation result is obtained by scoring and weighting.
Disclosure of Invention
In order to solve the problem that the evaluation of image quality is not accurate due to the fact that the prior art does not pay attention to the effect of other images on the quality evaluation of an original image and cannot give consideration to the global semantic information and the local information of the image, the invention provides a no-reference image quality evaluation method, a no-reference image quality evaluation device and a computer readable storage medium based on image feature fusion, which are used for carrying out fusion according to the features of the quality of the original image and the features of a gradient map of the original image, so that the quality evaluation of the original image is carried out, and the accuracy of the image quality evaluation is improved by fully considering the global semantic information and the local information of the original image.
The invention adopts the following technical scheme:
a no-reference image quality evaluation method based on image feature fusion comprises the following steps:
acquiring a natural distortion image to be evaluated, generating a gradient image according to the natural distortion image, and inputting the natural distortion image and the gradient image into a trained non-reference image quality evaluation model based on image feature fusion to obtain a quality evaluation score; the no-reference image quality evaluation model based on image feature fusion comprises: the system comprises a backbone network, a cross-domain feature fusion model, a cross-scale feature fusion model and two linear regression layers;
the process of training the no-reference image quality evaluation model based on image feature fusion comprises the following steps:
s1: acquiring a natural distortion image domain with a real label, wherein the real label represents a real score of a natural distortion image in the natural distortion image domain;
s2: generating a gradient image domain according to the natural distortion image domain;
s3: inputting the natural distortion image domain and the gradient image domain corresponding to the natural distortion image domain into a backbone network, and extracting the hierarchical features of the natural distortion image domain and the hierarchical features of the gradient image domain;
s4: inputting the hierarchical features of the natural distortion image domain and the hierarchical features of the gradient image domain into a cross-domain feature fusion model correspondingly for fusion, and calculating to obtain the cross-domain fusion hierarchical features of the natural distortion image domain;
s5: inputting the cross-domain fusion level features into a cross-scale feature fusion model for fusion, and calculating to obtain the cross-scale fusion features of the natural distortion image domain;
s6: inputting the cross-scale fusion characteristics into two linear regression layers for regression processing, and calculating to obtain a quality evaluation score of a natural distortion image domain;
s7: calculating a loss function of a no-reference image quality evaluation model based on image feature fusion according to the real label and the quality evaluation score of the natural distortion image domain;
s8: and continuously adjusting the parameters of the model, and finishing the training of the model when the loss function is smaller than a set threshold value.
To achieve the above object, the present invention further provides a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the above-mentioned method for evaluating quality of a non-reference image based on image feature fusion.
In order to achieve the above object, the present invention further provides a no-reference image quality evaluation device based on image feature fusion, which includes a processor and a memory; the memory is used for storing a computer program; the processor is connected with the memory and is used for executing the computer program stored in the memory so as to enable the non-reference image quality evaluation device based on image feature fusion to execute the non-reference image quality evaluation method based on image feature fusion.
The invention has at least the following beneficial effects:
the invention extracts the characteristics of a natural distortion image domain and a gradient image domain through a backbone network, performs cross-domain fusion on the characteristics between different levels of the gradient image domain and the natural distortion image domain through a cross-domain characteristic fusion model to obtain cross-domain fusion level characteristics with local semantic information, can perform cross-scale characteristic fusion on the cross-domain fusion level characteristics through a cross-scale characteristic fusion module so as to obtain natural distortion image domain characteristics with both global semantic information and local semantic information, performs regression processing on the natural distortion image domain characteristics with the global semantic information and the local semantic information, calculates the quality evaluation score of the natural distortion image domain, enables the obtained quality evaluation score to embody the quality information of the natural distortion image domain, can obtain accurate image quality information without a reference image through the invention, fully considers the gradient image domain as an auxiliary image domain of the natural distortion domain relative to the traditional non-reference image quality evaluation method, can obtain richer prior information and local information through fusing different image domains, is beneficial to improving the accuracy of the quality evaluation, obtains the reference image quality score of the image by applying the cross-domain image fusion characteristics of different image domains to a method of obtaining the artificial image with the cross-domain and the reference image quality score of the invention based on the overall quality evaluation and the invention, and the characteristics of the invention can obtain the quality of the image with the characteristics of the invention, and the quality evaluation capability of the natural distortion image is improved.
Drawings
FIG. 1: a block diagram of a system for implementing the invention;
FIG. 2 is a schematic diagram: a model training flow chart of the invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only some embodiments of the present invention, and based on the embodiments of the present invention, other embodiments obtained by a person skilled in the art without making creative efforts belong to the protection scope of the present invention.
Referring to fig. 1, the present invention provides a no-reference image quality evaluation method based on image feature fusion, which specifically includes the following steps:
acquiring a natural distortion image to be evaluated, generating a gradient image according to the natural distortion image, and inputting the natural distortion image and the gradient image into a trained non-reference image quality evaluation model based on image feature fusion to obtain a quality evaluation score; the no-reference image quality evaluation model based on image feature fusion comprises the following steps: the system comprises a backbone network, a cross-domain feature fusion model, a cross-scale feature fusion model and two linear regression layers, wherein a natural distortion image to be evaluated can be an image shot by any camera, or an image downloaded from a network, or an image acquired from other storage devices, or can be a static image or an image in a motion state, and the like, and the natural distortion image to be evaluated can comprise one or more objects, for example, when an image of a person is shot, the environment around the person can be shot, such as an automobile, a tree and the like, and then the person, the automobile, an electron, the tree and the like are all objects contained in the image to be processed.
Referring to fig. 2, the process of training the non-reference image quality evaluation model based on image feature fusion includes:
s1: and acquiring a natural distorted image domain with a real label, wherein the real label represents a real score of the natural distorted image in the natural distorted image domain.
In view of the fact that most of the conventional NR-IQA methods only use RGB images as input of a model, and whether it is advantageous to use multi-feature fusion or whether one image is sufficient, the gradient of an image plays a crucial role in many visual tasks, the gradient of an image sharply reflects structural components of the image, such as image edges, and the gradient image can robustly reflect details of the image structure under the variation of image intensity and color, therefore, a gradient map is used as data input to supplement a naturally distorted image and assist in feature extraction of the naturally distorted image, which assists in texture quality feature extraction of the naturally distorted image and reduces the difficulty in feature extraction from a single naturally distorted image, wherein in the present invention, the naturally distorted image domain mainly uses LIVE-Challenge data set which includes 1169 naturally distorted images without reference images; the image size is 500x 500, the real label is a subjective evaluation score (MOS value) of a natural distortion image in a natural distortion image domain, the MOS size range is [0,100], the higher the score is, the better the image quality is represented by 8100 testers, and the natural distortion image domain is represented by an image set containing a plurality of natural distortion images.
S2: a gradient image domain is generated from the naturally distorted image domain.
A specific embodiment of a no-reference image quality evaluation method based on image feature fusion is disclosed, wherein the specific implementation method for processing a natural distortion image into a gradient image comprises the following steps:
wherein (x, y) is a natural distortion image I Z The coordinates of the pixels of (a) and (b),
representing a naturally distorted image I Z A partial derivative in the X direction;
representing a naturally distorted image I Z A partial derivative in the Y direction;
representing a naturally distorted image I Z Total partial derivatives of (c); g (I) Z ) Representing a naturally distorted image I Z The gradient amplitude of (a).
S3: and inputting the natural distortion image domain and the gradient image domain corresponding to the natural distortion image domain into the backbone network, and extracting the hierarchical features of the natural distortion image domain and the hierarchical features of the gradient image domain.
A specific embodiment of a no-reference image quality evaluation method based on image feature fusion is shown in fig. 2:
inputting the processed natural distortion image domain and the gradient image domain into a Resnet50 backbone network, extracting L hierarchical features Stage1-L of the natural distortion image domain and the gradient image domain respectively, wherein a block containing D in fig. 2 represents L hierarchical features of the natural distortion image domain (actually, L hierarchical features are drawn only one layer), and a block containing G represents L hierarchical features of the gradient image domain (actually, L hierarchical features are drawn only one layer).
S4: and correspondingly inputting the hierarchical features of the natural distortion image domain and the hierarchical features of the gradient image domain into a cross-domain feature fusion model for fusion, and calculating to obtain the cross-domain fusion hierarchical features of the natural distortion image domain.
A specific embodiment of a method for evaluating quality of a reference-free image based on image feature fusion, as shown in fig. 2, the cross-domain feature fusion model (CDFM) includes: a full connection layer, a self-attention mechanism layer and a multi-layer perceptron MLP.
A specific embodiment of a no-reference image quality evaluation method based on image feature fusion, as shown in fig. 2, a computing manner of the cross-domain fusion hierarchical features includes:
s41: generating query Q by using characteristics of nth level of natural distortion image domain through full connection layer 1n Key K 1n Sum value V 1n And process Q according to a self-attention mechanism 1n 、K 1n And V 1n Calculating the first global semantic level characteristic X of the nth level of the natural distortion image domain n ；
S42: generating query Q by using features of nth level of gradient image domain through fully-connected layer 2n Key K 2n Sum value V 2n And process Q according to a self-attention mechanism 2n 、K 2n And V 2n Calculating the second global semantic level characteristic Y of the nth level of the gradient image domain n ；
S43: processing Q according to a self-attention mechanism 1n 、K 2n And V 2n Calculating the natural distortion image domain and the gradient image domainn levels of fusion features Z n ；
S44: mixing X n 、Y n And Z n Inputting into a multi-layer perceptron MLP, and calculating to obtain the cross-domain fusion level feature F of the nth level of the natural distortion image domain n The features of different levels of the natural distortion image domain and the gradient image domain are fused, so that the obtained cross-domain fusion level features have rich prior information and local information, and the accuracy of quality evaluation is improved.
S5: inputting the cross-domain fusion level features into a cross-scale feature fusion model for fusion, and calculating to obtain the cross-scale fusion features of the natural distortion image domain.
A specific embodiment of a no-reference image quality evaluation method based on image feature fusion, as shown in fig. 2, the cross-scale feature fusion model includes: STB model (Swin-Transformer Block) and Global average pooling layer (GAP);
a specific embodiment of a no-reference image quality evaluation method based on image feature fusion is shown in fig. 2, where a calculation manner of the cross-scale fusion features includes:
s51: the cross-domain fusion hierarchical feature F of the nth hierarchy of the natural distortion image domain n Calculating out a characteristic diagram A in an input STB model n ：
S52: the cross-domain fusion hierarchical feature F of the n +1 th level of the natural distortion image domain n+1 And characteristic diagram A n Splicing the Cat functions together and inputting the spliced Cat functions into the STB model to calculate a characteristic diagram A n+1 ；
S53: repeating the step S52 to obtain a characteristic diagram A L ；
S54: will feature map A L Inputting output cross-scale fusion characteristics in the GAP layer;
wherein A is L The method is characterized in that a characteristic diagram of the L-th level of a natural distortion image domain is represented, n =1,2,3 \8230, L-1, L represents the number of level characteristics extracted by a backbone network, and rich global and local image information can be obtained by performing cross-scale fusion on the characteristics of different levels in cross-domain fusion level characteristics, so that the method is favorable for improving the accuracy of quality evaluationIn another embodiment, only 4 hierarchical features extracted from the backbone network are selected, so that L is 4.
A specific embodiment of a no-reference image quality evaluation method based on image feature fusion is characterized in that a specific calculation mode of cross-scale fusion features comprises the following steps:
cross-domain fusion level features output by the cross-domain feature fusion model CDFM are sequentially sent into a Swin-transformer Block (STB), and the STB module: firstly, an input feature block is sent into a Window Multi-head SelfAttention (W-MSA) module through a LayerNorm layer to carry out window-based self attention, and then the feature block is fused with an MLP layer through the LayerNorm layer; then, the obtained product is sent into a ShiftWindowMulti-head SelfAttention (SW-MSA) module through a LayerNorm layer, then passes through a LayerNorm layer and an MLP layer, and finally is subjected to Conv (convolution) operation to reduce the characteristic scale.
S6: and inputting the cross-scale fusion characteristics into two linear regression layers (FC 1 and FC 2) for regression processing, and calculating to obtain the quality evaluation score of the natural distortion image domain.
S7: and calculating a loss function of a no-reference image quality evaluation model based on image feature fusion according to the real label and the quality evaluation score of the natural distortion image domain.
The loss function of the reference-free image quality evaluation model based on image feature fusion comprises the following steps:
wherein N represents the number of natural distortion images in the natural distortion image domain, F i (I D ,I G ) Quality assessment score representing the ith natural distortion image domain, I D Representing a naturally distorted image domain, I G Representing the gradient image field, Q i A true label representing the ith naturally distorted image, and loss represents the loss function.
S8: and continuously adjusting parameters of the model, and finishing the training of the model when the loss function is smaller than a set threshold value, wherein the threshold value is set by a person skilled in the art according to the actual situation.
The design selects ViT-B/8 as a pre-training model, the model is trained on ImageNet-21k and is finely adjusted on ImageNet 1k, the patch size P is set to be 8, the standard training strategy of the existing IQA algorithm is followed, the learning rate of the pre-training model is set to be 1 x 10 < -5 >, the Batch size (B) is set to be 32, and an adaptive motion Estimation (ADAM) optimizer with the weight attenuation of 1 x 10 < -5 > is used for optimizing the model and a cosine annealing learning rate (Cosinesinellinglr) strategy is used for optimizing the learning rate of the model.
In an embodiment of the present invention, the invention further includes a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement any one of the above-mentioned no-reference image quality evaluation methods based on image feature fusion.
Those of ordinary skill in the art will understand that: all or part of the steps for implementing the above method embodiments may be performed by hardware associated with a computer program. The aforementioned computer program may be stored in a computer readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
A no-reference image quality evaluation device based on image feature fusion comprises a processor and a memory; the memory is used for storing a computer program; the processor is connected with the memory and is used for executing the computer program stored in the memory so as to enable the non-reference image quality evaluation device based on image feature fusion to execute any one of the non-reference image quality evaluation based on image feature fusion.
Specifically, the memory includes: various media that can store program codes, such as ROM, RAM, magnetic disk, U-disk, memory card, or optical disk.
Preferably, the Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; the Integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, or discrete hardware components.
The method designed by the invention can improve the accuracy and generalization capability of natural image quality evaluation, and specifically comprises the following steps: the designed model can further utilize the local information and the global semantic information of the picture, so that an image quality evaluation result closer to a human visual system is given, and a quality evaluation task of various types of images under the condition of no reference image is realized, for example, the image restoration of the natural distortion image is assisted according to the quality evaluation score of the natural distortion image; when the robot is applied to artificial intelligence, the robot judges a target according to the quality evaluation score; when the method is used for photographing, parameters such as the resolution of a lens are automatically adjusted according to the quality evaluation score.
Furthermore, the above-described figures are merely schematic illustrations of processes involved in methods according to exemplary embodiments of the invention, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed, for example, synchronously or asynchronously in multiple modules.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice in the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is to be limited only by the terms of the appended claims.
Claims (8)
1. A no-reference image quality evaluation method based on image feature fusion is characterized by comprising the following steps:
acquiring a natural distortion image to be evaluated, generating a gradient image according to the natural distortion image, and inputting the natural distortion image and the gradient image into a trained non-reference image quality evaluation model based on image feature fusion to obtain a quality evaluation score; the no-reference image quality evaluation model based on image feature fusion comprises: the system comprises a backbone network, a cross-domain feature fusion model, a cross-scale feature fusion model and two linear regression layers;
the process of training the no-reference image quality evaluation model based on image feature fusion comprises the following steps:
s1: acquiring a natural distortion image domain with a real label, wherein the real label represents a real score of a natural distortion image in the natural distortion image domain;
s2: generating a gradient image domain according to the natural distortion image domain;
s3: inputting the natural distortion image domain and the gradient image domain corresponding to the natural distortion image domain into a backbone network, and extracting the hierarchical features of the natural distortion image domain and the hierarchical features of the gradient image domain;
s4: inputting the hierarchical features of the natural distortion image domain and the hierarchical features of the gradient image domain into a cross-domain feature fusion model correspondingly for fusion, and calculating to obtain the cross-domain fusion hierarchical features of the natural distortion image domain;
s5: inputting the cross-domain fusion level features into a cross-scale feature fusion model for fusion, and calculating to obtain the cross-scale fusion features of the natural distortion image domain;
s6: inputting the cross-scale fusion characteristics into two linear regression layers for regression processing, and calculating to obtain a quality evaluation score of a natural distortion image domain;
s7: calculating a loss function of a no-reference image quality evaluation model based on image feature fusion according to the real label and the quality evaluation score of the natural distortion image domain;
s8: and continuously adjusting the parameters of the model, and finishing the training of the model when the loss function is smaller than a set threshold value.
2. The method for evaluating the quality of the reference-free image based on the image feature fusion as claimed in claim 1, wherein the cross-domain feature fusion model comprises: a full connection layer, a self-attention mechanism layer and a multi-layer perceptron MLP.
3. The method for evaluating the quality of the reference-free image based on the image feature fusion as claimed in claim 2, wherein the calculation mode of the cross-domain fusion hierarchical feature comprises:
s41: generating query Q through fully-connected layers by using characteristics of nth level of natural distortion image domain 1n Key K 1n Sum value V 1n And processes Q according to a self-attention mechanism 1n 、K 1n And V 1n Calculating the first global semantic level characteristic X of the nth level of the natural distortion image domain n ；
S42: generating query Q by using features of nth level of gradient image domain through fully-connected layer 2n Key K 2n Sum value V 2n And processes Q according to a self-attention mechanism 2n 、K 2n And V 2n Calculating the second global semantic level characteristic Y of the nth level of the gradient image domain n ；
S43: processing Q according to a self-attention mechanism 1n 、K 2n And V 2n Calculating the fusion characteristic Z of the nth level of the natural distortion image domain and the gradient image domain n ；
S44: x is to be n 、Y n And Z n Inputting into a multi-layer perceptron MLP, and calculating to obtain the cross-domain fusion level feature F of the nth level of the natural distortion image domain n 。
4. The method according to claim 1, wherein the cross-scale feature fusion model comprises: STB model and GAP layer.
5. The method according to claim 4, wherein the calculation mode of the cross-scale fusion features comprises:
s51: the cross-domain fusion level feature F of the nth level of the natural distortion image domain n Calculating out a characteristic diagram A in an input STB model n ；
S52: the cross-domain fusion hierarchical feature F of the n +1 th level of the natural distortion image domain n+1 And characteristic diagram A n Splicing the Cat function together and inputting the spliced Cat function into the STB model to calculate a characteristic diagram A n+1 ；
S53: repeating the step S52 to obtain a characteristic diagram A L ；
S54: will feature map A L Inputting the output cross-scale fusion characteristics in the GAP layer;
wherein A is L The method comprises the steps of representing a characteristic diagram of the L-th level of a natural distortion image domain, wherein n =1,2,3 \ 8230 \8230, and L-1, L represents the number of level characteristics extracted by a backbone network.
6. The method according to claim 1, wherein the loss function of the no-reference image quality evaluation model based on image feature fusion comprises:
wherein N represents the number of natural distortion images, F i (I D ,I G ) Representing the quality assessment score, I, of the ith naturally distorted image D Representing a naturally distorted image domain, I G Representing the gradient image field, Q i The true label representing the ith naturally distorted image, loss represents the loss function.
7. A computer-readable storage medium, on which a computer program is stored, wherein the computer program is executed by a processor to implement the method for evaluating quality of a no-reference image based on image feature fusion according to any one of claims 1 to 7.
8. A no-reference image quality evaluation device based on image feature fusion is characterized by comprising a processor and a memory; the memory is used for storing a computer program; the processor is connected to the memory and configured to execute the computer program stored in the memory, so as to enable the apparatus for evaluating quality of non-reference images based on image feature fusion to perform the method for evaluating quality of non-reference images based on image feature fusion according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210839006.2A CN115222635A (en) | 2022-07-18 | 2022-07-18 | No-reference image quality evaluation method and device based on image feature fusion and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210839006.2A CN115222635A (en) | 2022-07-18 | 2022-07-18 | No-reference image quality evaluation method and device based on image feature fusion and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115222635A true CN115222635A (en) | 2022-10-21 |
Family
ID=83612404
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210839006.2A Pending CN115222635A (en) | 2022-07-18 | 2022-07-18 | No-reference image quality evaluation method and device based on image feature fusion and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115222635A (en) |
-
2022
- 2022-07-18 CN CN202210839006.2A patent/CN115222635A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108986050B (en) | Image and video enhancement method based on multi-branch convolutional neural network | |
| CN108520504B (en) | End-to-end blurred image blind restoration method based on generation countermeasure network | |
| Shi et al. | Low-light image enhancement algorithm based on retinex and generative adversarial network | |
| CN110634116B (en) | Facial image scoring method and camera | |
| Liu et al. | Learning hadamard-product-propagation for image dehazing and beyond | |
| CN111047543A (en) | Image enhancement method, device and storage medium | |
| CN114387365B (en) | Method and device for coloring line manuscript | |
| Saleh et al. | Adaptive uncertainty distribution in deep learning for unsupervised underwater image enhancement | |
| Zheng et al. | T-net: Deep stacked scale-iteration network for image dehazing | |
| CN112330613B (en) | Evaluation method and system for cytopathology digital image quality | |
| CN115731597A (en) | Automatic segmentation and restoration management platform and method for mask image of face mask | |
| CN112085717A (en) | Video prediction method and system for laparoscopic surgery | |
| CN113658091A (en) | Image evaluation method, storage medium and terminal equipment | |
| CN116091823A (en) | Single-feature anchor-frame-free target detection method based on fast grouping residual error module | |
| Wang et al. | Single underwater image enhancement using an analysis-synthesis network | |
| CN118172283A (en) | Marine target image defogging method based on improved gUNet model | |
| Peng et al. | Raune-Net: a residual and attention-driven underwater image enhancement method | |
| Zheng et al. | Overwater image dehazing via cycle-consistent generative adversarial network | |
| CN113781375A (en) | Vehicle-mounted vision enhancement method based on multi-exposure fusion | |
| Liu et al. | Dual UNet low-light image enhancement network based on attention mechanism | |
| CN116452469B (en) | Image defogging processing method and device based on deep learning | |
| CN117893843A (en) | Model training method, target detection device, and storage medium | |
| Du et al. | UIEDP: Underwater Image Enhancement with Diffusion Prior | |
| JP2021086284A (en) | Image processing device, image processing method, and program | |
| CN115222635A (en) | No-reference image quality evaluation method and device based on image feature fusion and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |