CN114445345A - Screen image quality evaluation method and related device - Google Patents

Screen image quality evaluation method and related device Download PDF

Info

Publication number
CN114445345A
CN114445345A CN202111640793.XA CN202111640793A CN114445345A CN 114445345 A CN114445345 A CN 114445345A CN 202111640793 A CN202111640793 A CN 202111640793A CN 114445345 A CN114445345 A CN 114445345A
Authority
CN
China
Prior art keywords
image
distorted
visual
target
representing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111640793.XA
Other languages
Chinese (zh)
Inventor
王革委
陈丽君
廖婷婷
袁畅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianyi Cloud Technology Co Ltd
Original Assignee
Tianyi Cloud Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianyi Cloud Technology Co Ltd filed Critical Tianyi Cloud Technology Co Ltd
Priority to CN202111640793.XA priority Critical patent/CN114445345A/en
Publication of CN114445345A publication Critical patent/CN114445345A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30168—Image quality inspection

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Image Processing (AREA)

Abstract

The application relates to the technical field of image processing, and provides a screen image quality evaluation method and a related device, which are used for solving the problem of how to simply and efficiently evaluate the screen image quality in the prior art. According to the method, the display characteristics of the display equipment are simulated through the distortion channel, the target image and the distortion image are modeled through the human eye visual perception channel model, a first visual image simulating the target image seen by human eyes and a second visual image simulating the distortion image are obtained, and the image quality of the target image displayed on the display equipment is determined according to the difference between the first visual image and the second visual image. Therefore, the situation that the human eyes observe the subjective perception dequantization image quality of the reference image relative to the target image is simulated by calculating the difference between the first visual image and the second visual image, and the efficiency of evaluating the image quality is improved. The problem that an objective quality evaluation method is not suitable for quality evaluation of screen display images is solved by determining the image quality of a target image after the target image is displayed by a display device.

Description

Screen image quality evaluation method and related device
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method and a related apparatus for evaluating screen image quality.
Background
The screen content image is a composite image that contains both computer text, computer graphics, and natural images. With the rapid development of the internet and cloud computing industry, distance education, video conferences, online virtual screen sharing and various cloud applications are widely popularized, and most of images in the applications belong to screen content images, so that a method for accurately measuring the quality of the screen content images becomes very important.
In the related art, there are two main image quality evaluation methods, namely, a subjective quality evaluation method and an objective quality evaluation method. The subjective evaluation method refers to that the image is directly observed through human eyes, and the image quality condition is dequantized based on the subjective perception of human beings on image information. The accuracy of the evaluation method is high, but the evaluation method is time-consuming, labor-consuming and tedious and is difficult to be applied in practice. The objective quality evaluation method is used for simulating the visual perception of human eyes to quantify the image quality by establishing an evaluation model. The method is simple to operate, low in cost and easy to practically apply. But the method is designed primarily for natural images and not for screen content images. Therefore, how to simply and efficiently evaluate the quality of the screen image is urgently needed to be solved.
Disclosure of Invention
The application aims to provide a screen image quality evaluation method which is used for solving the problem of how to simply and efficiently evaluate the screen image quality in the prior art.
In a first aspect, an embodiment of the present application provides a method for evaluating screen image quality, where the method includes:
acquiring a target image;
acquiring a distorted image of the target image according to a distorted channel; the distortion channel is obtained by modeling a display device;
respectively obtaining a first visual image of the target image and a second visual image of the distorted image based on a human eye visual perception channel model;
determining the image quality of the target image after being displayed by the display device based on the difference between the first visual image and the second visual image.
Optionally, the determining, based on a difference between the first visual image and the second visual image, the image quality of the target image after being displayed by the display device includes:
determining first image mutual information of the target image and the first visual image and second image mutual information of the target image and the second visual image;
determining the visual information fidelity of the distorted image relative to the target image according to the first image mutual information and the second image mutual information;
determining respective gradient information of the target image and the distorted image, and determining the gradient similarity of the target image and the distorted image according to the gradient information;
and performing pooling operation on the distorted image according to the visual information fidelity and the gradient similarity to obtain the quality score of the distorted image.
Optionally, the obtaining a distorted image of the target image according to a distorted channel includes:
dividing the target image into n x n image blocks;
respectively constructing vector expressions of the image blocks based on pixel values of the image blocks;
and processing the vector expression of each image block based on the distortion channel to obtain the distortion image.
Optionally, the constructing vector expressions of the image blocks based on the pixel values of the image blocks respectively includes:
aiming at any image block, acquiring a pixel value of the image block and constructing a pixel value vector;
and processing the pixel value vectors by adopting a Gaussian mixture model to obtain the vector expression of the image blocks, wherein the Gaussian mixture model is used for converting the pixel value vectors of the image blocks into normal distribution vectors with covariance as specified covariance.
Optionally, the processing the vector expression of each image block based on the distorted channel to obtain the distorted image includes:
processing the vector expression of each image block based on the following distorted image model to obtain the distorted image:
d＝gi×c′i+Vi
where d represents the vector representation of the distorted image of the image block, c'iVector representation of the ith image block in frequency domain, giScalar noise gain parameter, V, representing the ith image blockiAnd represents independent additive zero-mean Gaussian white noise of the ith image block.
Optionally, the human eye visual perception channel model is as follows:
Figure BDA0003443340840000031
wherein Q isiRepresenting the i-th image block of the input, P representing the corresponding image block QiA corresponding image block in the visual image; if QiRepresenting the ith image block in the target image, and then P represents the ith image block in the first visual image; if QiRepresenting the ith image block in the distorted image, then P represents the ith image block in the second visual image.
Optionally, the first image mutual information and the second image mutual information include mutual information of each image block, and determining the visual information fidelity of the distorted image relative to the target image according to the first image mutual information and the second image mutual information includes:
determining the visual information fidelity based on the following formula:
Figure BDA0003443340840000032
wherein PSM represents the fidelity of the visual information, M represents the number of image blocks divided by the target image, and fiRepresenting mutual information of the ith image block in a distorted image, eiRepresenting mutual information of the ith image block in the target image.
Optionally, the determining the gradient information of the target image and the gradient information of the distorted image, and determining the gradient similarity between the target image and the distorted image according to the gradient information includes:
determining a gradient similarity of the target image and the distorted image based on the following formula:
Figure BDA0003443340840000033
wherein G isr＝r*oh*ov，＝d*oh*ovR represents the target image, d represents the distorted image, o represents a convolution operationhRepresenting a gradient operator in the horizontal direction, ovRepresenting a vertical gradient operator, GrA gradient, G, representing the target imagedRepresenting the gradient of the distorted image, GSM representing the gradient similarity of the target image and the distorted image, and epsilon representing a preset parameter.
Optionally, the performing a pooling operation on the distorted image according to the visual information fidelity and the gradient similarity to obtain a quality score of the distorted image includes:
calculating a quality score of the distorted image based on the following formula:
Figure BDA0003443340840000041
among them, GSMiRepresenting the gradient similarity, PSM, of said image patchesiRepresenting the fidelity of the visual information of the image block, wherein K represents the total number of pixel points;
the quality score has a positive correlation with the quality of the target image.
In a second aspect, the present application provides a screen image quality evaluation apparatus, including:
an image acquisition module configured to perform acquiring a target image;
a distortion module configured to perform acquiring a distorted image of the target image according to a distortion channel; the distortion channel is obtained by modeling a display device;
a human visual perception channel module configured to perform deriving a first visual image of the target image and a second visual image of the distorted image based on a human visual perception channel model, respectively, on the target image and the distorted image;
a quality assessment module configured to perform determining an image quality of the target image after being displayed by the display device based on a difference of the first visual image and the second visual image.
Optionally, the determining the image quality of the target image after being displayed by the display device based on the difference between the first visual image and the second visual image is performed, and the quality evaluation module is configured to perform:
determining first image mutual information of the target image and the first visual image and second image mutual information of the target image and the second visual image;
determining the visual information fidelity of the distorted image relative to the target image according to the first image mutual information and the second image mutual information;
determining respective gradient information of the target image and the distorted image, and determining the gradient similarity of the target image and the distorted image according to the gradient information;
and performing pooling operation on the distorted image according to the visual information fidelity and the gradient similarity to obtain the quality score of the distorted image.
Optionally, the obtaining a distorted image of the target image according to a distorted channel is performed, and the distortion module is configured to perform:
dividing the target image into n x n image blocks;
respectively constructing vector expressions of the image blocks based on pixel values of the image blocks;
and processing the vector expression of each image block based on the distortion channel to obtain the distortion image.
Optionally, the step of constructing a vector representation of each image block based on the pixel values of each image block is performed, and the distortion module is configured to perform:
aiming at any image block, acquiring a pixel value of the image block and constructing a pixel value vector;
and processing the pixel value vectors by adopting a Gaussian mixture model to obtain the vector expression of the image blocks, wherein the Gaussian mixture model is used for converting the pixel value vectors of the image blocks into normal distribution vectors with covariance as specified covariance.
Optionally, the vector representation of each image block is processed based on the distorted channel, so as to obtain the distorted image, and the distortion module is configured to perform:
processing the vector expression of each image block based on the following distorted image model to obtain the distorted image:
d＝gi×c′i+Vi
wherein d represents a vector of a distorted image of the image block, c'iVector representation of the ith image block in frequency domain, giScalar noise gain parameter, V, representing the ith image blockiAnd represents independent additive zero-mean Gaussian white noise of the ith image block.
Optionally, the human eye visual perception channel model is as follows:
Figure BDA0003443340840000051
wherein Q isiRepresenting the i-th image block of the input, P representing the corresponding image block QiA corresponding image block in the visual image; if QiRepresenting the ith image block in the target image, and then P represents the ith image block in the first visual image; if QiRepresenting the ith image block in the distorted image, then P represents the ith image block in the second visual image.
Optionally, the first image mutual information and the second image mutual information include mutual information of each image block, the determining of the visual information fidelity of the distorted image relative to the target image according to the first image mutual information and the second image mutual information is performed, and the visual information fidelity module is configured to perform:
determining the visual information fidelity based on the following formula:
Figure BDA0003443340840000061
wherein PSM represents the fidelity of the visual information, M represents the number of image blocks divided by the target image, and fiRepresenting mutual information of the ith image block in a distorted image, eiRepresenting mutual information of the ith image block in the target image.
Optionally, the determining of the gradient information of the target image and the gradient information of the distorted image are performed, and the gradient similarity between the target image and the distorted image is determined according to the gradient information, where the gradient similarity module is configured to perform:
determining a gradient similarity of the target image and the distorted image based on the following formula:
Figure BDA0003443340840000062
wherein G isr＝r*oh*ov，Gd＝d*oh*ovR represents the target image, d represents the distorted image, o represents a convolution operationhRepresenting a gradient operator in the horizontal direction, ovRepresenting a vertical gradient operator, GrA gradient, G, representing the target imagedRepresenting the gradient of the distorted image, GSM representing the gradient similarity of the target image and the distorted image, and epsilon representing a preset parameter.
Optionally, performing the pooling operation on the distorted image according to the visual information fidelity and the gradient similarity to obtain a quality score of the distorted image, where the quality evaluation module 404 is configured to perform:
calculating a quality score of the distorted image based on the following formula:
Figure BDA0003443340840000071
among them, GSMiRepresenting the gradient similarity, PSM, of said image patchesiRepresenting the fidelity of the visual information of the image block, wherein K represents the total number of pixel points;
the quality score has a positive correlation with the quality of the target image.
In a third aspect, the present application further provides an electronic device, including:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to carry out the steps of the method according to any one of the first aspect.
In a fourth aspect, the present application also provides a computer-readable storage medium, in which instructions, when executed by a processor of an electronic device, enable the electronic device to perform the steps of the method according to any one of the first aspect.
In a fifth aspect, the present application further provides a computer program product comprising a computer program which, when executed by a processor, performs the steps of the method according to any one of the first aspect.
The technical scheme provided by the embodiment of the application at least has the following beneficial effects:
according to the method, the display characteristics of the display equipment are simulated through the distortion channel, the target image and the distortion image are modeled through the human eye vision perception channel model, the first visual image simulating the target image seen by human eyes and the second visual image simulating the distortion image are obtained, and the difference between the first visual image and the second visual image is calculated to determine the image quality of the target image after the target image is displayed through the display equipment. Therefore, the condition that the quality of the image is subjectively perceived and dequantized by human eyes of a reference image relative to a target image is simulated by calculating the difference between the first visual image and the second visual image, and the efficiency of evaluating the quality of the image is improved. By determining the image quality of the target image after being displayed by the display equipment, the problem that the objective quality evaluation method is not suitable for quality evaluation of the display screen image is solved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments of the present application will be briefly described below, and it is obvious that the drawings described below are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic diagram of an apparatus provided in an embodiment of the present application;
FIG. 2 is a schematic diagram illustrating a screen image quality evaluation process according to an embodiment of the present disclosure;
FIG. 3 is a second schematic view illustrating a screen image quality evaluation process according to an embodiment of the present application;
FIG. 4 is a diagram of a screen image quality evaluation apparatus according to an embodiment of the present application;
fig. 5 is a schematic view of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described in detail and clearly with reference to the accompanying drawings. In the description of the embodiments of the present application, "/" means "or" unless otherwise specified, for example, a/B may mean a or B; "and/or" in the text is only an association relationship describing an associated object, and means that three relationships may exist, for example, a and/or B may mean: three cases of a alone, a and B both, and B alone exist, and in addition, "a plurality" means two or more than two in the description of the embodiments of the present application.
In the description of the embodiments of the present application, the term "plurality" means two or more unless otherwise specified, and other terms and the like should be understood similarly, and the preferred embodiments described herein are only for the purpose of illustrating and explaining the present application, and are not intended to limit the present application, and features in the embodiments and examples of the present application may be combined with each other without conflict.
Hereinafter, some terms in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.
Gaussian Mixture Model (GMM): things are accurately quantified by a gaussian probability density function (normal distribution curve), which is a model formed by decomposing things into a plurality of things based on the gaussian probability density function (normal distribution curve).
Human Visual perception channel model (Human Visual System, HVS): the system is a system for forming vision by human beings through sensing light, and is one of the most critical systems for capturing external information by human beings.
Visual Information Fidelity (VIF): the method is a mode for judging the image quality based on a natural scene statistical model, image distortion and a human eye visual perception channel model.
The screen content image is a composite image that contains both computer text, computer graphics, and natural images. With the rapid development of the internet and cloud computing industry, distance education, video conferences, online virtual screen sharing and various cloud applications are widely popularized, and most of images in the applications belong to screen content images, so that a method for accurately measuring the quality of the screen content images becomes very important.
In the related art, there are two main methods for evaluating image quality, which are a subjective quality evaluation method and an objective quality evaluation method, respectively. The subjective evaluation method refers to that the image is directly observed through human eyes, and the image quality condition is dequantized based on the subjective perception of human beings on image information. The accuracy of the evaluation method is high, but the evaluation method is time-consuming, labor-consuming and tedious and is difficult to be applied in practice. The objective quality evaluation method is used for simulating the visual perception of human eyes to quantify the image quality by establishing an evaluation model. The method is simple to operate, low in cost and easy to practically apply. But the method is designed primarily for natural images and not for screen content images. Therefore, how to simply and efficiently evaluate the quality of the screen image is urgently needed to be solved.
In the embodiment of the application, the distortion channel simulates the display characteristics of the display device, the target image and the distortion image are modeled through the human eye visual perception channel model, a first visual image simulating the target image seen by human eyes and a second visual image simulating the distortion image are obtained, and the difference between the first visual image and the second visual image is calculated to determine the image quality of the target image after being displayed by the display device. Therefore, the difference between the first visual image and the second visual image is calculated to replace the situation that the human eyes observe the reference image to subjectively perceive and dequantize the image quality relative to the target image, and the efficiency of evaluating the image quality is improved. By determining the image quality of the target image after being displayed by the display equipment, the problem that the objective quality evaluation method is not suitable for quality evaluation of the display screen image is solved.
To further illustrate the technical solutions provided by the embodiments of the present application, the following detailed description is made with reference to the accompanying drawings and the detailed description. Although the embodiments of the present application provide method steps as shown in the following embodiments or figures, more or fewer steps may be included in the method based on conventional or non-inventive efforts. In steps where no necessary causal relationship exists logically, the order of execution of the steps is not limited to that provided by the embodiments of the present application. The method can be executed in the order of the embodiments or the method shown in the drawings or in parallel in the actual process or the control device.
Fig. 1 is a schematic diagram of a related apparatus according to an embodiment of the present disclosure.
After the target image r is obtained, the target image r ' obtained after modeling through the GMM model is modeled through a distortion channel to obtain a distortion image d, and the target image r ' and the distortion image d are subjected to an HVS model to obtain a first visual image of the target image r ' and a second visual image of the target image d. And (4) the target image r', the distorted image d, the first visual image and the second visual image pass through a mutual information extraction model in the VIF model, and the mutual information of the target image and the first image of the first visual image and the mutual information of the distorted image and the second image of the second visual image are calculated. And the VIF model calculates the visual information fidelity of the distorted image relative to the target image according to the first image mutual information and the second image mutual information.
And simultaneously inputting the target image r and the distorted image d into a gradient information extraction model to respectively obtain the gradient of the target image and the gradient of the distorted image, then calculating the gradient of the target image and the gradient of the distorted image through a gradient similarity model to obtain the gradient similarity of the distorted image and the target image, and finally inputting the gradient similarity and the visual information fidelity to a quality evaluation model to obtain the quality score of the distorted image relative to the target image.
For ease of understanding, a screen image quality evaluation method provided by the embodiment of the present application is further described below according to the steps shown in fig. 2.
Step 201, acquiring a target image.
The target image in the embodiment of the present application may be the original digital image content input to the screen display. The target image may be web page content, video conference content, video document content, or the like.
In the embodiment of the application, in order to facilitate the evaluation of the image quality and improve the accuracy of the evaluation, the obtained target image is divided into n × n image blocks. And taking the pixel values of all pixel points in the image block as a vector expression c of the image block. If the image is a gray image, directly taking the gray value of the pixel point as the pixel value of the pixel point; and if the image is a color image, converting the color image into a gray image, and taking the gray value of the gray image as the pixel value of the pixel point.
Step 202, obtaining a distorted image of the target image according to a distorted channel, wherein the distorted channel is obtained by modeling the display device.
In the embodiment of the present application, the image data is time domain data, and before obtaining the distorted image of the image block through the distorted channel, the vector representation c of the image block may be first converted from the time domain to the frequency domain c' according to GMM through formula (1).
c′＝u………(1)
In the formula (1), u is covariance CuThe term "normal distribution vector" means a distribution in which the vector expression c is distributed according to a normal distribution vector.
In the embodiment of the application, because the image is inevitably lost in the transmission and acquisition processes, in order to make the image as close as possible to the image after the loss, the target image r is displayed on the screen of the relevant device after being transmitted to the relevant device. The image displayed on the screen can generate distortion phenomenon due to loss in the transmission process, so that the parameter g of the distortion channel obtained by modeling about the distortion channel in formula (2) can be calculated according to the target image and the image displayed on the screeniAnd Vi。
After obtaining the vector expression c 'of the image block in the frequency domain, inputting c' into a distortion channel according to formula (2), and obtaining the vector expression d of the distortion image of the image block.
d＝gi×c′i+Vi………(2)
In equation (2), d represents a vector expression of a distorted image of the image block, c'iVector representation of the ith image block in frequency domain, giScalar noise gain parameter, V, representing the ith image blockiAnd represents independent additive zero-mean Gaussian white noise of the ith image block.
And 203, respectively obtaining a first visual image of the target image and a second visual image of the distorted image based on the human eye visual perception channel model.
In the embodiment of the present application, formula (3) is a channel model perceived by human vision provided in the embodiment of the present application.
Figure BDA0003443340840000111
In the formula (2), QiRepresenting the i-th image block of the input, P representing the corresponding image block QiA corresponding image block in the visual image; if QiIf the image block represents the ith image block in the target image, taking P as the ith image block in the first visual image; if QiRepresenting the ith image block in the distorted image, P is taken as the ith image block in the second visual image.
And step 204, determining the image quality of the target image after being displayed by the display device based on the difference between the first visual image and the second visual image.
The operation of determining the quality of the screen display image may be as shown in fig. 3, including the steps of:
step 301, determining the mutual information of the target image and the first image of the first visual image, and the mutual information of the target image and the second image of the second visual image.
In the embodiment of the present application, the calculation mode of the image mutual information is shown in formula (4).
Figure BDA0003443340840000121
Figure BDA0003443340840000122
In the formula (4), siRepresenting the i-th image block in the target image, fiRepresenting the ith image block in the second visual image, eiRepresenting the ith image block in the first visual image, I representing the identity matrix, σnAnd σvRepresents the fitting parameter, c'iA vector representation representing the image block converted into the frequency domain.
And step 302, determining the visual information fidelity of the distorted image relative to the target image according to the first image mutual information and the second image mutual information.
In the embodiment of the application, the first image mutual information and the second image mutual information include mutual information of each image block, and after the first image mutual information and the second image mutual information are obtained, visual information fidelity of a distorted image relative to a target image is calculated in a VIF module according to a formula (5).
Figure BDA0003443340840000123
In formula (5), PSM represents the fidelity of visual information, M represents the number of image blocks divided from the target image, and fiRepresenting mutual information of the ith image block in a distorted image, eiRepresenting mutual information of the ith image block in the target image.
The value of PSM in formula (5) is between [0,1], and the closer the value of PSM is to 1, the closer the distorted image is to the target image.
Step 303, determining respective gradient information of the target image and the distorted image, and determining the gradient similarity of the target image and the distorted image according to the gradient information.
In the embodiment of the present application, the gradient information of each of the target image and the distorted image is calculated according to formula (6).
Gr＝r*oh*ov，Gd＝d*oh*ov………(6)
In equation (6), r represents the target image, d represents the distorted image, a represents the convolution operation, ohRepresenting a gradient operator in the horizontal direction, ovRepresenting a vertical gradient operator, GrA gradient, G, representing the target imagedRepresenting the gradient of the distorted image.
In the examples of the present application, ovAnd ohCan be set empirically. For example, in the present application,
Figure BDA0003443340840000131
the o can also be modified if gradient information is desired that is different from that in the present applicationvAnd ohOf the matrix of (a) or change ovAnd ohMatrix size of, this applicationThis is not limiting.
After the gradient information of the target image and the distorted image is calculated, the gradient similarity of the target image and the distorted image is determined based on formula (7) in a gradient information extraction model.
Figure BDA0003443340840000132
In formula (7), ε represents a preset parameter.
And step 304, performing pooling operation on the distorted image according to the visual information fidelity and the gradient similarity to obtain the quality score of the distorted image.
In the embodiment of the application, the quality of the target image is judged according to the quality score based on the quality score of the distorted image in the formula (8), wherein the quality score has a positive correlation with the quality of the target image.
Calculating a quality score of the distorted image based on the following formula:
Figure BDA0003443340840000133
in equation (8), GSMiGradient similarity, PSM, representing image patchesiThe fidelity of the visual information of the image block is represented, and K represents the total number of the pixel points.
In the application, the distortion condition at the edge of the image can be considered by calculating the gradient similarity, and the gradient of the image can sensitively reflect the distortion degree of the image. The visual effects of the target image and the reference image in human eyes can be simulated through the VIF model, and the quality score of the distorted image is calculated based on the obtained PSM value and the GSM value, so that the visual information fidelity of the distorted image relative to the target image can be obtained in real time through the simulation of the human eyes, and the distorted image can be more accurately evaluated according to the visual information fidelity and the gradient similarity.
Based on the same inventive concept, the present application also provides a screen image quality evaluation apparatus 400, as shown in fig. 4, comprising:
an image acquisition module 401 configured to perform acquisition of a target image;
a distortion module 402 configured to perform acquiring a distorted image of the target image according to a distortion channel; the distortion channel is obtained by modeling a display device;
a human visual perception channel module 403 configured to perform deriving a first visual image of the target image and a second visual image of the distorted image based on a human visual perception channel model, respectively, on the target image and the distorted image;
a quality assessment module 404 configured to perform determining an image quality of the target image after being displayed by the display device based on a difference of the first visual image and the second visual image.
Optionally, the determining the image quality of the target image after being displayed by the display device based on the difference between the first visual image and the second visual image is performed, and the quality evaluation module 404 is configured to perform:
determining first image mutual information of the target image and the first visual image and second image mutual information of the target image and the second visual image;
determining the visual information fidelity of the distorted image relative to the target image according to the first image mutual information and the second image mutual information;
determining respective gradient information of the target image and the distorted image, and determining the gradient similarity of the target image and the distorted image according to the gradient information;
and performing pooling operation on the distorted image according to the visual information fidelity and the gradient similarity to obtain the quality score of the distorted image.
Optionally, executing the obtaining of the distorted image of the target image according to the distorted channel, the distortion module 402 is configured to execute:
dividing the target image into n x n image blocks;
respectively constructing vector expressions of the image blocks based on pixel values of the image blocks;
and processing the vector expression of each image block based on the distortion channel to obtain the distortion image.
Optionally, the step of separately constructing a vector representation of each image block based on the pixel values of each image block is performed, and the distortion module 402 is configured to perform:
aiming at any image block, acquiring a pixel value of the image block and constructing a pixel value vector;
and processing the pixel value vectors by adopting a Gaussian mixture model to obtain the vector expression of the image blocks, wherein the Gaussian mixture model is used for converting the pixel value vectors of the image blocks into normal distribution vectors with covariance as specified covariance.
Optionally, the vector representation of each image block is processed based on the distortion channel to obtain the distorted image, and the distortion module 402 is configured to perform:
processing the vector expression of each image block based on the following distorted image model to obtain the distorted image:
d＝gi×c′i+Vi
wherein d represents a vector of a distorted image of the image block, c'iVector representation of the ith image block in frequency domain, giScalar noise gain parameter, V, representing the ith image blockiAnd represents independent additive zero-mean Gaussian white noise of the ith image block.
Optionally, the human eye visual perception channel model is as follows:
Figure BDA0003443340840000151
wherein Q isiRepresenting the i-th image block of the input, P representing the corresponding image block QiA corresponding image block in the visual image; if QiRepresenting the ith image block in the target image, P represents the first viewDetecting the ith image block in the image; if QiRepresenting the ith image block in the distorted image, then P represents the ith image block in the second visual image.
Optionally, the first image mutual information and the second image mutual information include mutual information of each image block, the determining, according to the first image mutual information and the second image mutual information, a visual information fidelity of the distorted image with respect to the target image is performed, and the visual information fidelity module 405 is configured to perform:
determining the visual information fidelity based on the following formula:
Figure BDA0003443340840000161
wherein PSM represents the fidelity of the visual information, M represents the number of image blocks divided by the target image, and fiRepresenting mutual information of the ith image block in a distorted image, eiRepresenting mutual information of the ith image block in the target image.
Optionally, the determining of the gradient information of the target image and the gradient information of the distorted image are performed, and the gradient similarity between the target image and the distorted image is determined according to the gradient information, and the gradient similarity module 406 is configured to perform:
determining a gradient similarity of the target image and the distorted image based on the following formula:
Figure BDA0003443340840000162
wherein G isr＝r*oh*ov，Gd＝d*oh*ovR represents the target image, d represents the distorted image, represents a convolution operation, ohRepresenting a gradient operator in the horizontal direction, ovRepresenting a vertical gradient operator, GrA gradient, G, representing the target imagedRepresenting the gradient of the distorted image, GSM representing the target image and the stationAnd the gradient similarity of the distorted image, wherein epsilon represents a preset parameter.
Optionally, performing the pooling operation on the distorted image according to the visual information fidelity and the gradient similarity to obtain a quality score of the distorted image, where the quality evaluation module 404 is configured to perform:
calculating a quality score of the distorted image based on the following formula:
Figure BDA0003443340840000171
among them, GSMiRepresenting the gradient similarity, PSM, of said image patchesiRepresenting the fidelity of the visual information of the image block, wherein K represents the total number of pixel points;
the quality score has a positive correlation with the quality of the target image.
Having described the abnormal event detection method and the electronic apparatus according to the exemplary embodiments of the present application, next, an electronic apparatus according to another exemplary embodiment of the present application will be described.
As will be appreciated by one skilled in the art, aspects of the present application may be embodied as a system, method or program product. Accordingly, various aspects of the present application may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.
In some possible implementations, an electronic device according to the present application may include at least one processor, and at least one memory. The memory stores therein program code which, when executed by the processor, causes the processor to perform the steps of the method for searching for a monitoring node according to various exemplary embodiments of the present application described above in the present specification. For example, the processor may perform steps in a search method such as monitoring nodes.
The electronic device 130 according to this embodiment of the present application is described below with reference to fig. 5. The electronic device 130 shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 5, the electronic device 130 is represented in the form of a general electronic device. The components of the electronic device 130 may include, but are not limited to: the at least one processor 131, the at least one memory 132, and a bus 133 that connects the various system components (including the memory 132 and the processor 131).
Bus 133 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.
The memory 132 may include readable media in the form of volatile memory, such as Random Access Memory (RAM)1321 and/or cache memory 1322, and may further include Read Only Memory (ROM) 1323.
Memory 132 may also include a program/utility 1325 having a set (at least one) of program modules 1324, such program modules 1324 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 130 may also communicate with one or more external devices 134 (e.g., keyboard, pointing device, etc.), with one or more devices that enable a user to interact with the electronic device 130, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 130 to communicate with one or more other electronic devices. Such communication may occur via input/output (I/O) interfaces 135. Also, the electronic device 130 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via the network adapter 136. As shown, network adapter 136 communicates with other modules for electronic device 130 over bus 133. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with electronic device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
In some possible embodiments, the aspects of a search method for a monitoring node provided by the present application may also be implemented in the form of a program product including program code for causing a computer device to perform the steps in a monitoring according to various exemplary embodiments of the present application described above in this specification when the program product is run on the computer device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The program product for monitoring of the embodiments of the present application may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on an electronic device. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the consumer electronic device, partly on the consumer electronic device, as a stand-alone software package, partly on the consumer electronic device and partly on a remote electronic device, or entirely on the remote electronic device or server. In the case of remote electronic devices, the remote electronic devices may be connected to the consumer electronic device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external electronic device (e.g., through the internet using an internet service provider).
It should be noted that although several units or sub-units of the apparatus are mentioned in the above detailed description, such division is merely exemplary and not mandatory. Indeed, the features and functions of two or more units described above may be embodied in one unit, according to embodiments of the application. Conversely, the features and functions of one unit described above may be further divided into embodiments by a plurality of units.
Further, while the operations of the methods of the present application are depicted in the drawings in a particular order, this does not require or imply that these operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and block diagrams, and combinations of flows and blocks in the flow diagrams and block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and block diagram block or blocks.
While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims (10)

1. A screen image quality evaluation method, characterized by comprising:
acquiring a target image;
acquiring a distorted image of the target image according to a distorted channel; the distortion channel is obtained by modeling a display device;
respectively obtaining a first visual image of the target image and a second visual image of the distorted image based on a human eye visual perception channel model;
determining the image quality of the target image after being displayed by the display device based on the difference between the first visual image and the second visual image.
2. The method of claim 1, wherein determining the image quality of the target image after being displayed by the display device based on the difference between the first visual image and the second visual image comprises:
determining first image mutual information of the target image and the first visual image and second image mutual information of the target image and the second visual image;
determining the visual information fidelity of the distorted image relative to the target image according to the first image mutual information and the second image mutual information;
determining respective gradient information of the target image and the distorted image, and determining the gradient similarity of the target image and the distorted image according to the gradient information;
and performing pooling operation on the distorted image according to the visual information fidelity and the gradient similarity to obtain the quality score of the distorted image.
3. The method of claim 1, wherein the obtaining a distorted image of the target image according to a distorted channel comprises:
dividing the target image into n x n image blocks;
respectively constructing vector expressions of the image blocks based on pixel values of the image blocks;
and processing the vector expression of each image block based on the distortion channel to obtain the distortion image.
4. The method according to claim 3, wherein the separately constructing a vector representation for each image block based on pixel values of each image block comprises:
aiming at any image block, acquiring a pixel value of the image block and constructing a pixel value vector;
and processing the pixel value vectors by adopting a Gaussian mixture model to obtain the vector expression of the image blocks, wherein the Gaussian mixture model is used for converting the pixel value vectors of the image blocks into normal distribution vectors with covariance as specified covariance.
5. The method of claim 3, wherein processing the vector representation for each image block based on the distorted channel to obtain the distorted image comprises:
processing the vector expression of each image block based on the following distorted image model to obtain the distorted image:
d＝gi×c′i+Vi
where d represents the vector representation of the distorted image of the image block, c'iVector representation of the ith image block in frequency domain, giScalar noise gain parameter, V, representing the ith image blockiAnd represents independent additive zero-mean Gaussian white noise of the ith image block.
6. The method of claim 1, wherein the human eye visual perception channel model is:
Figure FDA0003443340830000021
wherein Q isiRepresenting the i-th image block of the input, P representing the corresponding image block QiA corresponding image block in the visual image; if QiRepresenting the ith image block in the target image, and then P represents the ith image block in the first visual image; if QiRepresenting the ith image block in the distorted image and P representing the ith image block in the second visual image.
7. The method of claim 2, wherein the first image mutual information and the second image mutual information comprise mutual information of each image block, and wherein determining the visual information fidelity of the distorted image relative to the target image according to the first image mutual information and the second image mutual information comprises:
determining the visual information fidelity based on the following formula:
Figure FDA0003443340830000022
wherein PSM represents the fidelity of the visual information, M represents the number of image blocks divided by the target image, and fiRepresenting mutual information of the ith image block in a distorted image, eiRepresenting the ith image in the target imageMutual information of the blocks.
8. The method of claim 7, wherein determining respective gradient information of the target image and the distorted image and determining a gradient similarity of the target image and the distorted image according to the gradient information comprises:
determining a gradient similarity of the target image and the distorted image based on the following formula:
Figure FDA0003443340830000031
wherein G isr＝r*oh*ov，Gd＝d*oh*ovR represents the target image, d represents the distorted image, represents a convolution operation, ohRepresenting a gradient operator in the horizontal direction, ovRepresenting a vertical gradient operator, GrA gradient, G, representing the target imagedRepresenting the gradient of the distorted image, GSM representing the gradient similarity of the target image and the distorted image, and epsilon representing a preset parameter.
9. The method of claim 8, wherein pooling the distorted image according to the visual information fidelity and the gradient similarity to obtain a quality score of the distorted image comprises:
calculating a quality score of the distorted image based on the following formula:
Figure FDA0003443340830000032
among them, GSMiRepresenting the gradient similarity, PSM, of said image patchesiRepresenting the fidelity of the visual information of the image block, wherein K represents the total number of pixel points;
the quality score has a positive correlation with the quality of the target image.
10. An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to carry out the steps of the method according to any one of claims 1 to 9.
CN202111640793.XA 2021-12-29 2021-12-29 Screen image quality evaluation method and related device Pending CN114445345A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111640793.XA CN114445345A (en) 2021-12-29 2021-12-29 Screen image quality evaluation method and related device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111640793.XA CN114445345A (en) 2021-12-29 2021-12-29 Screen image quality evaluation method and related device

Publications (1)

Publication Number Publication Date
CN114445345A true CN114445345A (en) 2022-05-06

Family

ID=81365762

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111640793.XA Pending CN114445345A (en) 2021-12-29 2021-12-29 Screen image quality evaluation method and related device

Country Status (1)

Country Link
CN (1) CN114445345A (en)

Similar Documents

Publication Publication Date Title
CN107507153B (en) Image denoising method and device
CN108197652B (en) Method and apparatus for generating information
CN114187624B (en) Image generation method, device, electronic equipment and storage medium
CN105282543A (en) Total blindness three-dimensional image quality objective evaluation method based on three-dimensional visual perception
EP3410714A1 (en) Video compressive sensing reconstruction method and system, and electronic apparatus and storage medium
KR20210118739A (en) Method and apparatus for processing information, electronic device, computer-readable storage media and computer program stored in media
CN113962965A (en) Image quality evaluation method, device, equipment and storage medium
CN114004700A (en) Service data processing method and device, electronic equipment and storage medium
CN113379877A (en) Face video generation method and device, electronic equipment and storage medium
Mereuta et al. Web page textual color contrast compensation for CVD users using optimization methods
CN113177466A (en) Identity recognition method and device based on face image, electronic equipment and medium
CN114842287B (en) Monocular three-dimensional target detection model training method and device of depth-guided deformer
US20220392251A1 (en) Method and apparatus for generating object model, electronic device and storage medium
Shen et al. No-reference stereoscopic 3D image quality assessment via combined model
CN114445345A (en) Screen image quality evaluation method and related device
CN116977195A (en) Method, device, equipment and storage medium for adjusting restoration model
EP3929866A2 (en) Inpainting method and apparatus for human image, and electronic device
CN109741250B (en) Image processing method and device, storage medium and electronic equipment
CN113255819A (en) Method and apparatus for identifying information
CN113573043B (en) Video noise point identification method, storage medium and equipment
CN114844889B (en) Video processing model updating method and device, electronic equipment and storage medium
CN114722061B (en) Data processing method and device, equipment and computer readable storage medium
US20240020348A1 (en) Measuring the readability of website pages
CN116597288B (en) Gaze point rendering method, gaze point rendering system, computer and readable storage medium
CN116168398B (en) Examination paper approval method, device and equipment based on image identification

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination